Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Citrine demo that is broken but downloads and runs all docker contain… #45

Merged
merged 10 commits into from
May 31, 2024

Conversation

shankari
Copy link
Collaborator

…ers successfully

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@louisg1337 please edit the existing demo script by specifying additional command line options instead of creating a new script. Also, please remember to sign your commits with -s, everest enforces DCO https://github.com/apps/dco

Copy link
Collaborator Author

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I try to run this without changes, I get

npm verb cli [
npm verb cli   '~/.nvm/versions/node/v13.12.0/bin/node',
npm verb cli   '~/.nvm/versions/node/v13.12.0/bin/npm',
npm verb cli   'run',
npm verb cli   'compile',
npm verb cli   '--workspaces',
npm verb cli   '--verbose'
npm verb cli ]
npm info using [email protected]
npm info using [email protected]
npm verb stack Error: missing script: compile
npm verb stack     at run (~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/lib/run-script.js:155:19)
npm verb stack     at ~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/lib/run-script.js:63:5
npm verb stack     at ~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:116:5
npm verb stack     at ~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:436:5
npm verb stack     at checkBinReferences_ (~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:391:45)
npm verb stack     at final (~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:434:3)
npm verb stack     at then (~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:161:5)
npm verb stack     at ~/.nvm/versions/node/v13.12.0/lib/node_modules/npm/node_modules/read-package-json/read-json.js:382:12
npm verb stack     at ~/versions/node/v13.12.0/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:115:16
npm verb stack     at FSReqCallback.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:63:3)
npm verb cwd /private/var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.TqHBaXyy/citrineos-csms
npm verb Darwin 22.6.0
npm verb argv "/Users/kshankar/Desktop/data/.nvm/versions/node/v13.12.0/bin/node" "/Users/kshankar/Desktop/data/.nvm/versions/node/v13.12.0/bin/npm" "run" "compile" "--workspaces" "--verbose"
npm verb node v13.12.0
npm verb npm  v6.14.4
npm ERR! missing script: compile
npm verb exit [ 1, true ]
npm timing npm Completed in 116ms

npm ERR! A complete log of this run can be found in:
npm ERR!     ~/.npm/_logs/2024-05-27T16_53_48_231Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @citrineos/[email protected] build: `npm run compile --workspaces --verbose`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @citrineos/[email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     ~/.npm/_logs/2024-05-27T16_53_48_263Z-debug.log
Error: Failed to build the project.


if [[ -f "$CITRINE_DOCKER" ]]; then
# Use sed to find and replace the string
sed -i '' 's/8082:8082/80:80/g' "$CITRINE_DOCKER"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@louisg1337 does this actually work? Are you able to connect to localhost:80 after this? I bet not. I would assume that you need 8082:80 instead. Please check the docker-compose docs!

@shankari
Copy link
Collaborator Author

The failure is because Citrine needs node 18, and I have node 13 installed by default. But for the one-line demo, we can't assume that the users have node 18 installed. We should also not upgrade their default version of node without letting them know. We should use a docker container for the citrine build as well, not just for the citrine dependencies.

@shankari
Copy link
Collaborator Author

@louisg1337 I switched this to port 8081 and am able to connect the station to the CSMS

2024-05-27 18:00:17.998582 [INFO] ocpp:OCPP201     :: OCPP client successfully connected to plain websocket server
2024-05-27 18:00:18.033248 [INFO] ocpp:OCPP201     :: libocpp: Updating OCSP cache on 0 certificates
2024-05-27 18:00:18.033443 [INFO] ocpp:OCPP201     :: libocpp: Done updating OCSP cache
2024-05-27 18:00:18.087564 [INFO] ocpp:OCPP201     :: Received BootNotificationResponse: {
    "currentTime": "2024-05-27T18:00:18.079Z",
    "interval": 60,
    "status": "Accepted"
}
with messageId: 6debe3d6-0e00-4d41-9277-7f7770924583

However, when I try to authenticate, the token is not registered. @louisg1337 can you see how to configure the token?

2024-05-27 18:08:23.748918 [INFO] auth:Auth        :: Received new token: {
    "authorization_type": "RFID",
    "connectors": [
        1
    ],
    "id_token": {
        "type": "ISO14443",
        "value": "DEADBEEF"
    },
    "prevalidated": false
}
2024-05-27 18:08:23.811961 [INFO] auth:Auth        :: Result for token: DEADBEEF: REJECTED

@shankari
Copy link
Collaborator Author

I interpret the instructions here #44 (comment) as using the "Monitoring" section of the API.

You can set this VariableAttribute on the CSMS side using the Variable Attribute CRUD endpoints on the Monitoring module. You can set this VariableAttribute on the Charging Station side using the SetVariables message, which can be sent from CitrineOS using the Monitoring module’s message API.

Screenshot 2024-05-27 at 11 38 12 AM

I used the Swagger API to make a GET for cp001 and got

curl -X 'GET' \
  'http://localhost:8080/data/monitoring/variableAttribute?stationId=cp001' \
  -H 'accept: */*'
And got this response
[
  {
    "id": 2,
    "stationId": "cp001",
    "type": "Actual",
    "dataType": "boolean",
    "value": "true",
    "mutability": "ReadOnly",
    "persistent": false,
    "constant": false,
    "variableId": 2,
    "componentId": 1,
    "evseDatabaseId": null,
    "createdAt": "2024-05-27T18:00:18.136Z",
    "updatedAt": "2024-05-27T18:00:18.136Z",
    "bootConfigId": null,
    "component": {
      "id": 1,
      "name": "ChargingStation",
      "instance": null,
      "evseDatabaseId": null,
      "createdAt": "2024-05-27T18:00:18.088Z",
      "updatedAt": "2024-05-27T18:00:18.088Z",
      "evse": null
    },
    "variable": {
      "id": 2,
      "name": "Available",
      "instance": null,
      "createdAt": "2024-05-27T18:00:18.132Z",
      "updatedAt": "2024-05-27T18:00:18.132Z",
      "variableCharacteristics": null
    },
    "statuses": []
  },
  {
    "id": 5,
    "stationId": "cp001",
    "type": "Actual",
    "dataType": null,
    "value": "",
    "mutability": "ReadOnly",
    "persistent": true,
    "constant": true,
    "variableId": 5,
    "componentId": 1,
    "evseDatabaseId": null,
    "createdAt": "2024-05-27T18:00:18.169Z",
    "updatedAt": "2024-05-27T18:00:18.169Z",
    "bootConfigId": null,
    "component": {
      "id": 1,
      "name": "ChargingStation",
      "instance": null,
      "evseDatabaseId": null,
      "createdAt": "2024-05-27T18:00:18.088Z",
      "updatedAt": "2024-05-27T18:00:18.088Z",
      "evse": null
    },
    "variable": {
      "id": 5,
      "name": "VendorName",
      "instance": null,
      "createdAt": "2024-05-27T18:00:18.164Z",
      "updatedAt": "2024-05-27T18:00:18.164Z",
      "variableCharacteristics": null
    },
    "statuses": []
  },
  {
    "id": 4,
    "stationId": "cp001",
    "type": "Actual",
    "dataType": null,
    "value": "",
    "mutability": "ReadOnly",
    "persistent": true,
    "constant": true,
    "variableId": 4,
    "componentId": 1,
    "evseDatabaseId": null,
    "createdAt": "2024-05-27T18:00:18.157Z",
    "updatedAt": "2024-05-27T18:00:18.157Z",
    "bootConfigId": null,
    "component": {
      "id": 1,
      "name": "ChargingStation",
      "instance": null,
      "evseDatabaseId": null,
      "createdAt": "2024-05-27T18:00:18.088Z",
      "updatedAt": "2024-05-27T18:00:18.088Z",
      "evse": null
    },
    "variable": {
      "id": 4,
      "name": "Model",
      "instance": null,
      "createdAt": "2024-05-27T18:00:18.149Z",
      "updatedAt": "2024-05-27T18:00:18.149Z",
      "variableCharacteristics": null
    },
    "statuses": []
  },
  {
    "id": 1,
    "stationId": "cp001",
    "type": "Actual",
    "dataType": "boolean",
    "value": "true",
    "mutability": "ReadOnly",
    "persistent": false,
    "constant": false,
    "variableId": 1,
    "componentId": 1,
    "evseDatabaseId": null,
    "createdAt": "2024-05-27T18:00:18.112Z",
    "updatedAt": "2024-05-27T18:00:18.112Z",
    "bootConfigId": null,
    "component": {
      "id": 1,
      "name": "ChargingStation",
      "instance": null,
      "evseDatabaseId": null,
      "createdAt": "2024-05-27T18:00:18.088Z",
      "updatedAt": "2024-05-27T18:00:18.088Z",
      "evse": null
    },
    "variable": {
      "id": 1,
      "name": "Present",
      "instance": null,
      "createdAt": "2024-05-27T18:00:18.109Z",
      "updatedAt": "2024-05-27T18:00:18.109Z",
      "variableCharacteristics": null
    },
    "statuses": []
  },
  {
    "id": 3,
    "stationId": "cp001",
    "type": "Actual",
    "dataType": "boolean",
    "value": "true",
    "mutability": "ReadOnly",
    "persistent": false,
    "constant": false,
    "variableId": 3,
    "componentId": 1,
    "evseDatabaseId": null,
    "createdAt": "2024-05-27T18:00:18.144Z",
    "updatedAt": "2024-05-27T18:00:18.144Z",
    "bootConfigId": null,
    "component": {
      "id": 1,
      "name": "ChargingStation",
      "instance": null,
      "evseDatabaseId": null,
      "createdAt": "2024-05-27T18:00:18.088Z",
      "updatedAt": "2024-05-27T18:00:18.088Z",
      "evse": null
    },
    "variable": {
      "id": 3,
      "name": "Enabled",
      "instance": null,
      "createdAt": "2024-05-27T18:00:18.142Z",
      "updatedAt": "2024-05-27T18:00:18.142Z",
      "variableCharacteristics": null
    },
    "statuses": []
  }
]

There is no SecurityCtrlr component listed. I guess we try to set one using the POST version of that command. Let's try with BasicAuth and see if it creates the station automatically for that case as well.

@shankari
Copy link
Collaborator Author

I think we can pause this for now since Citrine does not support Security Profile 3 anyway. But I think that if/when we get back to this, it will work.

@shankari
Copy link
Collaborator Author

For the record, I figured out an option to add tokens (this is using the DirectUS admin UI at http://localhost:8055
We can add variable attributes and tokens. We should be able to figure out the API calls to set the variable attributes and tokens by seeing what calls the admin UI generates.

@shankari
Copy link
Collaborator Author

Alas, I can't figure out where to put in the token string through the UI. But we are even closer!

@shankari
Copy link
Collaborator Author

shankari commented May 28, 2024

I am not able to edit this branch directly, so I have added the 3 line patch here for the record.
get_the_connection_to_work_without_auth.patch

Testing done: #45 (comment)

@ChrisWeissmann
Copy link
Contributor

Hi @shankari and @louisg1337 , for some local testing on my end I had forked this repo and run it with citrine. To add a location and password I created a small init script. See here: https://github.com/ChrisWeissmann/everest-demo/blob/main/citrineos/init.sh
Here is my start script: https://github.com/ChrisWeissmann/everest-demo/blob/main/citrine-demo-ac-plus-ocpp.sh which just re-uses what you did and adds in the citrine docker compose.

However since I was not working on SP3 things, I did not put in the effort to add support for it in the single line script.

MAEVE_REPO="https://github.com/thoughtworks/maeve-csms.git"
MAEVE_BRANCH="b990d0eddf2bf80be8d9524a7b08029fbb305c7d" # patch files are based on this commit
CSMS_REPO="https://github.com/thoughtworks/maeve-csms.git"
# MAEVE_BRANCH="b990d0eddf2bf80be8d9524a7b08029fbb305c7d" # patch files are based on this commit
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should not remove this. The patches won't won't at an arbitrary point in the repo.
We should have a separate branch for the CITRINE repo

exit 1
fi
else
git reset --hard ${MAEVE_BRANCH}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to restore the line above so that this can run

exit 1
fi

docker-compose -f ./docker-compose.yml up -d
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker-compose.yml is the default file, so no need to specify this here

# Set up CSMS
echo "Setting up ${CSMS}"
if [[ "$CSMS" == "citrine" ]]; then
npm run install-all
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I indicated earlier, this will not work for all versions of npm.
You need to install nvm or run this within a docker container.
#45 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure, I was working on figuring that out next, just wanted to get the initial working version out first

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this is not the first priority. We can document this for now and focus on fixing the other issues:

  • specifying a specific branch
  • scripts for basic auth and token

first

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please incorporate the scripts from #45 (comment)

@shankari
Copy link
Collaborator Author

@louisg1337 you also need to address the DCO

…the script using Citrine, only supports SP1 right now though.

Signed-off-by: louisg1337 <[email protected]>
@louisg1337
Copy link
Contributor

I just pushed out a working version of the one line demo that will run Citrine and connect to EVerest using SP1 (thanks to @ChrisWeissmann). There are a few things to note.

  • In order to run this you need docker, Node 18, and jq installed
  • If you run MaEVe first using SP2 or SP3, then you will need to delete the everest-ac-demo Docker cluster, before re-running the script for Citrine.
    • For some reason, if you run MaEVe SP2 or SP3 first, and then try to run Citrine using SP1, EVerest thinks it is still using the previous SP2/SP3, which Citrine doesn't support yet.
  • I have yet to get the charging session to work as the token authentication keeps on failing. If you run the command below it adds in our DEADBEEF token, yet it still doesn't work.
  curl -X 'PUT' \
  'http://localhost:8080/data/evdriver/authorization?idToken=DEADBEEF&type=ISO14443' \
  -H 'accept: */*' \
  -H 'Content-Type: application/json' \
  -d '{
  "idToken": {
    "customData": {
      "vendorId": "string"
    },
    "idToken": "DEADBEEF",
    "type": "ISO14443"
  },
  "idTokenInfo": {
    "status": "Accepted",
    "cacheExpiryDateTime": "2024-05-28T16:47:28.566Z",
    "chargingPriority": 0,
    "language1": "string",
    "evseId": [
      0
    ],
    "groupIdToken": {
      "customData": {
        "vendorId": "string"
      },
      "idToken": "DEADBEEF",
      "type": "ISO14443"
    },
    "language2": "string",
    "personalMessage": {
      "format": "ASCII",
      "language": "string",
      "content": "string"
    }
  }
}'

Testing Done

  • From a fresh slate (all docker containers removed), all of the commands below successfully connected to EVerest
    • bash ./demo-iso15118-2-ac-plus-ocpp.sh -c -1 (Citrine SP1)
    • bash ./demo-iso15118-2-ac-plus-ocpp.sh -1 (MaEVe SP1)
    • bash ./demo-iso15118-2-ac-plus-ocpp.sh -2 (MaEVe SP2)
    • bash ./demo-iso15118-2-ac-plus-ocpp.sh -3 (MaEVe SP3)

Copy link
Collaborator Author

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@louisg1337 I am fine with merging as soon as the citrine-demo-... file is deleted, and you have edited the README to highlight how to test Citrine versus MaEVe

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@louisg1337 please delete this file since the changes have been incorporated into the same demo-xxx file. This will avoid confusion.

local response=$(curl -s -X POST "$login_url" -H "Content-Type: application/json" -d "$json_body")

# Extract token from the response
local token=$(jq -r '.data.access_token' <<< "$response")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not going to hold up the review for this, but, similar to my previous comment about npm, our goal is to not request additional software installation. The goal behind the "single line demo" is that you have docker installed, and then run one command, and we launch the demo without making permanent changes to your laptop.

As the complexity of these scripts increases, in a future fix, I would suggest pulling them out into their own directory and launching them from the manager, or a docker container of their own that has all the required dependencies.

Comment on lines 352 to 354
if [[ ${CSMS} == "citrine" ]]; then
echo "TODO: Copy over certs for Citrine!"
else
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a TODO to copy over certs, but a TODO to set up the device model correctly.
In the future fix, when we move this out to a script, instead of having the separate sp1.db, sp2.db, ...
we can just use SQLite to edit the device model.

Note also that the device DB should not depend on the CSMS - the whole point of having a standard is that, from a station perspective, it doesn't matter which CSMS it connects to. The station configuration (which is what this is) should be the same.

@louisg1337
Copy link
Contributor

Sounds good to all the above. I just addressed the changes requested, so we should be all good.

@shankari
Copy link
Collaborator Author

@louisg1337 there is an indentation issue caught by the static code analysis

Signed-off-by: louisg1337 <[email protected]>
README.md Outdated Show resolved Hide resolved
louisg1337 and others added 3 commits May 31, 2024 15:56
Signed-off-by: louisg1337 <[email protected]>
Signed-off-by: louisg1337 <[email protected]>
Co-authored-by: K. Shankari <[email protected]>
Signed-off-by: Louis Grassi <[email protected]>
@shankari
Copy link
Collaborator Author

I don't see this getting fixed. I think it is a false positive. I am just going to merge this and we can fix (if needed) in the next PR. I don't want to waste our valuable time futzing around with this.

@shankari
Copy link
Collaborator Author

Squash merging to have a clean commit history; @louisg1337 please pull appropriately.

@shankari shankari merged commit 6cad080 into EVerest:main May 31, 2024
1 of 2 checks passed
docker cp manager/device_model_storage_maeve_sp3.db \
everest-ac-demo-manager-1:/workspace/dist/share/everest/modules/OCPP201/device_model_storage.db
if [[ ${CSMS} == "citrine" ]]; then
echo "TODO: Set up device model correctly!"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@louisg1337 I don't think we have to do any big device model changes. The main thing that we need to adjust is setting the CSMS url to also point to citrine. So we could just add an additional slot to the device model and adjust the priority. Then Everest would try both urls, based on priority, and one of them would work. Here is an example on what to adjust it to: https://github.com/ChrisWeissmann/everest-demo/blob/fe745cfdb0cb3c1abe48ab74e505454cf3d0c81b/manager/config.json

Setting this to the device model sqlite files that are in the repo, there won't need to really be a differentiation, if you are ok with EVerst trying both options. Or adding to the script a sqlite command that manipulates the network priority based on the CSMS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants