Enable Security Profile 3 (TLS with Client Side Certificates) #25
Comments
|
First decision: Do we try for profile 1 or just go ahead with profile 2? Given the time left, I am going to try directly for profile 2, but will fall back to profile 1 if I run into glitches. |
|
The fork () has several hardcoded certificates, which are the ones passed in to the maeve command line. Compared to the list of certificates created by the Makefile from (
It certainly seems like the checked in So I am not sure why it is not just named |
shankari
changed the title
|
Moving along, I see a new |
|
checking EVerest, it is consistent with OCPP and not with MaEVe. We might want to file an issue against MaEVe to make it consistent with the OCPP standard. The same 201 configuration that we used earlier has the securityProfile. if we just change this to so it is in fact easier to try with OCPP SP 2 (Maeve: 1) first and we will do it. |
|
It looks like this would be specified in the SecurityCtrl |
|
Forked Maeve is finally up.
Now let's change the connection to wss, the station to cp002 and the security profile to 2 Now we try to make a CSMS connection, but fail because we don't have the CSMS root installed correctly. Ha! figuring out code through log statements! |
|
Checking the mailing list, we also get (from https://lists.lfenergy.org/g/everest/message/1255) and For the record, these are parameters to the EVSE security module https://github.com/EVerest/everest-core/blob/55416be6d11fc85bf1943ecb3c57b267f24ccdaf/modules/EvseSecurity/manifest.yaml#L37 (as one might have guessed) and would be overridden here in our demo: https://github.com/EVerest/everest-core/blob/55416be6d11fc85bf1943ecb3c57b267f24ccdaf/config/config-sil-ocpp201.yaml#L134 (like the For now, we are going to copy the files to the correct locations. |
|
While copying the root V2G cert that is required, i wondered whether I should copy the (I originally found this by trying to verify the csms certificate against both roots and finding that it validated in both cases!!) |
|
After manually copying the root CA, the The handshake is still failing. Given that we are at security profile 2 (TLS with basic auth), with the following diagram from the spec, I would assume that as long as we had the root certificate installed in the CSMS, we would not need the CSMS certificate to be preinstalled. 
But I'm still getting the same failure even though the root cert is installed Maybe we have to copy in the entire bundle and not only the root? Do we also need to copy over the leaves in that case? |
|
From the code, it looks like there are multiple bundles and several of them seem to have the same name. I found the mapping code but it looks like it has changed since the previous EVerest version. But it looks like we do expect it to be a bundle, so let's copy over the bundle and see if it works. |
|
Aha! It turns out that EVerest autogenerates CSMS keys if it doesn't find them Let's delete all the certs and retry |
Here is the full list of certs that EVerest currently has installedDeleting all of them and only recreating the V2G bundle... |
|
Checking the
Let's get all of them in place and see if it works. We then need to ask the EVerest team to clean this up |
|
I've tried the bundle and the root in various places and they all fail. Concretely, this is really confusing - this is supposed to be the bundle, but the default is set to Next steps:
|
|
Ok so now the container has all the certs needed for validation Does it work? |
|
Nope, failed |
|
Before I upgrade, I am going to try one more thing: Take the certs from EVerest and put them into MaEVe and see if we can get that to work. Then it is just certificate wrangling, after an update to make sure that everything works properly. |
|
Checking the current version of the script, it generates keys directly via openssl - e.g. And then the SECC key is copied over as the CSMS key But we don't see the CSMS files although the SECC files exist Rerunning the script after tweaking the path, we get And the cert chain is vald Let's copy this over to the CSMS and see if we can get it to work. |
|
Copied over, the new chain in the CSMS is now valid |
|
The MaEVe certificate generation has the following But the trust.pem that is checked into the fork is not a concatenation but directly derived from the root Since we have replaced the certs, let's replace the trust as well. Now, let's restart everything! |
|
I'm still getting the same error. Aha! the error is in MaEVe and was masked because we started with |
|
All the keys are valid, but are EC (not RSA) and have a password |
|
This is thrown from Verified, by testing with the sample code in the go X509 code, that the Do the certs checked in to the fork actually work?! Screen.Recording.2024-03-10.at.6.13.32.PM.movBUT, 
Screen.Recording.2024-03-10.at.6.17.59.PM.mov |
|
However, the CSMS keypair is still fine, so the gateway starts up properly from the fork. |
|
|
This is where the python version would be helpful - it would allow us have an inspectable working version which we can then use to compare against the non-working versions. I should also be able to inspect the network but not sure what I can see with an encrypted connection. |
|
Tried turning off the load balancer
|
|
My next step will be to write a simple python program that we can run in the same docker container and make the same connection. This is an alternative to getting access to the python-based EVSE implementation. However, I am going to upgrade to the most recent release since there have been several changes to the security code since then. I will then create an example of a broken |
ISO 15118 states that if only 1 SubCA/ICA is present, it has to use the profile of SubCA 2. That's why I named it that way to avoid confusion. |
Where did you change the tag? Note that the script checks out another copy of the everest-demo repo - it is a single-line demo because it does not require the everest-demo code to be checked out before it is run. So changing the local |
I figured it out. The reason the python client code was working although MaEVe wasn't sending the intermediate CAs is that the certificate verification wasn't turned on for the client. So, the although the CSMS was verifying the charger's certs, the charger wasn't verifying the csms's certs. And that's why it didn't throw the error. After I turned the certificate verification on for the OCPP client (charger), it is working perfectly with the chain (leaf + ca2 + ca1) and doesn't work with the single leaf.
Yeah, the best practice is that both the client and the server send their respective chains.
As mentioned before, there was an error in the python client code.
I agree. But in both cases (Mobility House and Everest), the client is sending only the leaf. I think the server assumes that the subca's for the charger chain and it's own chain are the same? In reality, they can be different. Only the root is required to be the same. What is your thought on this?
They actually didn't hard code that. I did that when I issued the cert myself. Remember, that certificate is actually the Contract Certificate (I used same cert everywhere for simplicity) and they are required to have the EMAID as their CN. But in reality, the OCPP server will have a different certificate and it can contain the suitable FQDN as the CN. But we will have to make sure that the testers generate their certs properly. |
I figured this out also. I forked the demo repo, made a commit with those changes and pointed the script to my fork. Now, the one liner correctly pulls the It stays like that indefinitely. But it doesn't always freeze at the same point. I think I will have to create a PR for this. |
|
I've managed to get past all the above errors based on the advise from both this issue and the related Zulip chats. Now I end up in an endless loop of failed websocket connection attempts. They seem to be different errors than what @jhoshiko was seeing regarding websocket errors. I made a few adjustments to the demo script to get past all the other errors in order to get to this one. Has anyone else experienced the endless loop of WebSocket "Underlying Transport Error" errors? |
|
Now that I think about it more, perhaps it's the same cert CN / DNS issue mentioned previously, having to do with the host.docker.internal domain name. I'll do some digging later to better orient myself with how the charger's occp client knows what address to use to connect to the csms. For example, is that a config option or does it use the CN from the csms server cert? |
The current MRE does not have this error. Maybe you could generate an alternate MRE that outlines how you are getting this error. As you can see from the updates here, the difference between success and failure is the way that the certificates are configured, and it is hard to reproduce/debug without knowing that information.
Please see prior issue comments on how the csmsURL is set in the device DB using SQL update |
|
@sahabulh wrt #25 (comment), where did you get the sub_ca pair from? I originally thought that they were from EonTI, but I just got access to the certs there and their sub CA cert does not match the checked in values. I note from #25 (comment) that you issued some of the certs yourself. Are the versions that are checked in certs that you issued, or the ones from EonTI? And if from EonTI, for which test case? |
@shankari I've done some additional digging and see some service connections I had missed. I'll make a couple of additional changes to my local configs to get this demo example working further. More to come ASAP. |
|
@activeshadow sounds good. I hope you saw the existing MRE #29 and the fixes outlined after that in both the Zulip Chat and the issue. @sahabulh has the setup working on his laptop and plans to update the MRE to actually work today, so you might want to coordinate with him as well. |
Yes @shankari that's the one I've been working from, thank you! @sahabulh and I are already coordinating so we should be good. 👍🏻 |
|
@shankari please see this commit on my fork of the @sahabulh and I have both confirmed that it seems to be working, though we're both wondering why no OCPP logs are being generated when a car charging session is initiated in the Node-RED UI. I will submit a PR for the MRE once I'm back on my laptop later. In the meantime I wanted you to be able to test it as well if you're interested in doing so. Note that this MRE is using my fork of Maeve with the load balancer removed. It was giving me some trouble and I wanted to remove it from the equation since it's not part of what we were debugging here anyway. I know that @sahabulh has a fork of Maeve also that he's made some necessary changes to so at some point we'll just need to switch over to that one. Also note that @sahabulh is now going to work on testing security profile 3 since this one is using security profile 2 (at least we think it is anyway). |
|
@activeshadow @shankari I have submitted the pull request (#31) . Issues remaining are described there. But the server-client connection is good and the EV is able to charge using EIM. |
|
@sahabulh thank you for the PR! I have added my comments to it. Can you also respond to my earlier comment about the provenance of the EonTI certs? #25 (comment) |
|
@jhoshiko I just merged PRs to enable security profiles 2 and 3 (thanks to @activeshadow and @sahabulh for the MRE). Please try to run them and verify that they work for you as well. |
|
@shankari Ok, I've tested both However, it looks like I'm getting a connection error due to an invalid http status and the warning Full Log |
|
@jhoshiko The warning should not matter here. We are all getting this warning. Can you please check what's happening on the MaEVe side? Like check logs of MaEVe gateway and manager? The error at least means that the TLS connection was successful. And there is something wrong with the post TLS validation. That's why it is a HTTP error, not a TLS one. I bet you are getting a "bad organization" error from the gateway. If that's true, you will have to add the organization name "Everest" to the list of allowed organizations in MaEVe Which means we will have to modify the MaEVe code. I have already done that in my fork of MaEVe. |
|
@sahabulh @activeshadow the goal of the MRE is to have a script that showcases the steps needed to enable the functionality that you are interested in. It should run without requiring any changes. If we need to add an EVerest organization to MaEVe, let's do so with a patch, indicating that the change is required. We can then follow up with the MaEVe community to discuss whether the list of orgs should be in the source code or whether it can be pulled out to a config file for easier customization. |
|
@sahabulh Here are the logs from the MaEVe manager: MaEVe manager logand here are the logs from the MaEVe gateway: MaEVe gateway logLooks like you are correct, I am getting a bad organization error. |
@shankari I know how to use a patch. The reason it worked for me is that I used my own MaEVe fork which had the OrgName added already. I couldn't use the original repo as the load balancer doesn't work for me. Later, just before submitting the PR, I linked the original repo as you suggested doing so. But I forgot to add the changes needed in the original repo. |
|
I can submit another PR with the patch file and related command added to the one-liner. |
@sahabulh I'm working on this right now so don't worry about spending time on it. |
|
Marking this as done since all the changes have been merged |
In PR #22, we created a one-line demo that allowed us to test end to end charging with OCPP 2.0.1. However it only supports Basic Auth
This issue will track the changes required to change it to support Security Profile 3 (2 in MaEVe since it starts with 0), using a client certificate for authentication.
It will temporarily use a forked version of MaEVe that has hardcoded certificates from an adversarial PKI testing event. Eventually, we will want to have the demo use an open CA and non-proprietary certificates, but make it easy to configure so that testers can easily use proprietary certificates or implementations.
@jhoshiko @sahabulh @CRR-SNL for visibility
The text was updated successfully, but these errors were encountered: