Improper Parsing of AFDC Data and Many Missing Chargers

[kamranjon]

There are pretty large discrepancies between AFDC data and openChargeMaps data. One of the more obvious discrepancies is a large amount of Electrify America Chargers missing in the dataset. You can devise this by filtering on Washington and seeing that there are only 2 entries for Electrify America in all of Washington in the OCM data set - while the AFDC dataset contains 17. Oregon is also missing a few. But simply counting the results from each data set you can see OCM is missing thousands of chargers.

I also noticed that the isOperational field appears to be defaulting to false (there is no logic in ImportProvider_AFDC which references the isOperational flag). If you filter on isOperational using the data set for US - the available chargers are cut down to 5600 - the AFDC data has flags for these things: status_code and access_code are the fields - you can find the documentation on all of that here: https://afdc.energy.gov/data_download/alt_fuel_stations_format
The ImportProvider_AFDC appears to be parsing a single field: groups_with_access_code - to determine many values which are explicitly provided by the AFDC data through fields like status_code and access_code:
https://github.com/openchargemap/ocm-system/blob/master/Import/OCM.Import.Common/Providers/ImportProvider_AFDC.cs#L120

Another likely issue is that the entire parsing logic is wrapped in a try/catch - so if anything in there fails - the entire record is thrown out - this is probably not the best choice for datasets like this where a single field could be bad but the rest of the data could be valuable.

[webprofusion-chrisc]

Thanks for looking into this, the AFDC import was written about 9 years ago and hasn't had many major updates since. Clearly it's time for a review!

[kamranjon]

I saw that it had been a while since anyone touched this and figured the AFDC data had gotten better at specifying more distinct fields. Shouldn't be too hard of a fix (I don't code in C# otherwise I'd pop up a PR) - I am working on an app in Golang and would love to help contribute data through any users I might get.
Thanks for making this - it's really awesome to be able to get data for other countries - it's really hard to get good data.

[webprofusion-chrisc]

Thanks, c# is pretty easy to pick up but the overall build/testing is quite hard for the project overall. We need to add some unit tests for imports which would make this sort of thing easier to test and change.