Expand capability to download data from other APIs

[AdamTheisen]

A few that we should look at

NOAA FTP sites
https://psl.noaa.gov/data/obs/data/index.php?SiteID=ctd&InstTypeID=28
https://www.esrl.noaa.gov/gmd/dv/data/?site=brw

EPA API for PM2.5
https://docs.airnowapi.org/

Ameriflux?? Not sure if we can access data through API
https://ameriflux.lbl.gov/real-time-data-view-using-influxdb-and-grafana/

[AdamTheisen]

@kenkehoe any others you can think of?

[scollis]

Amazon S3 Have boto3 as optional dependency From: Adam Theisen ***@***.***> Reply-To: ARM-DOE/ACT ***@***.***> Date: Wednesday, April 7, 2021 at 2:03 PM To: ARM-DOE/ACT ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [ARM-DOE/ACT] Expand capability to download data from other APIs (#309) @kenkehoe any others you can think of? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[kenkehoe]

I can't think of any APIs. I think we should focus on readers of downloaded data, and adapt to APIs when needed. I'll take a look at the GMD data and see about reading the data.

[AdamTheisen]

@scollis we just removed boto3 when we shifted away from Bobby's bucket of data for testing. If we have a clear case for it, we can add back in.

@kenkehoe Completely agree on readers. When I was working on the csv reader, I tested against a number of different formats/delimiters and it was fairly robust against what I used (Ameriflux, NEON, aeronet, array of things, etc...). Maybe more examples for these other data types, adding to csvfiles or other readers as appropriate, would be more beneficial to focus on. Thoughts?

[AdamTheisen]

Also, I'm going to move this over to the discussion board since it's not really an issue.

[kenkehoe]

I've added a wrapper around reading NOAA GML ASCII data in PR #314.