ArcGIS Notes

JS Rest API for ArcGIS

https://esri.github.io/arcgis-rest-js/guides/node/

Uploading a csv file to an existing layer could look like this:

const upload = async () => {
  const csvFilePath = "data/alpine_2016.csv";

  const json = await csv().fromFile(csvFilePath);

  const geoJosn = json.map((j) => ({
    attributes: j,
    geometry: { x: j.landing_lng, y: j.landing_lat },
  }));

  delete json; // clear up the extra memory

  let index = 0;
  const size = 100;
  while (index < geoJosn.length) {
    const chunk = geoJosn.slice(index, size + index);
    console.log("got a chunk of size", chunk.length, index);

    try {
      await addFeatures({
        url: featureLayer,
        features: chunk,
        authentication,
      });

      index += size;
    } catch (e) {
      console.error(e);
      console.log("trying again from index ", index);
    }
  }
}

Query is simple

  queryFeatures({
    url: featureLayer,
    where: "cluster_no not in (123,321)",
    authentication,
  });

Delete isn't working for big data sets from the lib, but the API works fine https://developers.arcgis.com/rest/services-reference/delete-features.htm.

API Auth: Created an OAuth 2.0 application, which has access to private feature layers. Note -- once the data has been uploaded the feature layer can probably become public.

General Tricks

Combine all county processed CSV files into on big file awk '(NR == 1) || (FNR > 1)' *processed.csv > allcounties.csv