MongoDB, Prediction Model, Heroku, and Surge

MongoDB stores historical data as well as any data entered in Survey123:

• Historical data was manually entered via a POST call to the router.
• Potential TODO: Carissa may update the historical data csv (data/SPB2016_toDALI.csv) in which case the historical data should be replaced with the contents of the updated file
• Survey123 data is pulled from what ArcGIS calls the feature-service layer. This pipeline to mongo is setup but not automatic (a dev would have to POST to the uploadSurvey123 endpoint)
• Potential TODO: automatically edit mongoDB when survey123 data changes
• TODO: set up automatic switching between local mongoDB and production mongo URL
• note–on getting data from mongo: This call is made from viewdata.js (not from importFromMongo.js, which requires axios, which is not supported for now since the frontend gets deployed statically)

MongoDB data types:

• There are two sources of data: historical data the partners send us, and Survey123. Each of these has a corresponding controller file.
• Historical data matches the mongoose model Historical
• Survey123 data may be either of two types, a survey response concerning beetle counts, or a response concerning spots of dying trees. Trapped beetle data is saved with the mongoose model Trapping and tree spot data with the model Spots. On importing data from Survey123, the upload function tries to distinguish which each survey response is by searching for fields that we expect to differ between those two types of responses.

Getting predictions of spot outbreak probabilities:

• Carissa wrote an R script, SPB-Predictions.v02.R, which takes as input summary statistics of a region's data and outputs probabilities of various outbreak sizes
• We run her R script using the node package r-script. This allows us to wrap her R script in javascript, passing in a javascript object and similarly, receiving an object.
• TODO: the endpoint /getPredictions works but is not actually called from the frontend/UI

Deploying the backend:

• The backend is deployed to heroku. There are two build packages: first, the node package, but second, a package to support R. In order to have this package we're using heroku-16, since this is the latest version that supports the R build package.
• To re-deploy: yarn prod. This explicitly copies all R files into the dist/ folder
• We informed the partners that the limit on mongo sandbox (the free version) is 1GB. So far there is only ~1MB data and we don't expect that they'll exceed 1GB within the year (or even within a few years).

Deploying the frontend:

• The frontend is essentially statically deployed. To re-deploy: yarn deploy. All this does is publish src/ to surge.
• This means no node packages can be used unless they're importable into html files. To make calls to the router we've been using ajax, which doesn't need to be imported as it's built into javascript.