Speed/memory testing on app

[ehinman]

Describe the bug
The app can take nearly 3 GB of memory at full capacity, and this may be problematic for hosting online. We can try packages like profvis to figure out where the app is spending its time and try to speed things up: https://shiny.posit.co/r/articles/improve/profiling/

Previous issue text: I tried downloading a full period of record of data from Utah (2014-2020) in Module 1 and watched my Task Manager processes. Data successfully downloaded, visualized in a map, flagged, and censored data were handled with no issues. The max memory used was about 7.5 GB. We might want to dig into the main memory drains within the app using something like the profvis package. Also see: https://www.r-bloggers.com/2020/07/4-tips-to-make-your-shiny-dashboard-faster/

[cristinamullin]

Suggestions from Shelly:

I think this link speaks to the strategy we need to adopt for memory management: https://bookdown.org/content/d1e53ac9-28ce-472f-bc2c-f499f18264a3/releasememory.html This is closing in on the way memory is managed in more advanced programming languages like C and C##.

Also, below is an excerpt from our chat that is worth sharing. We can pick this conversation back up directly once TADA funding moves but I think they are saying you need to write very tight code because R Shiny has limits. We should also take up Brad’s suggestion about testing in RShiny.io.

[Friday 10:53 AM] Paul Dziemiela
R just does everything in memory to my knowledge
[Friday 10:54 AM] Paul Dziemiela
Paul the database developer generally recommends a database.
[Friday 10:55 AM] Brad Cooper
Brad the app developer says, there are reasons R is not promoted as a web service tier (i.e. running a bunch of stuff in memory on a server for concurrent users).
[Friday 10:57 AM] Brad Cooper
even if you can't officially run it on Shiny.io, maybe try it out to there to see if its code related vs. cloud.gov platform.

[cristinamullin]

I am not sure if this issue is still relevant or can be closed now that we switched to using Posit Connect for the public web app: rconnect-public.epa.gov/TADAShiny/. It would be helpful to test the current limits of the application again (max number of rows, speed, etc.) after some of the other bugs are addressed first.