Optimise memory usage

[bajtos]

We are running a Station Core instance on Fly.io. In the past, we observed out-of-memory errors when running with 512MB and 768MB available RAM. We solved this problem by bumping up the RAM size to 1GB.

If Station Core does indeed require at least 1GB of RAM, then running Station Desktop (which includes Station Core) on low-end consumer computers and Raspberry PIs will become unpleasant.

We should investigate our memory usage and reduce it to a more reasonable size.

[bajtos]

Slack discussion: https://filecoinproject.slack.com/archives/C03PQG6UT2B/p1697621696130519

[juliangruber]

Shall we move this to the zinnia repo? I don't think there's anything we can do about this in the core repo (as zinnia is aiming to be the sandbox) 🤔

[bajtos]

Well, we don't know yet where the bug is, right? It can be Zinnia or SPARK.

But I guess you are right, it's the responsibility of Zinnia to ensure modules are not using too much RAM.

[bajtos]

OK, I opened two issues in Zinnia:

• a long-term improvement: zinniad: restrict memory usage of modules zinnia#393
• a short-term fix: zinniad: handle out-of-memory errors zinnia#394

I want to keep this issue open until we land any required changes to Core, to make this discussion easily discoverable by Station Core operators.