"Failed to Fetch" error when fetching Task - Proxy Server can't fetch large JSON files

[shaunanoordin]

Functionality Issue

When attempting to fetch a Task (i.e. typing in an "ML Task Request ID" and clicking the "Fetch" button at Step 2), the process sometimes fails with the error message "Failed to Fetch"

For this specific indicent,

• URL was https://subject-assistant.zooniverse.org/#/tasks/fc7fe91f-070a-4ed8-b743-bf9bdaa8011e
• The Task fetch status was a success, but the Results fetch status was an error.

Dev Notes

Analysis:

• The fact that the Subject Assistant frontend was able to fetch the Task but NOT the Results indicated a very conditional issue (i.e. it's not that the whole Proxy Server died)
• Digging deeper, we discover the following:
  • The fetch for the Task JSON, via the Proxy Server, worked fine.
  • The fetch for the Results JSON, via the Proxy Server, resulted in an error.
  • Examining that Proxy Server URL, we discover that the Proxy server returned a 502 Bad Gateway.
  • Examining the target URL (the cameratraps.blob.core.windows.net one), we discover that the Results JSON file was properly generated by the ML service.
  • Downloading the Results JSON file, we find that it's about 30MB
• Putting this together, we can see that the failure is at the Proxy Server, and may be related to the target URL's actual file size.
  • This theory gains more credibility when we try to fetch a Task with a much smaller (1MB) Results JSON file, and find no issues.

Conclusion: 🕵️ It looks like the Proxy Server was set up to work with much smaller data sizes than what an actual project owner would actually require.

Note: sometimes the entire Proxy Server will die with a 503 Service Unavailable error, but this is usually an intermittent downtime caused by the 502 above.

Solution:

• Increase the Proxy Server's available RAM. This should allow it to handle larger ("truer-to-actual-use") results.

Further thoughts:

• We need to keep an eye on actual usage of the service by users, e.g. will 1GB JSON files be a standard expectation?? (I hope not - please break down Subject Set batches to smaller sizes!)

Status

Solved by #159, which increased the Proxy Server's memory limits from 100MB

⚠️ Please note that there may be other instances when the "Failed to Fetch" error might appear (e.g. the Proxy Server is dead altogether), this Issue is only documenting the one encountered during the Subject Assistant Zoom meeting today.

Credit to Sarah H for pointing out the problem, and Zach for delivering a quick solution.

[eatyourgreens]

Have you got HTTP compression enabled on that server? It might help if you're serving 30MB files.

[eatyourgreens]

Streaming large responses might help too, assuming that you're using Node/Express.