Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harvester error when request response bigger than 100mb #424

Closed
robitaillej opened this issue May 21, 2024 · 3 comments
Closed

Harvester error when request response bigger than 100mb #424

robitaillej opened this issue May 21, 2024 · 3 comments

Comments

@robitaillej
Copy link
Contributor

When CDE harvester is running, it seems that the limit of 100mb for request response is too conservative for some datasets. When harvesting datasets with millions of profiles (usually TimeseriesProfiles), requesting all those individual profiles will create a response over 100mb and raise an error, preventing this dataset to be included to the CDE.
A quick fix would be to increase that value. A more exhaustive one would be to rethink the logic behind the display of TimeseriesProfiles datasets.

@JessyBarrette
Copy link
Contributor

The issue was corrected by #423 and should be updated in the production deployment soon 🤞

@JessyBarrette
Copy link
Contributor

This is now merged in production, let's see if does resolve the issue

@robitaillej
Copy link
Contributor Author

Raising the limit ot 200mb has resolved this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants