-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Everything-up-front pagination #2029
Comments
This was referenced Mar 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There are annoyances we can't fix in the current implementation like the fact that if you go to the next page of results in the console and refresh the page, you'll land back on the first page of results because the page is not represented in the URL (#1102). We have long thought we were blocked on improving pagination UX by Dropshot limitations like the lack of a way to paginate backwards (oxidecomputer/dropshot#436). However, after @ahl explained to me that the intent with that design was that the client would fetch approximately everything up front and handle pagination itself, my mind was opened to a world of possibilities that are frankly very cool.
The default page size in Dropshot is 100, and the max is 10000 (source). Nexus does not override these defaults.
In general, we tend to think that 1000 is a rough upper bound for how many items can expect to get in any list of things. You can make more than that of some resources, but you really have to try. If instead of pulling one page at a time from the API, we fetched everything we can find, there would be numerous advantages:
page
param scheme we want, if we still want one when search and filtering is so goodThe obvious first objection is that fetching 400 things is probably slower than fetching the first 25, we would delay initial page load. Now, I'm not sure yet whether that's true enough for us to care, and it may be that we can get a lot of mileage out of fetching a huge page without getting clever. We might even be better off implementing gzip on the API side (oxidecomputer/dropshot#221) than we are getting clever on the client — JSON bodies have a ton of repetition and probably compress really well.
If we want to do this well, we should also address some long-standing pain points, like the fact that you have to manually make sure you are fetching the right number of things in the loader to match what
QueryTable
does automatically. I have been lamenting recently that we do quite bit of work fighting the way the paginated endpoints are handled in the client generator. I have a hunch there is a smart abstraction that would be easy to build on the generation side that would save a lot of work on the app side.The text was updated successfully, but these errors were encountered: