[Feature]: Test read of dandisets systematically in CI

[rly]

What would you like to see added to PyNWB?

Follow-up to #1794

It would be ideal if in the CI of this repo or in a separate repo like https://github.com/dandi/dandisets-healthstatus, we test reading of NWB files from dandisets using the latest commit of pynwb. Given limits on CI runtime, we have to split these checks into batches. Currently the batches are decided randomly but that means the tests can pass simply due to bad luck. We should test systematically instead of randomly, e.g., scan through dandisets 000001-000100 on monday, 000101-000200 on tuesday, etc.

Separately but relatedly, it would be nice if we could create copies of nwb files on dandi but without the bulk array data (datasets with size above some threshold). The files would be significantly smaller, and it would be significantly faster to test. Similar to https://github.com/bids-standard/bids-examples

Is your feature request related to a problem?

No response

What solution would you like?

See above.

Do you have any interest in helping implement the feature?

Yes.

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you checked the Contributing document?
• Have you ensured this change was not already requested?