Update new datasets layout. #16
Conversation
| @@ -590,7 +590,7 @@ mod tests { | |||
| ); | |||
| let downloaded_path = api | |||
| .repo(repo) | |||
| .download("wikitext-103-v1/wikitext-test.parquet") | |||
There was a problem hiding this comment.
Are you sure you don't just want to fix the version to a sha to avoid having to change this regularly?
There was a problem hiding this comment.
FYI we don't plan to do any other breaking changes™ in the path names ;)
You can keep it like this
There was a problem hiding this comment.
I prefer to know about those changes.
Still early but I think mnist shouldn't receive that many changes aside from big revamps like this :)
There was a problem hiding this comment.
I genuinely don't understand why you're not just using a sha for this, what's the point of having something that could potentially change in your test suite?
There was a problem hiding this comment.
The point is to make the actual request I'm interested about, this is the exact code used in the candle book.
I'm interested in making sure the answer from that specific call doesn't vary. I don't care (and don't want to ) pin a revision in a demo/tutorial.
Those integration sort of tests are by nature a bit flaky as they do depend on the exterior world (if the hub is down, or the dataset teams modifies the repo..). Having them few and far between is essential.
But this particular one I don't expect a lot of changes, and I'll attribute the recent changes to "bad timing". And the actual changes seem for the better as the structure is now more readable for instance.
No description provided.