Skip to content

Commit

Permalink
Correctly generate dataset filename in DatasetCompetitionFormat
Browse files Browse the repository at this point in the history
Since the way that BillionScaleDatasetCompetitionFormat and DatasetCompetitionFormat generates the dataset filename differs, we got the following error during the benchmarks:
  • Loading branch information
onurctirtir authored Aug 15, 2024
1 parent ea076f1 commit 735b064
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions benchmark/datasets.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,9 @@ def prepare(self, skip_data=False, original_size=10**9):

def get_dataset_fn(self):
fn = os.path.join(self.basedir, self.ds_fn)
if self.nb != 10**9:
fn += '.crop_nb_%d' % self.nb

if os.path.exists(fn):
return fn
else:
Expand Down

0 comments on commit 735b064

Please sign in to comment.