Skip to content

Commit

Permalink
fix: path
Browse files Browse the repository at this point in the history
  • Loading branch information
asawczyn committed Sep 18, 2024
1 parent 447376d commit 372ef0d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion scripts/nsa/save_pages_from_db_to_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def fetch_documents(collection, batch_size=5000):


def write_to_parquet_in_chunks(file_path, collection, batch_size=5000, chunk_size=50000):
file_path.mkdir(parents=True, exist_ok=True)
file_path.parent.mkdir(parents=True, exist_ok=True)
buffer = []
chunk_index = 0

Expand Down

0 comments on commit 372ef0d

Please sign in to comment.