Skip to content

Commit

Permalink
Update src/corppa/utils/filter.py
Browse files Browse the repository at this point in the history
Co-authored-by: Laure Thompson <[email protected]>
  • Loading branch information
rlskoeser and laurejt authored Mar 28, 2024
1 parent 206ccd4 commit a88e17e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/corppa/utils/filter.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def save_filtered_corpus(input_filename, output_filename, idfile):
"""
# read the id file and generate a list of ids
with open(idfile) as idfile_content:
source_ids = [line.strip() for line in idfile_content.readlines()]
source_ids = [line.strip() for line in idfile_content]

# use orjsonl to stream filtered pages to specified output file
orjsonl.save(output_filename, filter_pages(input_filename, source_ids))
Expand Down

0 comments on commit a88e17e

Please sign in to comment.