Skip to content

Commit

Permalink
Log details on how much was actually downloaded and in how many chunks
Browse files Browse the repository at this point in the history
  • Loading branch information
yarikoptic committed Sep 16, 2024
1 parent d7f0f6e commit d61f76b
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion dandi/dandiapi.py
Original file line number Diff line number Diff line change
Expand Up @@ -1592,10 +1592,23 @@ def downloader(start_at: int = 0) -> Iterator[bytes]:
# TODO: apparently we might need retries here as well etc
# if result.status_code not in (200, 201):
result.raise_for_status()
nbytes, nchunks = 0, 0
for chunk in result.iter_content(chunk_size=chunk_size):
nchunks += 1
if chunk: # could be some "keep alive"?
nbytes += len(chunk)
yield chunk
lgr.info("Asset %s successfully downloaded", self.identifier)
else:
lgr.debug("'Empty' chunk downloaded for %s", url)

Check warning on line 1602 in dandi/dandiapi.py

View check run for this annotation

Codecov / codecov/patch

dandi/dandiapi.py#L1602

Added line #L1602 was not covered by tests
lgr.info(
"Asset %s (%d bytes in %d chunks starting from %d) successfully "
"downloaded from %s",
self.identifier,
nbytes,
nchunks,
start_at,
url,
)

return downloader

Expand Down

0 comments on commit d61f76b

Please sign in to comment.