Some metadata csv files don't get downloaded from EcephysProjectCachce.from_warehouse

[jkbhagatio]

Describe the bug
Following this tutorial, I get an error when getting to the cache.get_session_table() command:

from pathlib import Path

from allensdk.brain_observatory.ecephys.ecephys_project_cache import EcephysProjectCache


DOWNLOAD_COMPLETE_DATASET = True

out_dir = Path(r"C:\Users\jai\mini\data")
out_dir.mkdir(exist_ok=True)
print(f"{out_dir.exists()=}")

manifest_path = out_dir / "manifest.json"
cache = EcephysProjectCache.from_warehouse(manifest=manifest_path)
sessions = cache.get_session_table()

FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\jai\mini\data\units.csv'

Upon checking my data directory, I see only probes.csv and sessions.csv have been downloaded, and units.csv and channels.csv are missing. I've tried re-running many times, and still these files do not get downloaded.

Environment (please complete the following information):

• OS & version: Windows 10
• Python version: 3.11
• AllenSDK version: 2.16.2

Additional context
Add any other context about the problem here.

Do you want to work on this issue?
Are you willing and able to fix this bug? If so, let us know here (and see the guide). Thank you!

[jkbhagatio]

Hi @kdharris101, this is a similar but different error to yours #2751

To move forward with a project I am working on I think I just need the four metadata csv files for the neuropixels visual coding dataset - sessions.csv, probes.csv, channels.csv, units.csv. If you have these, could you drop them here, or email them to me (jkbhagatio[at]gmail[dot]com) ?

I've also reached out to @jsiegle about trying to get this data. Thanks!

[jsiegle]

Hi @jkbhagatio, thanks for tagging me in this.

For anyone experiencing this issue, the relevant CSV files can be manually downloaded here.

However, I'm not able to replicate this with a clean install of the AllenSDK (Windows 10, Python 3.11). The initial download is slower than I'd expect (about 2 minutes), but all of the files eventually show up.

Is it possible there's something else in your environment that is affecting these downloads?

[jsiegle]

This looks like the same issue as #2742

[kdharris101]

My error wasn't actually a real problem - it was just taking very long for the files to download (like 5 minutes or more) and there was no message saying it was loading or please wait, so i thought it had hung. After waiting long enough everything worked fine.

So i closed #2751 and opened #2752 that suggests adding a progress bar for downloads.

[jkbhagatio]

This looks like the same issue as #2742

Ah yes @jsiegle sorry somehow I missed this issue when I searched for similar issues - yes it is basically the same issue, except that in my case python actually completes execution and I am able to run additional commands (not sure if the download is still trying to be attempted / executed in the background)

[jsiegle]

@jkbhagatio no worries, I was just tagging the other issue so it's visible to whoever ends up addressing this.

And @kdharris101 I definitely agree a progress bar would be ideal here!