[Bug]: nwbinspector fails on folders located on external ssd for mac due to ._ sidecar files

[pauladkisson]

What happened?

Tried to run nwbinspector on folder of nwb files located on an external ssd, but ran into an error.

>  nwbinspector /Volumes/T7/CatalystNeuro/NWB/Lerner/conversion_nwb                                              
Traceback (most recent call last):
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/bin/nwbinspector", line 8, in <module>
    sys.exit(inspect_all_cli())
             ^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/nwbinspector/nwbinspector.py", line 273, in inspect_all_cli
    messages = list(
               ^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/nwbinspector/nwbinspector.py", line 408, in inspect_all
    with pynwb.NWBHDF5IO(path=nwbfile_path, mode="r", load_namespaces=True, driver=driver) as io:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/pynwb/__init__.py", line 274, in __init__
    super().load_namespaces(tm, path, file=file_obj, driver=driver)
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/backends/hdf5/h5tools.py", line 169, in load_namespaces
    open_file_obj = cls.__resolve_file_obj(path, file_obj, driver)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/backends/hdf5/h5tools.py", line 144, in __resolve_file_obj
    file_obj = File(path, 'r', **file_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/h5py/_hl/files.py", line 562, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lerner_lab_to_nwb_env/lib/python3.12/site-packages/h5py/_hl/files.py", line 235, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 102, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

Operating System

macOS

Python Version

3.12

Were you streaming with ROS3?

No

Package Versions

aiobotocore==2.11.1 aiohttp==3.9.2 aioitertools==0.11.0 aiosignal==1.3.1 annotated-types==0.7.0 appdirs==1.4.4 appnope==0.1.4 arrow==1.3.0 asciitree==0.3.3 asttokens==2.4.1 attrs==23.2.0 bidsschematools==0.7.2 blessed==1.20.0 boto3==1.34.27 botocore==1.34.27 bqplot==0.12.42 certifi==2023.11.17 cfgv==3.4.0 charset-normalizer==3.3.2 ci-info==0.3.0 click==8.1.7 click-didyoumean==0.3.0 comm==0.2.1 contourpy==1.2.0 cycler==0.12.1 dandi==0.59.1 dandischema==0.8.2 debugpy==1.8.1 decorator==5.1.1 distlib==0.3.8 dnspython==2.5.0 docstring_parser==0.16 email-validator==2.1.0.post1 entrypoints==0.4 et-xmlfile==1.1.0 etelemetry==0.3.1 executing==2.0.1 fasteners==0.19 filelock==3.13.1 fonttools==4.47.2 fparse==1.20.1 fqdn==1.5.1 frozenlist==1.4.1 fscacher==0.4.0 fsspec==2023.12.2 gast==0.4.0 h5py==3.10.0 hdmf==3.13.0 hdmf_zarr==0.7.0 humanize==4.9.0 identify==2.5.33 idna==3.6 imageio==2.33.1 importlib-metadata==4.13.0 iniconfig==2.0.0 interleave==0.2.1 ipydatagrid==1.2.1 ipydatawidgets==4.3.2 ipyfilechooser==0.6.0 ipykernel==6.29.3 ipympl==0.9.3 ipython==8.20.0 ipython-genutils==0.2.0 ipyvolume==0.6.3 ipyvue==1.10.1 ipyvuetify==1.8.10 ipywebrtc==0.6.0 ipywidgets==8.1.1 isodate==0.6.1 isoduration==20.11.0 jaraco.classes==3.3.0 jedi==0.19.1 jmespath==1.0.1 joblib==1.3.2 jsonpointer==2.4 jsonschema==4.21.1 jsonschema-specifications==2023.12.1 jupyter_client==8.6.0 jupyter_core==5.7.1 jupyterlab-widgets==3.0.9 keyring==24.3.0 keyrings.alt==5.0.0 kiwisolver==1.4.5 lazy_loader==0.3 matplotlib==3.8.2 matplotlib-inline==0.1.6 more-itertools==10.2.0 multidict==6.0.4 natsort==8.4.0 ndx-events==0.2.0 ndx-fiber-photometry @ git+https://github.com/catalystneuro/ndx-fiber-photometry.git@14e05cda86026db694c1c577918284f435b7ec10 ndx-grayscalevolume==0.0.2 ndx-icephys-meta==0.1.0 ndx-photometry @ git+https://github.com/catalystneuro/ndx-photometry.git@c1284c91a7e0a5dfd19f84bbf683e17fa607d4ec ndx-spectrum==0.2.2 nest-asyncio==1.6.0 networkx==3.2.1 neuroconv @ git+https://github.com/catalystneuro/neuroconv.git@f45e77e152369916459a764c918d4c5cbbf1fad1 nodeenv==1.8.0 numcodecs==0.11.0 numpy==1.26.3 nwbinspector==0.4.35 nwbwidgets==0.11.3 openpyxl==3.1.2 packaging==23.2 pandas==2.2.0 parse==1.20.1 parso==0.8.3 pexpect==4.9.0 pillow==10.2.0 platformdirs==4.1.0 plotly==5.13.1 pluggy==1.4.0 pre-commit==3.6.2 prompt-toolkit==3.0.43 psutil==5.9.8 ptyprocess==0.7.0 pure-eval==0.2.2 py2vega==0.6.1 pycryptodomex==3.20.0 pydantic==2.7.1 pydantic_core==2.18.2 Pygments==2.17.2 pynwb==2.7.0 pyout==0.7.3 pyparsing==3.1.1 pytest==8.1.1 python-dateutil==2.8.2 pythreejs==2.4.2 pytz==2023.4 PyYAML==6.0.1 pyzmq==25.1.2 referencing==0.33.0 requests==2.31.0 rfc3339-validator==0.1.4 rfc3987==1.3.8 rpds-py==0.17.1 ruamel.yaml==0.18.5 ruamel.yaml.clib==0.2.8 s3fs==2023.12.2 s3transfer==0.10.0 scikit-image==0.22.0 scikit-learn==1.4.1.post1 scipy==1.12.0 semantic-version==2.10.0 setuptools==69.0.3 six==1.16.0 stack-data==0.6.3 tdt==0.6.6 tenacity==8.2.3 threadpoolctl==3.2.0 tifffile==2023.12.9 tornado==6.4 tqdm==4.66.1 traitlets==5.14.1 traittypes==0.2.1 trimesh==4.1.0 types-python-dateutil==2.8.19.20240106 typing_extensions==4.9.0 tzdata==2023.4 uri-template==1.3.0 urllib3==2.0.7 virtualenv==20.25.0 wcwidth==0.2.13 webcolors==1.13 wheel==0.42.0 widgetsnbextension==4.0.9 wrapt==1.16.0 yarl==1.9.4 zarr==2.16.1 zarr-checksum==0.4.0 zipp==3.17.0

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you ensured this bug was not already reported?
• To the best of your ability, have you ensured this is a bug within the code that checks the NWBFile, rather than a bug in the NWBFile reader (e.g., PyNWB or MatNWB)?

[CodyCBakerPhD]

Thanks for the official issue (for context, Paul reached out on Slack earlier)

After some thought, I think it would be fine to just filter the *.nwb glob by anything not preceded by ._

Though it's in the title, not the issue description (and traceback doesn't do it justice) - this is a very particular issue for certain types of external drives on certain types of Macs - there seem to be some tools in existence for managing and even cleaning up these ghost files, but we can also just ignore them here