From 6ab9bb53c2b0d1a683bd35e3090cd2b7b327ea90 Mon Sep 17 00:00:00 2001 From: Heberto Mayorquin Date: Wed, 3 Apr 2024 19:28:38 -0600 Subject: [PATCH 1/6] Add documentation about duplicated metadata in `Electrodes` (#442) * add documentation about duplicated metadata * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- docs/best_practices/ecephys.rst | 6 ++++++ docs/best_practices/general.rst | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/docs/best_practices/ecephys.rst b/docs/best_practices/ecephys.rst index fffe593af..e2a86fb6a 100644 --- a/docs/best_practices/ecephys.rst +++ b/docs/best_practices/ecephys.rst @@ -58,6 +58,12 @@ For relative position of an electrode on a probe, use ``rel_x``, ``rel_y``, and that are close enough to share a neuron. +Avoid Duplication of Metadata +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``ElectrodeTable`` should not contain redundant information that is present somewhere else within the :ref:`nwb-schema:sec-NWBFile` . Avoid adding columns to the `ElectrodeTable` that correspond to properties of the :ref:`nwb-schema:sec-ElectricalSeries` such as ``unit``, ``offsets`` or ``channel gains`` These properties should be stored in the corresponding attributes of the :ref:`nwb-schema:sec-ElectricalSeries` object. + +As a concrete example, the package objects from the `SpikeInterface `__ package contain two properties named ``gain_to_uv`` and ``offset_to_uv`` that are used to convert the raw data to microvolts. These properties should not be stored in the `ElectrodeTable` but rather in the ``ElectricalSeries`` object as ``channel_conversion`` and ``offset`` respectively. Units Table ----------- diff --git a/docs/best_practices/general.rst b/docs/best_practices/general.rst index 2005e22f9..1f68f6f3c 100644 --- a/docs/best_practices/general.rst +++ b/docs/best_practices/general.rst @@ -68,3 +68,9 @@ Empty Strings Required free-text fields for neurodata types should not use placeholders such as empty strings (`""`), ``"no description"``, or ``"PLACEHOLDER"``. For example, the :py:attr:`description` field should always richly describe that particular neurodata type and its interpretation within the experiment. Many attributes of neurodata types in NWB are optional details to include. It is not necessary, therefore, to use placeholders for these attributes. Instead, they should not be specified at all. + + +Avoid Duplication of Metadata +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Avoid duplication of metadata across different objects. If a piece of metadata is shared between multiple objects, consider creating a separate object to store that metadata and linking to it from the other objects. This will help to keep the metadata consistent and reduce the risk of errors when updating the metadata. From 37b6e5bfc1588feca12f5abd24dc0eac44be6450 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 8 Apr 2024 21:49:36 -0400 Subject: [PATCH 2/6] [pre-commit.ci] pre-commit autoupdate (#444) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/pre-commit/pre-commit-hooks: v4.5.0 → v4.6.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.5.0...v4.6.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index af433f5e1..87f207f17 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,6 +1,6 @@ repos: - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.5.0 + rev: v4.6.0 hooks: - id: check-yaml - id: end-of-file-fixer From 455e1f93e8b9ff29b8e8d58a139e10d229d6ac7b Mon Sep 17 00:00:00 2001 From: Cody Baker <51133164+CodyCBakerPhD@users.noreply.github.com> Date: Wed, 10 Apr 2024 22:29:18 -0400 Subject: [PATCH 3/6] Update version.py --- src/nwbinspector/version.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/nwbinspector/version.py b/src/nwbinspector/version.py index 86de6bae4..8a64aa006 100644 --- a/src/nwbinspector/version.py +++ b/src/nwbinspector/version.py @@ -1 +1 @@ -__version__ = "0.4.34" +__version__ = "0.4.35" From ad0f5ddfc72479c37edd50123c338536e51306fa Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 15 Apr 2024 16:27:28 -0400 Subject: [PATCH 4/6] [pre-commit.ci] pre-commit autoupdate (#445) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/psf/black: 24.3.0 → 24.4.0](https://github.com/psf/black/compare/24.3.0...24.4.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 87f207f17..9390a986c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -6,6 +6,6 @@ repos: - id: end-of-file-fixer - id: trailing-whitespace - repo: https://github.com/psf/black - rev: 24.3.0 + rev: 24.4.0 hooks: - id: black From faf9cda0701ea6e9030a9c4157af4ef24ee179be Mon Sep 17 00:00:00 2001 From: Daniel Sprague <42728889+dysprague@users.noreply.github.com> Date: Fri, 19 Apr 2024 12:09:01 -0500 Subject: [PATCH 5/6] fixed module search flag in nwbinspector (#446) * fixed module search flag in nwbinspector * add test for module import * added to changelog * Update CHANGELOG.md --------- Co-authored-by: Cody Baker <51133164+CodyCBakerPhD@users.noreply.github.com> --- CHANGELOG.md | 6 ++++++ src/nwbinspector/nwbinspector.py | 1 + tests/test_inspector.py | 3 ++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6fa3f46e8..32a13b50d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,11 @@ # Upcoming +# v0.4.34 + +### Fixes + +* Fixed `--modules` flag in `nwbinspector` command line interface to allow for import of additional modules in the command line. This was necessary to be able to register new customized checks to the NWB Inspector. [#446](https://github.com/NeurodataWithoutBorders/nwbinspector/pull/446) + # v0.4.33 ### Fixes diff --git a/src/nwbinspector/nwbinspector.py b/src/nwbinspector/nwbinspector.py index b06b6ad82..704f98722 100644 --- a/src/nwbinspector/nwbinspector.py +++ b/src/nwbinspector/nwbinspector.py @@ -252,6 +252,7 @@ def inspect_all_cli( DANDI archive (i.e., https://dandiarchive.org/dandiset/{dandiset_id}/{version_id}), or a six-digit Dandiset ID. """ levels = ["importance", "file_path"] if levels is None else levels.split(",") + modules = [] if modules is None else modules.split(",") reverse = [False] * len(levels) if reverse is None else [strtobool(x) for x in reverse.split(",")] progress_bar = strtobool(progress_bar) if progress_bar is not None else True if config is not None: diff --git a/tests/test_inspector.py b/tests/test_inspector.py index 1c207f4db..4db14f219 100644 --- a/tests/test_inspector.py +++ b/tests/test_inspector.py @@ -390,7 +390,8 @@ def test_command_line_runs_cli_only(self): console_output_file = self.tempdir / "test_console_output.txt" os.system( f"nwbinspector {str(self.tempdir)} --overwrite --select check_timestamps_match_first_dimension," - "check_data_orientation,check_regular_timestamps,check_small_dataset_compression" + "check_data_orientation,check_regular_timestamps,check_small_dataset_compression " + "--modules random,math,datetime" f"> {console_output_file}" ) self.assertLogFileContentsEqual( From 9ced2cc8b80d6e934e4be067b60400a9a204b57c Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 22 Apr 2024 09:24:17 -0700 Subject: [PATCH 6/6] Add codespell support (config, workflow to detect/not fix) and make it fix few typos (#448) * Add github action to codespell dev on push and PRs * Add rudimentary codespell config * Add pre-commit definition for codespell * [DATALAD RUNCMD] run codespell throughout fixing typos automagically === Do not change lines below === { "chain": [], "cmd": "codespell -w", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ --- .github/workflows/codespell.yml | 23 +++++++++++++++++++ .pre-commit-config.yaml | 8 +++++++ CHANGELOG.md | 2 +- docs/best_practices/best_practices_index.rst | 2 +- docs/best_practices/extensions.rst | 2 +- .../using_the_command_line_interface.rst | 4 ++-- pyproject.toml | 7 ++++++ 7 files changed, 43 insertions(+), 5 deletions(-) create mode 100644 .github/workflows/codespell.yml diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml new file mode 100644 index 000000000..a7e91c56e --- /dev/null +++ b/.github/workflows/codespell.yml @@ -0,0 +1,23 @@ +# Codespell configuration is within pyproject.toml +--- +name: Codespell + +on: + push: + branches: [dev] + pull_request: + branches: [dev] + +permissions: + contents: read + +jobs: + codespell: + name: Check for spelling errors + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Codespell + uses: codespell-project/actions-codespell@v2 diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 9390a986c..9cc67c9a8 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -9,3 +9,11 @@ repos: rev: 24.4.0 hooks: - id: black + +- repo: https://github.com/codespell-project/codespell + # Configuration for codespell is in pyproject.toml + rev: v2.2.6 + hooks: + - id: codespell + additional_dependencies: + - tomli diff --git a/CHANGELOG.md b/CHANGELOG.md index 32a13b50d..80a94c15c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -191,7 +191,7 @@ # v0.4.14 ### Fixes -* Fixed an error with attribute retrieval specific to the `cell_id` of the `IntracellularElectrode` neurodata type that occured with respect to older versions of PyNWB. [PR #264](https://github.com/NeurodataWithoutBorders/nwbinspector/pull/264) +* Fixed an error with attribute retrieval specific to the `cell_id` of the `IntracellularElectrode` neurodata type that occurred with respect to older versions of PyNWB. [PR #264](https://github.com/NeurodataWithoutBorders/nwbinspector/pull/264) diff --git a/docs/best_practices/best_practices_index.rst b/docs/best_practices/best_practices_index.rst index c138e0ef2..25632460c 100644 --- a/docs/best_practices/best_practices_index.rst +++ b/docs/best_practices/best_practices_index.rst @@ -7,7 +7,7 @@ and its ecosystem of software tools. To enable NWB to accommodate the needs of the diverse neuroscience community, NWB provides a great degree of flexibility. In particular, the number of instances of a particular neurodata_type and corresponding names are often not fixed, to enable, -e.g., storage of data from arbitrary numbers of devices withing the same file. While this flexibility is essential to enable +e.g., storage of data from arbitrary numbers of devices within the same file. While this flexibility is essential to enable coverage of a broad range of use-cases, it can also lead to ambiguity. At the same time, we ultimately have the desire to have the schema as strict-as-possible to provide users and tool builders with a consistent organization of data. As such, we need to strike a fine balance between flexibility to enable support for varying experiments and use-cases and strictness in the schema diff --git a/docs/best_practices/extensions.rst b/docs/best_practices/extensions.rst index c6f7b43c9..bfaa988e2 100644 --- a/docs/best_practices/extensions.rst +++ b/docs/best_practices/extensions.rst @@ -5,7 +5,7 @@ Extend the core NWB schema only when necessary. Extensions are an essential mech data with NWB that is otherwise not supported. However, we here need to consider that there are certain costs associated with extensions, *e.g.*, cost of creating, supporting, documenting, and maintaining new extensions and effort for users to use and learn already-created extensions. As such, users should attempt to use core ``neurodata_types`` or -pre-existing extentions before creating new ones. :ref:`hdmf-schema:sec-dynamictable`, which are used throughout the +pre-existing extensions before creating new ones. :ref:`hdmf-schema:sec-dynamictable`, which are used throughout the NWB schema to store information about time intervals, electrodes, or spiking output, provide the ability to dynamically add columns without the need for extensions, and can help avoid the need for custom extensions in many cases. diff --git a/docs/user_guide/using_the_command_line_interface.rst b/docs/user_guide/using_the_command_line_interface.rst index d9c104a79..47f1612f1 100644 --- a/docs/user_guide/using_the_command_line_interface.rst +++ b/docs/user_guide/using_the_command_line_interface.rst @@ -25,7 +25,7 @@ the most useful of these options. Streaming --------- -If the NWB file(s) you wish to inspect are already on the :dandi-archive:`DANDI archive <>`, you can run the NWB Inspector directly on that DANDI set instead of having to download it. All that is needed is to specfy the DANDI set ID (six-digit identifier) as the path and add the ``--stream`` flag. +If the NWB file(s) you wish to inspect are already on the :dandi-archive:`DANDI archive <>`, you can run the NWB Inspector directly on that DANDI set instead of having to download it. All that is needed is to specify the DANDI set ID (six-digit identifier) as the path and add the ``--stream`` flag. :: @@ -75,7 +75,7 @@ For example, numbers like ``"0, 1"``. -The defalt report also aggregates identical outputs into a summary over multiple files; running +The default report also aggregates identical outputs into a summary over multiple files; running :: diff --git a/pyproject.toml b/pyproject.toml index bee38d675..2b3b8638f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -20,3 +20,10 @@ extend-exclude = ''' |dist )/ ''' + +[tool.codespell] +# Ref: https://github.com/codespell-project/codespell#using-a-config-file +skip = '.git*,*.pdf,*.css' +check-hidden = true +# ignore-regex = '' +# ignore-words-list = ''