Modifications to run on openmind, refactoring, and auto-formatting.

README.md

@@ -40,27 +40,22 @@ Each conversion is organized in a directory of its own in the `src` directory:
     └── src
         ├── jazayeri_lab_to_nwb
         │   ├── watters
-        │       ├── wattersbehaviorinterface.py
-        │       ├── watters_convert_session.py
-        │       ├── watters_metadata.yml
-        │       ├── wattersnwbconverter.py
-        │       ├── watters_requirements.txt
-        │       ├── watters_notes.md
-
+        │       ├── behavior_interface.py
+        │       ├── main_convert_session.py
+        │       ├── metadata.yml
+        │       ├── nwb_converter.py
+        │       ├── requirements.txt
         │       └── __init__.py
-
         │   └── another_conversion
-
         └── __init__.py
 
  For example, for the conversion `watters` you can find a directory located in `src/jazayeri-lab-to-nwb/watters`. Inside each conversion directory you can find the following files:
 
-* `watters_convert_sesion.py`: this script defines the function to convert one full session of the conversion.
-* `watters_requirements.txt`: dependencies specific to this conversion.
-* `watters_metadata.yml`: metadata in yaml format for this specific conversion.
-* `wattersbehaviorinterface.py`: the behavior interface. Usually ad-hoc for each conversion.
-* `wattersnwbconverter.py`: the place where the `NWBConverter` class is defined.
-* `watters_notes.md`: notes and comments concerning this specific conversion.
+* `main_convert_sesion.py`: this script defines the function to convert one full session of the conversion.
+* `requirements.txt`: dependencies specific to this conversion.
+* `metadata.yml`: metadata in yaml format for this specific conversion.
+* `behavior_interface.py`: the behavior interface. Usually ad-hoc for each conversion.
+* `nwb_converter.py`: the place where the `NWBConverter` class is defined.
 
 The directory might contain other files that are necessary for the conversion but those are the central ones.
 
@@ -73,15 +68,16 @@ pip install -r src/jazayeri_lab_to_nwb/watters/watters_requirements.txt
 
 You can run a specific conversion with the following command:
 ```
-python src/jazayeri_lab_to_nwb/watters/watters_convert_session.py
+python src/jazayeri_lab_to_nwb/watters/main_convert_session.py $SUBJECT $SESSION
 ```
 
 ### Watters working memory task data
-The conversion function for this experiment, `session_to_nwb`, is found in `src/watters/watters_convert_session.py`. The function takes three arguments:
-* `data_dir_path` points to the root directory for the data for a given session.
-* `output_dir_path` points to where the converted data should be saved.
+The conversion function for this experiment, `session_to_nwb`, is found in `src/watters/main_convert_session.py`. The function takes arguments:
+* `subject` subject name, either `'Perle'` or `'Elgar'`.
+* `session` session date in format `'YYYY-MM-DD'`.
 * `stub_test` indicates whether only a small portion of the data should be saved (mainly used by us for testing purposes).
-* `overwrite` indicates whether existing NWB files at the auto-generated output file paths should be overwritten.
+* `overwrite` indicates whether to overwrite nwb output files.
+* `dandiset_id` optional dandiset ID.
 
 The function can be imported in a separate script with and run, or you can run the file directly and specify the arguments in the `if name == "__main__"` block at the bottom.
 
@@ -111,8 +107,8 @@ The function expects the raw data in `data_dir_path` to follow this structure:
         └── spikeglx
     ...
 
-The conversion will try to automatically fetch metadata from the provided data directory. However, some information, such as the subject's name and age, must be specified by the user in the file `src/jazayeri_lab_to_nwb/watters/watters_metadata.yaml`. If any of the automatically fetched metadata is incorrect, it can also be overriden from this file.
+The conversion will try to automatically fetch metadata from the provided data directory. However, some information, such as the subject's name and age, must be specified by the user in the file `src/jazayeri_lab_to_nwb/watters/metadata.yaml`. If any of the automatically fetched metadata is incorrect, it can also be overriden from this file.
 
 The converted data will be saved in two files, one called `{session_id}_raw.nwb`, which contains the raw electrophysiology data from the Neuropixels and V-Probes, and one called `{session_id}_processed.nwb` with behavioral data, trial info, and sorted unit spiking.
 
-If you run into memory issues when writing the `{session_id}_raw.nwb` files, you may want to set `buffer_gb` to a value smaller than 1 (its default) in the `conversion_options` dicts for the recording interfaces, i.e. [here](https://github.com/catalystneuro/jazayeri-lab-to-nwb/blob/vprobe_dev/src/jazayeri_lab_to_nwb/watters/watters_convert_session.py#L49) and [here](https://github.com/catalystneuro/jazayeri-lab-to-nwb/blob/vprobe_dev/src/jazayeri_lab_to_nwb/watters/watters_convert_session.py#L71).
+If you run into memory issues when writing the `{session_id}_raw.nwb` files, you may want to set `buffer_gb` to a value smaller than 1 (its default) in the `conversion_options` dicts for the recording interfaces, i.e. [here](https://github.com/catalystneuro/jazayeri-lab-to-nwb/blob/vprobe_dev/src/jazayeri_lab_to_nwb/watters/main_convert_session.py#L189).

src/jazayeri_lab_to_nwb/watters/__init__.py

@@ -1,4 +1,4 @@
-from .wattersbehaviorinterface import WattersEyePositionInterface, WattersPupilSizeInterface
-from .watterstrialsinterface import WattersTrialsInterface
-from .wattersrecordinginterface import WattersDatRecordingInterface
-from .wattersnwbconverter import WattersNWBConverter
+from .behavior_interface import EyePositionInterface, PupilSizeInterface
+from .trials_interface import TrialsInterface
+from .recording_interface import DatRecordingInterface
+from .nwb_converter import NWBConverter

src/jazayeri_lab_to_nwb/watters/wattersbehaviorinterface.py → src/jazayeri_lab_to_nwb/watters/behavior_interface.py

@@ -26,8 +26,8 @@ def set_aligned_timestamps(self, aligned_timestamps: np.ndarray) -> None:
         self.timestamps = aligned_timestamps
 
 
-class WattersEyePositionInterface(NumpyTemporalAlignmentMixin, BaseTemporalAlignmentInterface):
-    """Eye position interface for Watters conversion"""
+class EyePositionInterface(NumpyTemporalAlignmentMixin, BaseTemporalAlignmentInterface):
+    """Eye position interface."""
 
     def __init__(self, folder_path: FolderPathType):
         # initialize interface
@@ -83,8 +83,8 @@ def add_to_nwbfile(self, nwbfile: NWBFile, metadata: dict):
         return nwbfile
 
 
-class WattersPupilSizeInterface(NumpyTemporalAlignmentMixin, BaseTemporalAlignmentInterface):
-    """Pupil size interface for Watters conversion"""
+class PupilSizeInterface(NumpyTemporalAlignmentMixin, BaseTemporalAlignmentInterface):
+    """Pupil size interface."""
 
     def __init__(self, folder_path: FolderPathType):
         # initialize interface with timestamps

src/jazayeri_lab_to_nwb/watters/get_session_paths.py

@@ -0,0 +1,71 @@
+"""Function for getting paths to data on openmind."""
+
+import collections
+import pathlib
+
+SUBJECT_NAME_TO_ID = {
+    'Perle': 'monkey0',
+    'Elgar': 'monkey1',
+}
+
+SessionPaths = collections.namedtuple(
+    'SessionPaths',
+    [
+        'output',
+        'raw_data',
+        'data_open_source',
+        'sync_pulses',
+        'spike_sorting_raw',
+    ],
+)
+
+
+def get_session_paths(subject, session, stub_test=False):
+    """Get paths to all components of the data.
+
+    Returns:
+        SessionPaths namedtuple.
+    """
+    subject_id = SUBJECT_NAME_TO_ID[subject]
+
+    # Path to write output nwb files to
+    output_path = (
+        f'/om/user/nwatters/nwb_data_multi_prediction/{subject}/{session}'
+    )
+    if stub_test:
+        output_path = f'{output_path}/stub'
+
+    # Path to the raw data. This is used for reading raw physiology data.
+    raw_data_path = (
+        f'/om4/group/jazlab/nwatters/multi_prediction/phys_data/{subject}/'
+        f'{session}/raw_data'
+    )
+
+    # Path to open-source data. This is used for reading behavior and task data.
+    data_open_source_path = (
+        '/om4/group/jazlab/nwatters/multi_prediction/datasets/data_open_source/'
+        f'Subjects/{subject_id}/{session}/001'
+    )
+
+    # Path to sync pulses. This is used for reading timescale transformations
+    # between physiology and mworks data streams.
+    sync_pulses_path = (
+        '/om4/group/jazlab/nwatters/multi_prediction/data_processed/'
+        f'{subject}/{session}/sync_pulses'
+    )
+
+    # Path to spike sorting. This is used for reading spike sorted data.
+    spike_sorting_raw_path = (
+        f'/om4/group/jazlab/nwatters/multi_prediction/phys_data/{subject}/'
+        f'{session}/spike_sorting'
+    )
+
+    session_paths = SessionPaths(
+        output=pathlib.Path(output_path),
+        raw_data=pathlib.Path(raw_data_path),
+        data_open_source=pathlib.Path(data_open_source_path),
+        sync_pulses=pathlib.Path(sync_pulses_path),
+        spike_sorting_raw=pathlib.Path(spike_sorting_raw_path),
+    )
+
+    return session_paths