-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
379ad5f
commit ad79689
Showing
3 changed files
with
3,125 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2,893 changes: 2,893 additions & 0 deletions
2,893
src/constantinople_lab_to_nwb/schierek_embargo_2024/tutorials/ephys_example_notebook.ipynb
Large diffs are not rendered by default.
Oops, something went wrong.
231 changes: 231 additions & 0 deletions
231
src/constantinople_lab_to_nwb/schierek_embargo_2024/tutorials/ephys_run_session_to_nwb.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,231 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "352a2fb2-54fe-4418-ae34-eda86223c180", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Neuropixels data conversion to NWB\n", | ||
| "\n", | ||
| "This notebook demonstrates how to convert an example session from the Neuropixels dataset to NWB.\n", | ||
| "\n", | ||
| "## Neuropixels dataset\n", | ||
| "\n", | ||
| "This dataset contains in vivo extracellular electrophysiology recordings from rats performing a value-based decision-making task. Neural data were acquired using Neuropixels probes (384 channels, 30 kHz sampling rate) with Neuropix-PXI hardware and OpenEphys, and preprocessed using Kilosort 2.5 with manual curation in Phy. Trials were initiated by a nose-poke in a lit center port and required maintaining a center fixation for 0.8 to 1.2 seconds, during which a tone indicated the possible reward size. A subsequent side LED indicated the potential reward location, followed by a delay period drawn from an exponential distribution (mean = 2.5 s). Rats could opt out at any time by poking the unlit port, restarting the trial. Catch trials, where the delay period only ended if the rat opted out, constituted 15-25% of the trials. Rats received penalties for premature fixation breaks. Additionally, the tasks introduced semi-observable hidden states by varying reward statistics across uncued blocks (high, low, and mixed), structured hierarchically, with blocks transitioning after 40 successfully completed trials.\n", | ||
| "This notebook demonstrates how to convert an example session to NWB.\n", | ||
| "\n", | ||
| "This dataset have the following data streams:\n", | ||
| "- Behavior: Bpod output (.mat)\n", | ||
| "- Recording AP, LFP: OpenEphys (binary format)\n", | ||
| "- Units: Phy output\n", | ||
| "\n", | ||
| "## Notes on the conversion\n", | ||
| "\n", | ||
| "The conversion notes is located in `src/constantinople_lab_to_nwb/schierek_embargo_2024/schierek_embargo_2024_notes.md`. This file contains information about the expected file structure and the conversion process.\n", | ||
| "\n", | ||
| "## Running the conversion\n", | ||
| "\n", | ||
| "To run a specific conversion, you might need to install first some conversion specific dependencies that are located in each conversion directory:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 3, | ||
| "id": "6408c740-2030-4f51-a44b-5e6b6dece298", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "#!pip install -r ../schierek_embargo_2024_requirements.txt" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "b221cfff-c7e5-44e7-bd3a-024bf2f26f67", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Convert a single session to NWB\n", | ||
| "\n", | ||
| "The `schierek_embargo_2024_convert_session.py` script defines the `session_to_nwb` function that converts a session of Neuropixels data to NWB." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 7, | ||
| "id": "695d7f2c-5b40-4e0d-b4b7-38546a4dce52", | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "Source data is valid!\n", | ||
| "Metadata is valid!\n", | ||
| "conversion_options is valid!\n" | ||
| ] | ||
| }, | ||
| { | ||
| "name": "stderr", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "/Users/weian/catalystneuro/constantinople-lab-to-nwb/src/constantinople_lab_to_nwb/general_interfaces/bpodbehaviorinterface.py:496: UserWarning: Task argument 'HiITI' not in metadata. Skipping.\n", | ||
| " warn(f\"Task argument '{task_argument_name}' not in metadata. Skipping.\")\n", | ||
| "/Users/weian/anaconda3/envs/constantinople_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/container.py:536: UserWarning: The linked table for DynamicTableRegion 'event_type' does not share an ancestor with the DynamicTableRegion.\n", | ||
| " child._validate_on_set_parent()\n", | ||
| "/Users/weian/anaconda3/envs/constantinople_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/container.py:536: UserWarning: The linked table for DynamicTableRegion 'state_type' does not share an ancestor with the DynamicTableRegion.\n", | ||
| " child._validate_on_set_parent()\n", | ||
| "/Users/weian/anaconda3/envs/constantinople_lab_to_nwb_env/lib/python3.12/site-packages/hdmf/container.py:536: UserWarning: The linked table for DynamicTableRegion 'action_type' does not share an ancestor with the DynamicTableRegion.\n", | ||
| " child._validate_on_set_parent()\n" | ||
| ] | ||
| }, | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "NWB file saved at /Volumes/T9/Constantinople/nwbfiles/sub-J076_ephys.nwb!\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "from constantinople_lab_to_nwb.schierek_embargo_2024.schierek_embargo_2024_convert_session import session_to_nwb\n", | ||
| "\n", | ||
| "# Parameters for conversion\n", | ||
| "\n", | ||
| "# The path to the NWB file to write.\n", | ||
| "nwbfile_path = \"/Volumes/T9/Constantinople/nwbfiles/sub-J076_ephys.nwb\"\n", | ||
| "# The OpenEphys recording folder path (make sure to include the 'Record Node #' in the folder path)\n", | ||
| "folder_path = \"/Volumes/T9/Constantinople/Ephys Data/J076_2023-12-12_14-52-04/Record Node 117\"\n", | ||
| "# The name of the *raw* recording stream (e.g. )\n", | ||
| "raw_stream_name = \"Neuropix-PXI-119.ProbeA-AP\"\n", | ||
| "# The name of the *LFP* recording stream (e.g. )\n", | ||
| "lfp_stream_name = \"Neuropix-PXI-119.ProbeA-LFP\"\n", | ||
| "# The path to the processed spike sorting file (.mat). This file contains the \"SU\" named struct that contains the processed spike data.\n", | ||
| "# This file also contains the \"S\" named struct containing the processed behavior data.\n", | ||
| "spike_sorting_mat_file_path = \"/Volumes/T9/Constantinople/Ephys Data/J076_2023-12-12.mat\"\n", | ||
| "# The path to the raw Bpod data\n", | ||
| "bpod_file_path = \"/Volumes/T9/Constantinople/raw_Bpod/J076/DataFiles/J076_RWTautowait2_20231212_145250.mat\"\n", | ||
| "\n", | ||
| "# Task specific parameters\n", | ||
| "\n", | ||
| "# The column name mapping is used to rename the columns in the processed data to more descriptive column names.\n", | ||
| "# New (unseen) task parameters should be added here.\n", | ||
| "column_name_mapping = dict(\n", | ||
| " NoseInCenter=\"nose_in_center\",\n", | ||
| " TrainingStage=\"training_stage\",\n", | ||
| " Block=\"block_type\",\n", | ||
| " BlockLengthAd=\"num_trials_in_adaptation_blocks\",\n", | ||
| " BlockLengthTest=\"num_trials_in_test_blocks\",\n", | ||
| " ProbCatch=\"catch_percentage\",\n", | ||
| " RewardDelay=\"reward_delay\",\n", | ||
| " RewardAmount=\"reward_volume_ul\",\n", | ||
| " WaitForPoke=\"wait_for_center_poke\",\n", | ||
| " hits=\"is_rewarded\",\n", | ||
| " vios=\"is_violation\",\n", | ||
| " optout=\"is_opt_out\",\n", | ||
| " wait_time=\"wait_time\",\n", | ||
| " wait_thresh=\"wait_time_threshold\",\n", | ||
| " wait_for_cpoke=\"wait_for_center_poke\",\n", | ||
| " zwait_for_cpoke=\"z_scored_wait_for_center_poke\",\n", | ||
| " RewardedSide=\"rewarded_port\",\n", | ||
| " Cled=\"center_poke_times\",\n", | ||
| " Lled=\"left_poke_times\",\n", | ||
| " Rled=\"right_poke_times\",\n", | ||
| " l_opt=\"left_opt_out_times\",\n", | ||
| " r_opt=\"right_opt_out_times\",\n", | ||
| " ReactionTime=\"reaction_time\",\n", | ||
| " slrt=\"short_latency_reaction_time\",\n", | ||
| " iti=\"inter_trial_interval\",\n", | ||
| ")\n", | ||
| "# The column descriptions are used to add descriptions to the columns in the processed data.\n", | ||
| "# New (unseen) task parameter descriptions should be added here.\n", | ||
| "column_descriptions = dict(\n", | ||
| " NoseInCenter=\"The time in seconds when the animal is required to maintain center port to initiate the trial (uniformly drawn from 0.8 - 1.2 seconds).\",\n", | ||
| " TrainingStage=\"The stage of the training.\",\n", | ||
| " Block=\"The block type (High, Low or Test). High and Low blocks are high reward (20, 40, or 80μL) or low reward (5, 10, or 20μL) blocks. Test blocks are mixed blocks.\",\n", | ||
| " BlockLengthAd=\"The number of trials in each high reward (20, 40, or 80μL) or low reward (5, 10, or 20μL) blocks.\",\n", | ||
| " BlockLengthTest=\"The number of trials in each mixed blocks.\",\n", | ||
| " ProbCatch=\"The percentage of catch trials.\",\n", | ||
| " RewardDelay=\"The delay in seconds to receive reward, drawn from exponential distribution with mean = 2.5 seconds.\",\n", | ||
| " RewardAmount=\"The volume of reward in microliters.\",\n", | ||
| " hits=\"Whether the subject received reward for each trial.\",\n", | ||
| " vios=\"Whether the subject violated the trial by not maintaining center poke for the time required by 'nose_in_center'.\",\n", | ||
| " optout=\"Whether the subject opted out for each trial.\",\n", | ||
| " WaitForPoke=\"The time (s) between side port poke and center poke.\",\n", | ||
| " wait_time=\"The wait time for the subject for for each trial in seconds, after removing outliers.\"\n", | ||
| " \" For hit trials (when reward was delivered) the wait time is equal to the reward delay.\"\n", | ||
| " \" For opt-out trials, the wait time is equal to the time waited from trial start to opting out.\",\n", | ||
| " wait_for_cpoke=\"The time between side port poke and center poke in seconds, includes the time when the subject is consuming the reward.\",\n", | ||
| " zwait_for_cpoke=\"The z-scored wait_for_cpoke using all trials.\",\n", | ||
| " RewardedSide=\"The rewarded port (Left or Right) for each trial.\",\n", | ||
| " Cled=\"The time of center port LED on/off for each trial (2 x ntrials).\",\n", | ||
| " Lled=\"The time of left port LED on/off for each trial (2 x ntrials).\",\n", | ||
| " Rled=\"The time of right port LED on/off for each trial (2 x ntrials).\",\n", | ||
| " l_opt=\"The time of left port entered/exited for each trial (2 x ntrials).\",\n", | ||
| " r_opt=\"The time of right port entered/exited for each trial (2 x ntrials).\",\n", | ||
| " ReactionTime=\"The reaction time in seconds.\",\n", | ||
| " slrt=\"The short-latency reaction time in seconds.\",\n", | ||
| " iti=\"The time to initiate trial in seconds (the time between the end of the consummatory period and the time to initiate the next trial).\",\n", | ||
| " wait_thresh=\"The threshold in seconds to remove wait-times (mean + 1*std of all cumulative wait-times).\",\n", | ||
| ")\n", | ||
| "\n", | ||
| "# Optional parameters\n", | ||
| "\n", | ||
| "# Whether to run a stub test conversion. The stubbed file will only contain a small portion of data.\n", | ||
| "# When running the full conversion stub_test should be disabled.\n", | ||
| "stub_test = True\n", | ||
| "# Whether to overwrite an existing NWB file.\n", | ||
| "overwrite = True\n", | ||
| "\n", | ||
| "# Run the `session_to_nwb` function with the parameters to convert to NWB\n", | ||
| "session_to_nwb(\n", | ||
| " nwbfile_path=nwbfile_path,\n", | ||
| " openephys_recording_folder_path=folder_path,\n", | ||
| " ap_stream_name=raw_stream_name,\n", | ||
| " lfp_stream_name=lfp_stream_name,\n", | ||
| " processed_spike_sorting_file_path=spike_sorting_mat_file_path,\n", | ||
| " raw_behavior_file_path=bpod_file_path,\n", | ||
| " column_name_mapping=column_name_mapping,\n", | ||
| " column_descriptions=column_descriptions,\n", | ||
| " stub_test=stub_test,\n", | ||
| " overwrite=overwrite,\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 1, | ||
| "id": "9e70b3e5-de74-4f15-bec0-688151d80ce4", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "37140fad-10d0-436d-a4a8-452083d33d10", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3 (ipykernel)", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.12.6" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 5 | ||
| } |