Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-Session Tests and Tutorial #709

Merged
merged 25 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8b7b8b2
Add multi-session workflow, screenshots, and tutorial
garrettmflynn Mar 28, 2024
8885005
Update conf.py
garrettmflynn Mar 28, 2024
7064c4f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 28, 2024
1040f25
Merge branch 'singlesession-tutorial' into multisession-tutorial
garrettmflynn Mar 28, 2024
e05a670
Link to path expansion docs
garrettmflynn Mar 28, 2024
cb03af3
Merge branch 'main' into multisession-tutorial
CodyCBakerPhD Mar 28, 2024
5669d54
Merge branch 'main' into multisession-tutorial
CodyCBakerPhD Mar 29, 2024
4773cc5
Add workflow page screenshot. Update subject configuration
garrettmflynn Mar 31, 2024
aca82b6
Apply suggestions from code review
garrettmflynn Mar 31, 2024
d50cfbe
Clean up the final few sections of the multi-session workflow
garrettmflynn Mar 31, 2024
7b06f9d
Merge branch 'multisession-tutorial' of https://github.com/NeurodataW…
garrettmflynn Mar 31, 2024
454b6c5
Regenerate screenshots and manually select relevant changes
garrettmflynn Mar 31, 2024
5ca20a3
Merge branch 'main' into multisession-tutorial
CodyCBakerPhD Apr 1, 2024
7ac398a
Apply suggestions from code review
garrettmflynn Apr 3, 2024
2195fca
Merge branch 'ensure-workflow-persistence' into multisession-tutorial
garrettmflynn Apr 3, 2024
34ad812
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 3, 2024
b270aee
Update screenshots after workflow re-rendering
garrettmflynn Apr 3, 2024
42c8d7b
Merge branch 'multisession-tutorial' of https://github.com/NeurodataW…
garrettmflynn Apr 3, 2024
2979f0f
Show autocomplete
garrettmflynn Apr 3, 2024
fa55c01
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 3, 2024
52b1335
Merge branch 'main' into multisession-tutorial
CodyCBakerPhD Apr 3, 2024
2df5c53
Update single_session.rst
garrettmflynn Apr 3, 2024
017a395
Update multiple_sessions.rst
garrettmflynn Apr 3, 2024
5023b8f
Merge branch 'main' into multisession-tutorial
CodyCBakerPhD Apr 3, 2024
1316566
Merge branch 'main' into multisession-tutorial
CodyCBakerPhD Apr 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/assets/tutorials/home-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/fail-name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/formats-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/info-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/inspect-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/intro-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/metadata-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/preview-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/subject-error.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/subject-page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/valid-name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/tutorials/multiple/workflow-page.png
Binary file modified docs/assets/tutorials/single/fail-name.png
Binary file modified docs/assets/tutorials/single/format-options.png
Binary file modified docs/assets/tutorials/single/home-page-complete.png
Binary file modified docs/assets/tutorials/single/inspect-page.png
Binary file modified docs/assets/tutorials/single/intro-page.png
CodyCBakerPhD marked this conversation as resolved.
Show resolved Hide resolved
Binary file modified docs/assets/tutorials/single/metadata-nwbfile.png
Binary file modified docs/assets/tutorials/single/metadata-subject-complete.png
Binary file modified docs/assets/tutorials/single/preview-page.png
Binary file modified docs/assets/tutorials/single/search-behavior.png
1 change: 1 addition & 0 deletions docs/conf_extlinks.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"pynwb-docs": ("https://pynwb.readthedocs.io/en/stable/%s", "%s"),
"matnwb-src": ("https://github.com/NeurodataWithoutBorders/matnwb/%s", "%s"),
"nwb-overview": ("https://nwb-overview.readthedocs.io/en/latest/%s", "%s"),
"path-expansion-guide": ("https://neuroconv.readthedocs.io/en/main/user_guide/expand_path.html%s", "%s"),
"conda-install": (
"https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation%s",
"%s",
Expand Down
108 changes: 107 additions & 1 deletion docs/tutorials/multiple_sessions.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,109 @@
Managing Multiple Sessions
==========================
Coming soon...

Now, let’s say that you’ve already run some of your experiments and now you want to convert them all at the same time. This is where a multi-session workflow will come in handy.

Workflow Configuration
----------------------

On the Workflow page, confirm that this pipeline will be run on multiple sessions. After this, also select that you’d like to locate the source data programmatically and skip dataset publication.

.. figure:: ../assets/tutorials/multiple/workflow-page.png
:align: center
:alt: Workflow page with multiple sessions and locate data selected

Complete the first section of the GUIDE as normal until you reach a new **Locate Data** page after the Data Formats page.

Locate Data
-----------
This page helps you automatically identify source data for multiple subjects / sessions as long as your files are organized consistently.

.. figure:: ../assets/tutorials/multiple/pathexpansion-page.png
:align: center
:alt: Blank path expansion page

File locations are specified as **format strings** that define source data paths of each selected data format.

.. note::
Format strings are one component of NeuroConv's **path expansion language**, which has some nifty features for manually specifying complex paths. Complete documentation of the path expansion feature of NeuroConv can be found :path-expansion-guide:`here <>` .

While you don’t have to specify format strings for all of the pipeline’s data formats, we’re going to find all of our data here for this tutorial. You'll always be able to confirm or manually select the final paths on the Source Data page later in the workflow.

Format strings are specified using two components: the **base directory**, which is the directory to search in, and the **format string path**, where the source data is within that directory.

Given the structure of the tutorial dataset, we’ll select **~/NWB_GUIDE/test-data/dataset** as the base_directory, where `~` is the home directory of your system.
garrettmflynn marked this conversation as resolved.
Show resolved Hide resolved

We can take advantage of the **Autocomplete** feature of this page. Instead of manually filling out the format string, click the Autocomplete button and provide an example source data path (for example, the ``mouse1_Session2_g0_t0.imec0.lf.bin`` file for SpikeGLX). Then, indicate the Subject (``mouse1``) and Session ID (``Session2``) for this particular path. When you submit this form, you’ll notice that the Format String Path input has been auto-filled with a pattern for all the sessions.
garrettmflynn marked this conversation as resolved.
Show resolved Hide resolved
garrettmflynn marked this conversation as resolved.
Show resolved Hide resolved

Repeat this process for Phy, where mouse1_Session2_phy will be the example source data path.
garrettmflynn marked this conversation as resolved.
Show resolved Hide resolved

.. figure:: ../assets/tutorials/multiple/pathexpansion-completed.png
:align: center
:alt: Blank path expansion page

Advance to the next page when you have entered the data locations for both formats.

Subject Metadata
----------------
On this page you’ll edit subject-level metadata across all related sessions. Unlike the previous few pages, you’ll notice that
Sex and Species both have gray asterisks next to their name; this means they are **loose requirements**, which aren’t currently required
but could later block progress if left unspecified.

.. figure:: ../assets/tutorials/multiple/subject-page.png
:align: center
:alt: Blank subject table

In this case, we have two subjects with two sessions each. Let’s say that each of their sessions happened close enough in time that they can be identified using the same **age** entry: ``P29W`` for ``mouse1`` and ``P30W`` for ``mouse2``.

We should also indicate the ``sex`` of each subject since this is a requirement for `uploading to the DANDI Archive <https://www.dandiarchive.org/handbook/135_validation/#missing-dandi-metadata>`_.

.. figure:: ../assets/tutorials/multiple/subject-complete.png
:align: center
:alt: Complete subject table

.. note::
If you're trying to specify metadata that is shared across sessions, you can use the **Global Metadata** feature.

Pressing the Edit Global Metadata button at the top of the page will show a pop-up form which allows you to provide a
single default value for each property, as long as it’s expected not to be unique.

These values will take effect as soon as the pop-up form has been submitted.

While Global Metadata is less relevant when we’re working with two subjeccts, this feature can be very powerful when you’re working with tens or even hundreds of subjects in one conversion.

We recommend using Global Metadata to correct issues caught by the **NWB Inspector** that are seen across several sessions.

You’ll be able to specify Global Metadata on the Source Data and File Metadata pages as well.


Source Data Information
-----------------------
Because we used the Locate Data page to programmatically identify our source data, this page should mostly be complete. You can use this opportunity to verify that the identified paths appear as expected for each session.

.. figure:: ../assets/tutorials/multiple/sourcedata-page.png
:align: center
:alt: Complete source data forms

One notable difference between this and the single-session workflow, however, is that the next few pages will allow you to toggle between sessions using the **session manager** sidebar on the left.

Session Metadata
garrettmflynn marked this conversation as resolved.
Show resolved Hide resolved
----------------
Aside from the session manager and global metadata features noted above, the file metadata page in the multi-session workflow is nearly identical to the single-session version.

.. figure:: ../assets/tutorials/multiple/metadata-nwbfile.png
:align: center
:alt: Complete General Metadata form

A complete General Metadata form

Acting as global metadata, the information supplied on the subject metadata page has pre-filled the Subject metadata for each session.

.. figure:: ../assets/tutorials/multiple/metadata-subject-complete.png
:align: center
:alt: Complete Subject metadata form

A complete Subject metadata form

Finish the rest of the workflow as you would for a single session by completing a full conversion after you review the preview files with the NWB Inspector and Neurosift.

Congratulations on completing your first multi-session conversion! You can now convert multiple sessions at once, saving you time and effort.
6 changes: 3 additions & 3 deletions docs/tutorials/single_session.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,18 +105,18 @@ Session Metadata
^^^^^^^^^^^^^^^^
The file metadata page is a great opportunity to add rich annotations to the file, which will be read by anyone reusing your data in the future!

The Session Start Time in the General Metadata section is already specified because this field was automatically extracted from the SpikeGLX source data.
The Session Start Time in the **General Metadata** section is already specified because this field was automatically extracted from the SpikeGLX source data.

.. figure:: ../assets/tutorials/single/metadata-nwbfile.png
:align: center
:alt: Metadata page with invalid Subject information


However, we still need to add the Subject information—as noted by the red accents around that item. Let’s say that our subject is a male mouse with an age of P30D.
However, we still need to add the Subject information—as noted by the red accents around that item. Let’s say that our subject is a male mouse with an age of P25W.

.. figure:: ../assets/tutorials/single/metadata-subject-complete.png
:align: center
:alt: Metadata page with valid Subject information
:alt: Metadata page with valid **Subject** information

The status of the Subject information will update in real-time as you fill out the form.

Expand Down
2 changes: 2 additions & 0 deletions src/renderer/src/stories/Main.js
Original file line number Diff line number Diff line change
Expand Up @@ -186,8 +186,10 @@ export class Main extends LitElement {
}

const headerEl = header ? (this.header = new GuidedHeader(header)) : html`<div></div>`; // Render for grid
if (!header) delete this.header; // Reset header

const footerEl = footer ? (this.footer = new GuidedFooter(footer)) : html`<div></div>`; // Render for grid
if (!footer) delete this.footer; // Reset footer

const title = header?.title ?? page.info?.title;

Expand Down
2 changes: 1 addition & 1 deletion src/renderer/src/stories/pages/guided-mode/GuidedFooter.js
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export class GuidedFooter extends LitElement {
}

updated() {
this.to = (transition) => this.parentElement.to(transition);
this.to = (transition) => this.parentElement.to(transition); // Run main page's transition function
}

render() {
Expand Down
21 changes: 18 additions & 3 deletions tests/e2e/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,24 @@ export const testInterfaceInfo = {
}

export const subjectInfo = {
sex: 'M',
species: 'Mus musculus',
age: 'P30D'
common: {
sex: 'M',
species: 'Mus musculus',
},

single: {
age: 'P25W'
},

multiple: {
mouse1: {
age: 'P29W',
sex: 'F'
},
mouse2: {
age: 'P30W'
}
}
}

// export const regenerateTestData = !existsSync(testDataRootPath) || false // Generate only if doesn't exist
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/e2e.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ describe('E2E Test', () => {
})
})

describe.skip('Complete a multi-session workflow', async () => {
describe('Complete a multi-session workflow', async () => {
const subdirectory = 'multiple'
await runWorkflow('Multi Session Workflow', { upload_to_dandi: false, multiple_sessions: true, locate_data: true }, subdirectory)

Expand Down
11 changes: 7 additions & 4 deletions tests/e2e/workflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export default async function runWorkflow (name, workflow, identifier) {
// Advance to instructions page
await toNextPage('start')

await takeScreenshot(join(identifier, 'intro-page'), 300)
await takeScreenshot(join(identifier, 'intro-page'), 500)

// Advance to general information page
await toNextPage('details')
Expand Down Expand Up @@ -179,7 +179,7 @@ export default async function runWorkflow (name, workflow, identifier) {

await takeScreenshot(join(identifier, 'subject-error'), 500)

await evaluate(( subjectInfo ) => {
await evaluate(( { common, multiple } ) => {
const dashboard = document.querySelector('nwb-dashboard')

const page = dashboard.page
Expand All @@ -191,6 +191,7 @@ export default async function runWorkflow (name, workflow, identifier) {


for (let name in data) {
const subjectInfo = { ...common, ...( multiple[name] ?? {} )}
data[name] = { ...data[name], ...subjectInfo }
}

Expand Down Expand Up @@ -276,12 +277,14 @@ export default async function runWorkflow (name, workflow, identifier) {
if (!willProvideSubjectInfo) {

// Update for single session
await evaluate((subjectInfo) => {
await evaluate(({ common, single = {} }) => {
const dashboard = document.querySelector('nwb-dashboard')
const page = dashboard.page
const firstSessionForm = page.forms[0].form
const form = firstSessionForm.forms['Subject']

const subjectInfo = { ...common, ...single }

for (let key in subjectInfo) {
const input = form.getFormElement([key])
input.updateData(subjectInfo[key])
Expand Down Expand Up @@ -309,7 +312,7 @@ export default async function runWorkflow (name, workflow, identifier) {

test('Review NWB Inspector output', async () => {

await takeScreenshot(join(identifier, 'inspect-page'), 2000) // Finish file inspection
await takeScreenshot(join(identifier, 'inspect-page'), 5000) // Finish file inspection and allow full load of Neurosift page
await toNextPage('preview')

})
Expand Down
Loading