Multi-Session Tests and Tutorial

docs/conf_extlinks.py

@@ -5,6 +5,7 @@
     "pynwb-docs": ("https://pynwb.readthedocs.io/en/stable/%s", "%s"),
     "matnwb-src": ("https://github.com/NeurodataWithoutBorders/matnwb/%s", "%s"),
     "nwb-overview": ("https://nwb-overview.readthedocs.io/en/latest/%s", "%s"),
+    "path-expansion-guide": ("https://neuroconv.readthedocs.io/en/main/user_guide/expand_path.html%s", "%s"),
     "conda-install": (
         "https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation%s",
         "%s",

docs/tutorials/multiple_sessions.rst

@@ -1,3 +1,109 @@
 Managing Multiple Sessions
 ==========================
-Coming soon...
+
+Now, let’s say that you’ve fallen behind and have a weeks worth of experiments to convert. This is where a multi-session workflow will come in handy.
+
+Workflow Configuration
+----------------------
+
+On the Workflow page, confirm that this pipeline will be run on multiple sessions. After this, also select that you’d like to locate the source data programmatically.
+
+Complete the first section of the GUIDE as normal until you reach a new **Locate Data** page after the Data Formats page.
+
+Locate Data
+-----------
+This page helps you automatically identify source data for multiple subjects / sessions as long as your files are organized consistently.
+
+.. figure:: ../assets/tutorials/multiple/pathexpansion-page.png
+  :align: center
+  :alt: Blank path expansion page
+
+File locations are specified as **format strings** that define source data paths of each selected data format.
+
+.. note::
+    Format strings are one component of NeuroConv's **path expansion language**, which has some nifty features for manually specifying complex paths. Complete documentation of the path expansion feature of NeuroConv can be found :path-expansion-guide:`here <>` .
+
+While you don’t have to specify a format strings for all of the pipeline’s data formats, we’re going to find all of our data here for this tutorial.
+
+Format strings are specified using two components: the **base directory**, which is the directory to search in, and the **format string path**, where the source data is within that directory.
+
+Given the structure of the tutorial dataset, we’ll select **~/NWB_GUIDE/test-data/dataset** as the base_directory.
+
+We can take advantage of the **Autocomplete** feature of this page. Instead of manually filling out the format string, click the Autocomplete button and provide an example source data path (e.g. the mouse1_Session2_g0_t0.imec0.lf.bin file for SpikeGLX). Then, indicate the Subject  (mouse1) and Session ID (Session2) for this particular path. When you submit this form, you’ll notice that the Format String Path input has been auto-filled with a pattern for all the sessions.
+
+Repeat this process for Phy, where mouse1_Session2_phy will be the example source data path.
+
+.. figure:: ../assets/tutorials/multiple/pathexpansion-completed.png
+  :align: center
+  :alt: Blank path expansion page
+
+Advance to the next page when you have entered the data locations for both formats.
+
+Subject Metadata
+----------------
+On this page you’ll edit subject-level metadata across all related sessions. Unlike the previous few pages, you’ll notice that
+Sex and Species both have gray asterisks next to their name; this means they are **loose requirements**, which aren’t currently required
+but could later block progress if left unspecified.
+
+.. figure:: ../assets/tutorials/multiple/subject-page.png
+  :align: center
+  :alt: Blank subject table
+
+In this case, we have two subjects with two sessions each. Let’s say that each of their sessions happened close enough in time
+that they can be identified using the same **age** entry: P29W for mouse1 and P30W for mouse2.
+
+Let’s also say that both of our subjects share some properties: both are male mice.
+
+.. figure:: ../assets/tutorials/multiple/subject-complete.png
+  :align: center
+  :alt: Complete subject table
+
+.. note::
+    If you're trying to specify metadata that is shared across sessions, you can use the **Global Metadata** feature.
+
+    Pressing the Edit Global Metadata button at the top of the page will show a pop-up form which allows you to provide a
+    single default value for each property, as long as it’s expected not to be unique.
+
+    These values will take effect as soon as the pop-up form has been submitted.
+
+    While Global Metadata is less relevant when we’re working with two subjeccts, this feature can be very powerful when you’re working with tens or even hundreds of subjects in one conversion.
+
+    We recommend using Global Metadata to correct issues caught by the **NWB Inspector** that are seen across several sessions.
+
+    You’ll be able to specify Global Metadata on the Source Data and File Metadata pages as well.
+
+Now that we’ve specified all of the required and suggested metadata—which will be required later—we can move onto the next page.
+
+Source Data Information
+-----------------------
+Because we used the Locate Data page to programmatically identify our source data, this page will largely be complete.
+
+.. figure:: ../assets/tutorials/multiple/sourcedata-page.png
+  :align: center
+  :alt: Complete source data forms
+
+
+Session Metadata
+----------------
+The file metadata page is a great opportunity to add rich annotations to the file, which will be read by anyone reusing your data in the future!
+
+In conjunction with the subject information declared previously, all the required metadata to convert our dataset to NWB has already been specified—particularly because the Session Start Time has automatically been extracted from the SpikeGLX source data.
+
+.. figure:: ../assets/tutorials/multiple/metadata-nwbfile.png
+  :align: center
+  :alt: Complete General Metadata form
+
+  A complete General Metadata form
+
+.. figure:: ../assets/tutorials/multiple/metadata-subject-complete.png
+  :align: center
+  :alt: Complete Subject metadata form
+
+  A complete Subject metadata form
+
+
+When advancing to the next page, a progress bar will begin to show how many files are left to run a stub conversion on.
+
+Continuing Forward
+------------------
+The rest of this workflow is identical to the single-session version, except that multiple sessions will be available to view on subsequent pages.

docs/tutorials/single_session.rst

@@ -112,7 +112,7 @@ The Session Start Time in the General Metadata section is already specified beca
   :alt: Metadata page with invalid Subject information
 
 
-However, we still need to add the Subject information—as noted by the red accents around that item. Let’s say that our subject is a male mouse with an age of P30D.
+However, we still need to add the Subject information—as noted by the red accents around that item. Let’s say that our subject is a male mouse with an age of P25W.
 
 .. figure:: ../assets/tutorials/single/metadata-subject-complete.png
   :align: center

src/renderer/src/stories/Main.js

@@ -186,8 +186,10 @@ export class Main extends LitElement {
         }
 
         const headerEl = header ? (this.header = new GuidedHeader(header)) : html`<div></div>`; // Render for grid
+        if (!header) delete this.header; // Reset header
 
         const footerEl = footer ? (this.footer = new GuidedFooter(footer)) : html`<div></div>`; // Render for grid
+        if (!footer) delete this.footer; // Reset footer
 
         const title = header?.title ?? page.info?.title;
 

src/renderer/src/stories/pages/guided-mode/GuidedFooter.js

@@ -48,7 +48,7 @@ export class GuidedFooter extends LitElement {
     }
 
     updated() {
-        this.to = (transition) => this.parentElement.to(transition);
+        this.to = (transition) => this.parentElement.to(transition); // Run main page's transition function
     }
 
     render() {

tests/e2e/config.ts

@@ -63,9 +63,21 @@ export const testInterfaceInfo = {
 }
 
 export const subjectInfo = {
-  sex: 'M',
-  species: 'Mus musculus',
-  age: 'P30D'
+  common: {
+    sex: 'M',
+    species: 'Mus musculus',
+    age: 'P30W'
+  },
+
+  single: {
+    age: 'P25W'
+  },
+
+  multiple: {
+    mouse1: {
+      age: 'P29W'
+    }
+  }
 }
 
 // export const regenerateTestData = !existsSync(testDataRootPath) || false // Generate only if doesn't exist

tests/e2e/e2e.test.ts

@@ -98,7 +98,7 @@ describe('E2E Test', () => {
       })
     })
 
-    describe.skip('Complete a multi-session workflow', async () => {
+    describe('Complete a multi-session workflow', async () => {
       const subdirectory = 'multiple'
       await runWorkflow('Multi Session Workflow', { upload_to_dandi: false, multiple_sessions: true, locate_data: true }, subdirectory)
 

tests/e2e/workflow.ts

@@ -179,7 +179,7 @@ export default async function runWorkflow (name, workflow, identifier) {
 
     await takeScreenshot(join(identifier, 'subject-error'), 500)
 
-    await evaluate(( subjectInfo ) => {
+    await evaluate(( { common, multiple } ) => {
       const dashboard = document.querySelector('nwb-dashboard')
 
       const page = dashboard.page
@@ -191,6 +191,7 @@ export default async function runWorkflow (name, workflow, identifier) {
 
 
       for (let name in data) {
+        const subjectInfo = { ...common, ...( multiple[name] ?? {} )}
         data[name] = { ...data[name], ...subjectInfo }
       }
 
@@ -276,12 +277,14 @@ export default async function runWorkflow (name, workflow, identifier) {
     if (!willProvideSubjectInfo) {
 
       // Update for single session
-      await evaluate((subjectInfo) => {
+      await evaluate(({ common, single = {} }) => {
         const dashboard = document.querySelector('nwb-dashboard')
         const page = dashboard.page
         const firstSessionForm = page.forms[0].form
         const form = firstSessionForm.forms['Subject']
 
+        const subjectInfo = { ...common, ...single }
+
         for (let key in subjectInfo) {
           const input = form.getFormElement([key])
           input.updateData(subjectInfo[key])