The U19-pipeline_python repository defines the DataJoint tables for the U19 projects. There is a companion MATLAB DataJoint pipeline where much of this repository has mirrored table definitions.

• The following instructions will allow a user to access and fetch data from the database.

conda activate <environment_name>
cd U19-pipeline_python
pip install -e .
python initial_conf.py

(Username and password will be prompted at this moment: Princeton NETiD & NETiD password usually works)

• The initial_conf.py script will store a global file with credentials to access DB and configuration variables/filepaths.
• Now that the virtual modules are created to access the tables in the database, you can query and fetch from the database.

We have created some tutorial notebooks to help you start working with DataJoint.

1. Querying data

   • jupyter notebook notebooks/tutorials/1-Explore U19 data pipeline with DataJoint.ipynb

2. Building analysis pipelines

   • Recommended if you are going to create new databases or tables for analysis.
   • jupyter notebook notebooks/tutorials/2-Analyze data with U19 pipeline and save results.ipynb
   • jupyter notebook notebooks/tutorials/3-Build a simple data pipeline.ipynb

• There are several data files (behavior, imaging & electrophysiology) that are referenced in the database.

• To access thse files you should mount PNI file server volumes on your system.

• There are three main file servers across PNI where data is stored (braininit, Bezos & u19_dj).

  Click to expand details

  On windows systems

  • From Windows Explorer, select "Map Network Drive" and enter:
    \\cup.pni.princeton.edu\braininit\ (for braininit)
    \\cup.pni.princeton.edu\Bezos-center\ (for Bezos)
    \\cup.pni.princeton.edu\u19_dj\ (for u19_dj)
    
  • Authenticate with your NetID and PU password (NOT your PNI password, which may be different). When prompted for your username, enter PRINCETON\netid (note that PRINCETON can be upper or lower case) where netid is your PU NetID.

  On OS X systems

  • Select "Go->Connect to Server..." from Finder and enter:
    smb://cup.pni.princeton.edu/braininit/ (for braininit)
    smb://cup.pni.princeton.edu/Bezos-center/ (for Bezos)
    smb://cup.pni.princeton.edu/u19_dj/ (for u19_dj)
    
  • Authenticate with your NetID and PU password (NOT your PNI password, which may be different).

  On Linux systems

  • Follow extra steps depicted in this link: https://npcdocs.princeton.edu/index.php/Mounting_the_PNI_file_server_on_your_desktop

  Notable data

  Here are some shortcuts to common used data accross PNI

  Sue Ann's Towers Task

  • Imaging: /Bezos-center/RigData/scope/bay3/sakoay/{protocol_name}/imaging/{subject_nickname}/
  • Behavior: /braininit/RigData/scope/bay3/sakoay/{protocol_name}/data/{subject_nickname}/

  Lucas Pinto's Widefield

  • Imaging /braininit/RigData/VRwidefield/widefield/{subject_nickname}/{session_date}/
  • Behavior /braininit/RigData/VRwidefield/behavior/lucas/blocksReboot/data/{subject_nickname}/

  Lucas Pinto's Opto inactivacion experiments

  • Imaging /braininit/RigData/VRLaser/LaserGalvo1/{subject_nickname}/
  • Behavior /braininit/RigData/VRLaser/behav/lucas/blocksReboot/data/{subject_nickname}/

  Get path info for the session behavioral file

  1. Mount needed file server
  2. Connect to the Database
  3. Create a structure with subject_fullname and session_date from the session
     key['subject_fullname'] = 'koay_K65'
key['session_Date'] = '2018-02-05'
     
  4. Fetch filepath info: data_dir = (acquisition.SessionStarted & key).fetch('remote_path_behavior_file')
     

Import datajoint as follows:

import datajoint as dj

dj.Table._update(schema.Table & key, 'column_name', 'new_data')

table.heading.attributes.keys()

This also works on a query object:

schema = dj.create_virtual_module("some_schema","some_schema")
query_object = schema.Sample() & 'sample_name ="test"'
query_object.heading.attributes.keys()