datajoint · kabilar · Mar 3, 2022 · Jan 4, 2022 · Jan 5, 2022 · Jan 5, 2022
diff --git a/.gitignore b/.gitignore
@@ -123,3 +123,4 @@ docker-compose.y*ml
 
 # notes
 temp*
+*/temp*
diff --git a/Background.md b/Background.md
@@ -0,0 +1,194 @@
+# Trial Behavior Element
+
+## Pipeline Development
+Initial Element development began in December of 2021 in discussion with the team led 
+by Adam Kepecs at Washington University School of Medicine in St. Louis. Micheal Wulf 
+has been instrumental our understanding of the diversity of Bpod files. DataJoint provides
+a diverse set of bpod examples via [DJArchive](https://github.com/datajoint/djarchive-client/).
+
+<details>
+<summary>Click to expand notes on the organization of these files</summary>
+
+```
+The RAW prefix in bpod structures always has data elsewhere
+    Raw is where it is initially stored, and then ported elsewhere during export
+
+Inconsistent across examples:
+    BalbC top level: LeftValveTime, RightValveTime, LeftAmount, RightAmount,
+                     changebridge, bridgepos, subject, delays
+    1119a top level: Notes, MarkerCodes
+
+Custom is determined by user:
+    1119a Custom: contrastLevels, dbSplTrialType3, dbSplTrialType6,startSessionDatetime,
+                  startSessionTimeStamp,TQ03_Dual2AFC_Jun18_2021_Session1.mat,
+                  BlockNumber, BlockTrial, ChoiceLeft, ChoiceCorrect, Feedback,
+                  FeedbackTimeFixBroke, EarlyWithdrawal, FixDur, MT, CatchTrial,
+                  OdorFracA, OdorID, OdorPair, ST, ResolutionTime, Rewarded,
+                  RewardMagnitude, number_trials, LaserTrial,LaserTrialTrainStart,
+                  AuditoryTrial, ClickTask,OlfactometerStartup, PsychtoolboxStartup,
+                  AuditoryOmega,LeftClickRate, RightClickRate, LeftClickTrain,
+                  RightClickTrain, LeftRewarded, DV, Rig, Subject,
+                  PulsePalParamStimulus, PulsePalParamFeedback, StimDelay,
+                  FeedbackDelay, MinSampleAud,
+    TP24 Custom: BlockNumber, BlockTrial, ChoiceLeft, ChoiceCorrect, Feedback,
+                 FeedbackTime, FixBroke, EarlyWithdrawal, FixDur, MT,CatchTrial,
+                 OdorFracA, OdorID, OdorPair, ST, Rewarded,RewardMagnitude,
+                 number_trials, LaserTrial, LaserTrialTrainStart,AuditoryTrial,
+                 ClickTask, OlfactometerStartup,PsychtoolboxStartup, AuditoryOmega,
+                 LeftClickRate,RightClickRate, LeftClickTrain, RightClickTrain,
+                 LeftRewarded,DV, Rig, Subject, PulsePalParamStimulus,
+                 PulsePalParamFeedback,StimDelay, FeedbackDelay, MinSampleAud
+
+Settings is inconsistently used:
+    BalcC: Nothing
+    TQ03 : same as TrialSettings
+
+TrialSettings is also file-specific:
+    BalbC TrialSettings: Subject, RewardAmount, RewardDelay, ChangeOver, block, depleft,
+                         depright, MaxTrials, randomDelay, Lange, Bridge, Drugs
+    1119A TrialSettings: MaxTrialNum, StartPhaseLength, TrialType1, TrialType2,
+                         TrialType3, TrialType4, TrialType5, TrialType6, TrialType7,
+                         RewardAmount, ContrastLevelMin, ContrastLevelMax, UseBonsai,
+                         UsePulsePal, PulsePalComPort, PulsePalParameterFile,
+                         UseOptoGeneticPulseLength, NumberOfPulses, PulseFrequency,
+                         Bpod_BNC_Ch_Opto, PulsePal_Trig_Ch_Opto, PulsePal_Out_Ch_Opto,
+                         Opto_Stim_In_TrialType2, Opto_Stim_In_TrialType3,
+                         Opto_Stim_In_TrialType5, Opto_Stim_In_TrialType6,
+                         UseSoundStimulation, dB_SPL_min, dB_SPL_max, dB_SPL_step,
+                         Photometry, DbleFibers, Isobestic405, RedChannel,
+                         PhotometryVersion, Modulation, NidaqDuration,
+                         NidaqSamplingRate, DecimateFactor, LED1_Name, LED1_Amp,
+                         LED1_Freq, LED2_Name, LED2_Amp, LED2_Freq, LED1b_Name,
+                         LED1b_Amp, LED1b_Freq
+    TQ03_ TrialSettings: Aud_Levels, Aud_NoEvidence, Aud_Ramp, Aud_SamplingRate,
+                         Aud_ToneDuration, Aud_ToneOverlap, Aud_UseMiddleOctave,
+                         Aud_Volume, Aud_maxFreq, Aud_minFreq, Aud_nFreq,
+                         AuditoryAlpha, AuditoryStimulusTime, AuditoryStimulusType,
+                         BaselineBegin, BaselineEnd, BlockTable, CatchError,
+                         CenterWaitMax, ChoiceDeadLine, DbleFibers, DecimateFactor,
+                         DrinkingGrace, DrinkingTime, FeedbackDelay, FeedbackDelayDecr,
+                         FeedbackDelayGrace, FeedbackDelayIncr, FeedbackDelayMax,
+                         FeedbackDelayMin, FeedbackDelaySelection, FeedbackDelayTau,
+                         ITI, IncorrectChoiceFeedbackType, Isobestic405, LED1_Amp,
+                         LED1_Freq, LED1_Name, LED1b_Amp, LED1b_Freq, LED1b_Name,
+                         LED2_Amp, LED2_Freq, LED2_Name, LaserAmp, LaserFeedback,
+                         LaserITI, LaserMov, LaserPreStim, LaserPulseDuration_ms,
+                         LaserRampDuration_ms, LaserRew, LaserSoftCode, LaserStim,
+                         LaserStimFreq, LaserTimeInvestment, LaserTrainDuration_ms,
+                         LaserTrainRandStart, LaserTrainStartMax_s,
+                         LaserTrainStartMin_s, LaserTrials, LeftBiasAud, MaxSessionTime,
+                         MinSampleAud, MinSampleAudAutoincrement, MinSampleAudDecr,
+                         MinSampleAudIncr, MinSampleAudMax, MinSampleAudMin, Modulation,
+                         NidaqDuration, NidaqMax, NidaqMin, NidaqSamplingRate,
+                         OdorA_bank, OdorB_bank, OdorStimulusTimeMin, OdorTable,
+                         Percent50Fifty, PercentAuditory, PercentCatch, PhotoPlotReward,
+                         PhotoPlotSidePokeIn, PhotoPlotSidePokeLeave, Photometry,
+                         PhotometryVersion, PortLEDs, Ports_LMR, PreITI, RedChannel,
+                         RewardAmount, ShowFeedback, ShowFix, ShowPsycAud, ShowPsycOlf,
+                         ShowST, ShowTrialRate, ShowVevaiometric,
+                         SkippedFeedbackFeedbackType, StartEasyTrials, StimDelay,
+                         StimDelayAutoincrement, StimDelayDecr, StimDelayIncr,
+                         StimDelayMax, StimDelayMin, SumRates, TimeMax, TimeMin,
+                         TimeOutBrokeFixation, TimeOutEarlyWithdrawal,
+                         TimeOutIncorrectChoice, TimeOutSkippedFeedback, TrialSelection,
+                         VevaiometricMinWT, VevaiometricNBin, VevaiometricShowPoints,
+                         VideoTrials, Wire1VideoTrigger, nidaqDev
+    TP24_ TrialSettings: Aud_Levels, Aud_NoEvidence, Aud_Ramp, Aud_SamplingRate,
+                         Aud_ToneDuration, Aud_ToneOverlap, Aud_UseMiddleOctave,
+                         Aud_Volume, Aud_maxFreq, Aud_minFreq, Aud_nFreq,
+                         AuditoryAlpha, AuditoryStimulusTime, AuditoryStimulusType,
+                         BlockTable, CatchError, ChoiceDeadLine, FeedbackDelay,
+                         FeedbackDelayDecr, FeedbackDelayGrace, FeedbackDelayIncr,
+                         FeedbackDelayMax, FeedbackDelayMin, FeedbackDelaySelection,
+                         FeedbackDelayTau, ITI, IncorrectChoiceFeedbackType,
+                         LaserFeedback, LaserITI, LaserMov, LaserPreStim,
+                         LaserPulseDuration_ms, LaserRew, LaserStim, LaserStimFreq,
+                         LaserTimeInvestment, LaserTrainDuration_ms, LaserTrainRandStart,
+                         LaserTrainStartMax_s, LaserTrainStartMin_s, LaserTrials,
+                         LeftBiasAud, MaxSessionTime, MinSampleAud,
+                         MinSampleAudAutoincrement, MinSampleAudDecr, MinSampleAudIncr,
+                         MinSampleAudMax, MinSampleAudMin, OdorA_bank, OdorB_bank,
+                         OdorStimulusTimeMin, OdorTable, Percent50Fifty,
+                         PercentAuditory, PercentCatch, PortLEDs, Ports_LMR,
+                         RewardAmount, ShowFeedback, ShowFix, ShowPsycAud, ShowPsycOlf,
+                         ShowST, ShowTrialRate, ShowVevaiometric,
+                         SkippedFeedbackFeedbackType, StartEasyTrials, StimDelay,
+                         StimDelayAutoincrement, StimDelayDecr, StimDelayIncr,
+                         StimDelayMax, StimDelayMin, SumRates, TimeOutBrokeFixation,
+                         TimeOutEarlyWithdrawal, TimeOutIncorrectChoice,
+                         TimeOutSkippedFeedback, TrialSelection, VevaiometricMinWT,
+                         VevaiometricNBin, VevaiometricShowPoints, Wire1VideoTrigger
+
+Alternate SessionData structure
+    TrialTypes - 1,2,3,1,2,3
+    TrialTypeNames - Visibile,Visible,Fading
+    Info
+        StateMachineVersion
+        SessionDate
+        SessionStartTime_UTC
+        SessionStartTime_MATLAB
+    nTrials (# trials in session, here 54)
+    RawEvents (timestamps for each trial's state transitions/recorded events)
+        Trial{1,n}.States #Which of these are important?
+            WaitForPosTriggerSoftCode
+            CueDelay
+            WaitForResponse
+            Port2RewardDelay
+            Port2Reward
+            CloseValves
+            Drinking
+            Port1RewardDelay
+            Port3RewardDelay
+            Port4RewardDelay
+            Port5RewardDelay
+            Port6RewardDelay
+            Port7RewardDelay
+            Port8RewardDelay
+            Port1Reward
+            Port3Reward
+            Port4Reward
+            Port5Reward
+            Port6Reward
+            Port7Reward
+            Port8Reward
+            Punish
+            Punishexit
+            EarlyWithdrawal
+        Trial{1,n}.Events
+            Port4In
+            Port4Out
+            SoftCode10
+            Tup
+            Port2In
+            Port2Out
+    RawData (copy of raw data from state machine)
+    TrialStartTimestamp (time when trial started on Bpod's clock)
+        Note: Timestamps in RawEvents are relative to each trial's start
+    TrialEndTimestamp
+    SettingsFile (the settings file you selected in the launch manager)
+    Notes
+    MarkerCodes
+    CurrentSubjectName
+    TrialSettings
+        GUI
+        GUIMeta
+        GUIPanels
+        polling
+        debug
+        debugvis
+        Data
+        arm_number
+        arm_baited_orig
+        arm_baited
+        SF
+        rotation
+        position
+        StimAlpha
+    StimPos
+        TriggerLocPix
+        TriggerLocOptitrackHitbox
+        TriggerLocOptitrackCenter
+        TriggerLocOptitrackCircleHitRadius
+        tform
+```
+</details>
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,10 +2,10 @@
 
 Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
 
-## [0.1.0c0] - 2021-11-03
+## [0.1.0b0] - [unreleased]
 ### Added
-+ First draft begins
++ First beta release
 
-## [0.1.0b0] - 2021-00-00
+## [0.1.0a0] - 2021-11-15
 ### Added
-+ First beta release
++ First draft begins
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,3 @@
+# Contribution Guidelines
+
+This project follows the [DataJoint Contribution Guidelines](https://docs.datajoint.org/python/community/02-Contribute.html). Please reference the link for more full details.
diff --git a/README.md b/README.md
@@ -1,33 +1,62 @@
-# element-trial-behavior
-This repository is a work in progress. It serves as a draft of a DataJoints element for trial-based behavior for our U24 itiative.
-
-## Notes:
-I looked at the structure for `element-array-ephys` for general principle on how to call and load files. I mirrored the main DataJoint implementation as split from 'readers'. I incorporated feedback from project-specific `behavior.py` elsewhere in table development.
-
-## To do:
-- [ ] Support functions
-   - [ ] Other elements/workflows pull `find_full_path` and `find_root_directory` either from their own `__init__.py` files or from `element-data-loader.utils`. Which is best practice?
-   - [ ] `workflow-array-ephys` relies on the linking module for functions to get root and session directories, but the MAP project defines these internally.  Which is best practice?
-- [ ] Table definitions: Discuss table structure
-- [ ] Decide supported filetypes
-   -  [ ] BPOD
-   -  [ ] Kepec standard, TBD
-   -  [ ] Generalizable CSV with user-determined column name to DJ variable name correspondence?
-- [ ] Contact the [BPod team](https://github.com/sanworks/)
-   - [ ] Already an implementation of loading to Python?
-   - [ ] Create joint sustainability roadmap
-- [ ] Contact Kepec team - joint sustainability roadmap
-- [ ] Analysis package
-   - [ ] Load processed data to table structure
-   - [ ] Trigger analysis on raw data import
-- [ ] Quality control metrics
-- [ ] GitHub Actions for PyPI release
-- [ ] example workflow
-   - [ ] Integration tests with pytest
-   - [ ] Tutorials in text format (i.e. Jupyter notebook)
-   - [ ] Tutorial in video format
-   - [ ] Docker for tests
-   - [ ] Example dataset(s) for public release, in DJ Archive
-   - [ ] NWB export
-   - [ ] README
-- [ ] RRID
+# DataJoint Element - Experimental trials
+This repository is a work in progress not yet ready for public release.
+It serves as a draft of a DataJoint element for trialized experiments behavior
+for our U24 itiative.
+
+Work in progress.
+
+## Element architecture
+
+In both of the following diagrams, the trial element starts immediately downstream from ***Session***. In one case, Sessions are first segmented into trials, and then segmented into events. This might be appropriate, for example, in a paradigm with repeated conditions and response behaviors associated with different conditions. In the next, Sessions are directly upstream from Events. This might be appropropriate for a paradigm that recorded events within naturalistic free behavior.
+We  provide an [example workflow](https://github.com/datajoint/workflow-trial/) with a
+[pipeline script](https://github.com/datajoint/workflow-trial/blob/main/workflow_trial/pipeline.py) that models combining this Element with the corresponding [Element-Session](https://github.com/datajoint/element-session).
+
+
+<!---
+![element-trial diagram](images/attached_trial_element_trialized.svg)
+![element-trial diagram](images/attached_trial_element_events.svg)
+-->
+
+## Installation
+
++ Install `element-trial`
+    ```
+    pip install element-trial
+    ```
+
++ Upgrade `element-trial` previously installed with `pip`
+    ```
+    pip install --upgrade element-trial
+    ```
+
+<!---
++ Install `element-interface`
+
+    + `element-interface` is a dependency of `element-trial`, however it is not contained within `requirements.txt`.
+
+    ```
+    pip install "element-interface @ git+https://github.com/datajoint/element-interface"
+    ```
+-->
+
+## Usage
+
+### Element activation
+
+To activate the `element-trial`, one need to provide:
+
+1. Schema names for the event or trial module
+
+2. Upstream Session table: A set of keys identifying a recording session (see [Element-Session](https://github.com/datajoint/element-session)).
+3. Utility functions. See example definitions here](https://github.com/datajoint/workflow-trial/blob/main/workflow_trial/paths...
+
+For more detail, check the docstring of the `element-trial`:
+```python
+from element_trial import event, trial
+help(event.activate)
+help(trial.activate)
+```
+
+### Example usage
+
+See [this project](https://github.com/datajoint/workflow-trial) for an example usage of this Trial Element.
diff --git a/dj_local_conf_example.json b/dj_local_conf_example.json
diff --git a/element_trial/__init__.py b/element_trial/__init__.py
diff --git a/element_trial/event.py b/element_trial/event.py
diff --git a/element_trial/export/__init__.py b/element_trial/export/__init__.py
Original file line number	Diff line number	Diff line change
Expand Up		@@ -123,3 +123,4 @@ docker-compose.y*ml

		# notes
		temp*
		/temp
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		# Contribution Guidelines

		This project follows the [DataJoint Contribution Guidelines](https://docs.datajoint.org/python/community/02-Contribute.html). Please reference the link for more full details.