Merge pull request #2 from CBroz1/master

Add table definitions

.gitignore

@@ -123,3 +123,4 @@ docker-compose.y*ml
 
 # notes
 temp*
+*/temp*

CHANGELOG.md

@@ -2,10 +2,8 @@
 
 Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
 
-## [0.1.0c0] - 2021-11-03
-### Added
-+ First draft begins
-
-## [0.1.0b0] - 2021-00-00
+## 0.1.0b0 - Unreleased
 ### Added
 + First beta release
+
++ First draft based on [Cajal](https://github.com/cajal/pipeline) and [Kavli Institute](https://github.com/kavli-ntnu/dj-docs) precursor projects

CONTRIBUTING.md

@@ -0,0 +1,3 @@
+# Contribution Guidelines
+
+This project follows the [DataJoint Contribution Guidelines](https://docs.datajoint.org/python/community/02-Contribute.html). Please reference the link for more full details.

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2021 DataJoint
+Copyright (c) 2022 DataJoint
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,33 +1,65 @@
-# element-trial-behavior
-This repository is a work in progress. It serves as a draft of a DataJoints element for trial-based behavior for our U24 itiative.
-
-## Notes:
-I looked at the structure for `element-array-ephys` for general principle on how to call and load files. I mirrored the main DataJoint implementation as split from 'readers'. I incorporated feedback from project-specific `behavior.py` elsewhere in table development.
-
-## To do:
-- [ ] Support functions
-   - [ ] Other elements/workflows pull `find_full_path` and `find_root_directory` either from their own `__init__.py` files or from `element-data-loader.utils`. Which is best practice?
-   - [ ] `workflow-array-ephys` relies on the linking module for functions to get root and session directories, but the MAP project defines these internally.  Which is best practice?
-- [ ] Table definitions: Discuss table structure
-- [ ] Decide supported filetypes
-   -  [ ] BPOD
-   -  [ ] Kepec standard, TBD
-   -  [ ] Generalizable CSV with user-determined column name to DJ variable name correspondence?
-- [ ] Contact the [BPod team](https://github.com/sanworks/)
-   - [ ] Already an implementation of loading to Python?
-   - [ ] Create joint sustainability roadmap
-- [ ] Contact Kepec team - joint sustainability roadmap
-- [ ] Analysis package
-   - [ ] Load processed data to table structure
-   - [ ] Trigger analysis on raw data import
-- [ ] Quality control metrics
-- [ ] GitHub Actions for PyPI release
-- [ ] example workflow
-   - [ ] Integration tests with pytest
-   - [ ] Tutorials in text format (i.e. Jupyter notebook)
-   - [ ] Tutorial in video format
-   - [ ] Docker for tests
-   - [ ] Example dataset(s) for public release, in DJ Archive
-   - [ ] NWB export
-   - [ ] README
-- [ ] RRID
+# DataJoint Element - Experimental trials
+This repository is a work in progress not yet ready for public release.
+It serves as a draft of a DataJoint element for trialized experiments behavior
+for our U24 itiative.
+
+## Element architecture
+
+In both of the following diagrams, the trial table starts immediately downstream from
+***Session***. In one case, Sessions are first segmented into trials, and then 
+segmented into events. This might be appropriate, for example, in a paradigm with 
+repeated conditions and response behaviors associated with different conditions. In the 
+next, Sessions are directly upstream from both Trials and Events. This might be appropropriate for a paradigm that recorded events within naturalistic free behavior. We  provide an
+[example workflow](https://github.com/datajoint/workflow-trial/) with a
+[pipeline script](https://github.com/datajoint/workflow-trial/blob/main/workflow_trial/pipeline.py)
+that models combining this Element with the corresponding 
+[Element-Session](https://github.com/datajoint/element-session).
+
+### Trial Schema
+
+![trial schema](./images/diagram_trial.svg)
+
+### Event Schema
+![event schema](./images/diagram_event.svg)
+
+## Installation
+
++ Install `element-trial`
+    ```
+    pip install element-trial
+    ```
+
++ Upgrade `element-trial` previously installed with `pip`
+    ```
+    pip install --upgrade element-trial
+    ```
+
+<!---
++ Install `element-interface`
+
+    + `element-interface` is a dependency of `element-trial`, however it is not 
+      contained within `requirements.txt`.
+
+    ```
+    pip install "element-interface @ git+https://github.com/datajoint/element-interface"
+    ```
+-->
+
+## Usage
+
+### Element activation
+
+To activate the `element-trial`, one need to provide:
+
+1. Schema names for the event or trial module
+2. Upstream Session table: A set of keys identifying a recording session (see [
+Element-Session](https://github.com/datajoint/element-session)).
+3. Utility functions. See 
+[example definitions here](https://github.com/datajoint/workflow-trial/blob/main/workflow_trial/paths.py)
+
+For more detail, check the docstring of the `element-trial`:
+```python
+from element_trial import event, trial
+help(event.activate)
+help(trial.activate)
+```

element_trial/event.py

@@ -0,0 +1,156 @@
+"""Events are linked directly to session"""
+
+import datajoint as dj
+import inspect
+import importlib
+
+schema = dj.schema()
+
+_linking_module = None
+
+
+def activate(schema_name, *, create_schema=True, create_tables=True,
+             linking_module=None):
+    """
+    activate(schema_name, *, create_schema=True, create_tables=True,
+             linking_module=None)
+        :param schema_name: schema name on the database server to activate
+                            the `behavior` element
+        :param create_schema: when True (default), create schema in the
+                              database if it does not yet exist.
+        :param create_tables: when True (default), create tables in the
+                              database if they do not yet exist.
+        :param linking_module: a module (or name) containing the required
+                               dependencies to activate the `session` element:
+            Upstream tables:
+                + Session: parent table to ProbeInsertion, typically
+                           identifying a recording session.
+            Functions:
+                + get_trial_root_data_dir() -> list
+                    Retrieve the root data director(y/ies) with behavioral
+                    recordings (e.g., bpod files) for all subject/sessions.
+                    :return: a string for full path to the root data directory
+                + get_trial_sess_dir(session_key: dict) -> str
+                    Retrieve the session directory containing the recording(s)
+                    for a given Session
+                    :param session_key: a dictionary of one Session `key`
+                    :return: a string for full path to the session directory
+    """
+    if isinstance(linking_module, str):
+        linking_module = importlib.import_module(linking_module)
+    assert inspect.ismodule(linking_module), "The argument 'dependency' must"\
+                                             + " be a module or module name"
+
+    schema.activate(schema_name, create_schema=create_schema,
+                    create_tables=create_tables,
+                    add_objects=linking_module.__dict__)
+
+# -------------- Functions required by the element-trial   ---------------
+
+
+def get_trial_root_data_dir() -> list:
+    """
+    All data paths, directories in DataJoint Elements are recommended to be
+    stored as relative paths, with respect to some user-configured "root"
+    directory, which varies from machine to machine
+
+    get_trial_root_data_dir() -> list
+        This user-provided function retrieves the list of possible root data
+        directories containing the behavioral data for all subjects/sessions
+        :return: a string for full path to the behavioral root data directory,
+         or list of strings for possible root data directories
+    """
+    return _linking_module.get_trial_root_data_dir()
+
+
+def get_trial_sess_dir(session_key: dict) -> str:
+    """
+    get_trial_sess_dir(session_key: dict) -> str
+        Retrieve the session directory, with all recordings for a given Session
+        :param session_key: a dictionary of one Session `key`
+        :return: a string for full path to the session directory
+    """
+    return _linking_module.get_trial_sess_dir(session_key)
+
+# ----------------------------- Table declarations ----------------------
+
+
+@schema
+class BehaviorRecording(dj.Manual):
+    definition = """
+    -> Session
+    recording_id    : varchar(16)
+    ---
+    recording_notes : varchar(256)
+    """
+
+    class BehaviorFile(dj.Part):
+        definition = """
+        -> master
+        filepath    : varchar(16)
+        """
+
+
+@schema
+class TrialType(dj.Lookup):
+    definition = """
+    trial_type              : varchar(16)
+    ---
+    trial_type_description  : varchar(256)
+    """
+
+
+@schema
+class Trial(dj.Imported):
+    definition = """
+    -> Session
+    trial      : smallint # trial number (1-based indexing)
+    ---
+    -> TrialType
+    start_time : float  # (second) relative to recording start
+    stop_time  : float  # (second) relative to recording start
+    """
+
+    class TrialVariable(dj.Part):
+        definition = """
+        -> master
+        variable_name: varchar(16)
+        ---
+        variable_value: varchar(2000)
+        """
+
+
+@schema
+class EventType(dj.Lookup):
+    definition = """
+    event_type               : varchar(16)
+    ---
+    event_type_description='': varchar(256)
+    """
+
+
+@schema
+class Event(dj.Imported):
+    definition = """ #
+    -> Session
+    event_start_time   : float  # (second) relative to recording start
+    ---
+    -> EventType
+    event_end_time=null: float  # (second) relative to recording start
+    """
+
+    class EventVariable(dj.Part):
+        definition = """
+        -> master
+        variable_name: varchar(16)
+        ---
+        variable_value: varchar(2000)
+        """
+
+
+@schema
+class TrialEvent(dj.Imported):
+    definition = """
+    -> Trial
+    -> Event
+    """