No curation ephys AND kilosort triggering#54

Closed

ttngu207 wants to merge 136 commits intodatajoint:mainfrom ttngu207:no-curation

+2,570-220

Commits on Aug 18, 2021

added NWB export
Thinh Nguyen
committed

Commits on Sep 19, 2021

copied ephys to ephys-no-curation, added recording_duration, make
Thinh Nguyen
committed
remove Curation
Thinh Nguyen
committed
make clustering_output_dir user-input optional, auto infer
Thinh Nguyen
committed
minor bug fix
Thinh Nguyen
committed

Commits on Sep 20, 2021

improve kilosort loading routine - add validate() method
Thinh Nguyen
committed
helper for ProbeInsertion - auto_generate_entries(session_key)
Thinh Nguyen
committed

Commits on Sep 21, 2021

no-curation, store processed data in user-specified processed_data_dir if provided
Thinh Nguyen
committed
using find_full_path for session dir as well - improve robustness
Thinh Nguyen
committed
Update ephys_no_curation.py
Thinh Nguyen
committed

Commits on Sep 22, 2021

bugfix
Thinh Nguyen
committed

Commits on Sep 23, 2021

automate ClusteringTask insertion
Thinh Nguyen
committed

Commits on Sep 30, 2021

triggering Kilosort with ecephys_spike_sorting package
Thinh Nguyen
committed
remove _required_packages_paths
Thinh Nguyen
committed
minor stylistic improvements
Thinh Nguyen
committed
Apply suggestions from code review

Thinh Nguyen
and
dimitri-yatsenko
authored

Commits on Oct 1, 2021

typo fix
Thinh Nguyen
committed
Merge branch 'no-curation' of https://github.com/ttngu207/element-array-ephys into no-curation
Thinh Nguyen
committed
handles a weird windows/unix path incompatibility (even with pathlib)
Thinh Nguyen
committed
fix missing clustering_method
Thinh Nguyen
committed
flag to create spike sorting output dir
Thinh Nguyen
committed
Update kilosort_trigger.py
Thinh Nguyen
committed
minor tweak/improvements in kilosort triggering
Thinh Nguyen
committed
bugfix in triggering ecephys_spike_sorting
Thinh Nguyen
committed
handle cases where fileTimeSecs is not available
Thinh Nguyen
committed

Commits on Oct 2, 2021

bugfix
Thinh Nguyen
committed
missing generate module json
Thinh Nguyen
committed
change default noise_template_use_rf to False
Thinh Nguyen
committed

Commits on Oct 4, 2021

minor bugfix
Thinh Nguyen
committed
add kilosort 2.5 as default content
Thinh Nguyen
committed
make variable naming consistent
Thinh Nguyen
committed
include clustering_method into the calculation of param_set_hash
Thinh Nguyen
committed
updating kilosort_repository depending on which KSVer to be used
Thinh Nguyen
committed
bugfix
Thinh Nguyen
committed
bugfix, convert path to string
Thinh Nguyen
committed

Commits on Oct 5, 2021

log the folder creation
Thinh Nguyen
committed
add debugging lines
Thinh Nguyen
committed
bugfix - no dir created
Thinh Nguyen
committed

Commits on Oct 6, 2021

Update kilosort.py
Thinh Nguyen
committed
new version 0.1.0b1
Thinh Nguyen
committed

Commits on Oct 7, 2021

improve clusteringtask and waveform ingestion routine
Thinh Nguyen
committed
minor cleanup
Thinh Nguyen
committed
stage tracking and resumable kilosort run
Thinh Nguyen
committed
minor bugfix
Thinh Nguyen
committed
bugfix logging for kilosort triggering
Thinh Nguyen
committed
duration for each module run
Thinh Nguyen
committed

Commits on Oct 8, 2021

check ap.bin file validity before triggering kilosort (based on filesize)
Thinh Nguyen
committed

Commits on Oct 11, 2021

Apply suggestions from code review

Thinh Nguyen
and
dimitri-yatsenko
authored
stylistic improvements, addressing code review comments
Thinh Nguyen
committed

Commits on Nov 2, 2021

handle NWB export with multiple curated clusterings from one session
Thinh Nguyen
committed
Update nwb.py
Thinh Nguyen
committed
minor bugfix
Thinh Nguyen
committed
Merge pull request #2 from ttngu207/nwb-export
Thinh Nguyen
authored
add nwb export to no-curation ephys
Thinh Nguyen
committed

Commits on Nov 9, 2021

first prototype for running the ecephys_pipeline with OpenEphys
Thinh Nguyen
committed
bugfix for running kilosort for Open Ephys data
Thinh Nguyen
committed

Commits on Nov 11, 2021

specify additional recording-info as part of the params
Thinh Nguyen
committed

Commits on Nov 18, 2021

add neuropixels UHD probe type
Thinh Nguyen
committed
triggering kilosort analysis for open-ephys
Thinh Nguyen
committed

Commits on Nov 19, 2021

first prototype for pykilosort
Thinh Nguyen
committed
use_C_waves=False for OpenEphys
Thinh Nguyen
committed

Commits on Nov 22, 2021

using fork of pyopenephys
Thinh Nguyen
committed
minor bugfix in running pykilosort
Thinh Nguyen
committed
update openephys loader
Thinh Nguyen
committed
fix package requirement formatting error
Thinh Nguyen
committed
handles format differences between npx1 vs 3A
Thinh Nguyen
committed

Commits on Nov 23, 2021

supporting neuropixels UHD in ephys.EphysRecording
Thinh Nguyen
committed
bugfix in electrode sites design for Neuropixels UHD probe
Thinh Nguyen
committed
search recording channels for Open Ephys based on channel names
Thinh Nguyen
committed
bugfix openephys loader
Thinh Nguyen
committed
bugfix - extract recording datetime (instead of using experiment datetime)
Thinh Nguyen
committed
bugfix, timedelta as seconds
Thinh Nguyen
committed
handles extraction of connected channels for NP_PROBE format in OpenEphys
Thinh Nguyen
committed

Commits on Nov 25, 2021

clusters extraction - check cluster_group.tsv and cluster_KSLabel.tsv
Thinh Nguyen
committed
bugfix in triggering pykilosort
Thinh Nguyen
committed
handle missing sample_rate from pykilosort params.py
Thinh Nguyen
committed
for pykilosort's probe, provide both Nchan and NchanTOT
Thinh Nguyen
committed

Commits on Nov 26, 2021

openephys loader - handles signalchain and processor as single element or list
Thinh Nguyen
committed
handles multi-probe for older OE version
Thinh Nguyen
committed

Commits on Nov 29, 2021

NWB export fix, specifying which ephys module
Thinh Nguyen
committed
version bump
Thinh Nguyen
committed
fix NWB export - null waveforms - back to version 0.1.0b1
Thinh Nguyen
committed
version bump
Thinh Nguyen
committed
include probe as part of the electrode_group name for uniqueness
Thinh Nguyen
committed

Commits on Dec 7, 2021

bugfix in assigning unit electrode indices
ttngu207
committed
Merge pull request #3 from ttngu207/nwb-export
Thinh Nguyen
authored

Commits on Jan 24, 2022

all three ephys files up to date
A-Baji
committed
applied requested changes
A-Baji
committed
Merge pull request #4 from A-Baji/no-curation
Thinh Nguyen
authored

Commits on Jan 26, 2022

DEPRECATING NWB EXPORT
Thinh Nguyen
committed
Merge branch 'main' of https://github.com/datajoint/element-array-ephys into no-curation
Thinh Nguyen
committed

Commits on Jan 27, 2022

minor bugfix
Thinh Nguyen
committed
name tweak
Thinh Nguyen
committed
Apply suggestions from code review

Thinh Nguyen
and
dimitri-yatsenko
authored

Commits on Jan 31, 2022

rename sess_dir -> session_dir
Thinh Nguyen
committed
Update requirements.txt
Thinh Nguyen
authored
Update requirements.txt
Thinh Nguyen
authored
Update requirements.txt
Thinh Nguyen
authored

Commits on Feb 18, 2022

handle older open ephys format for single probe
Thinh Nguyen
authored
remove unintended prototyping work
Thinh Nguyen
authored

Commits on Feb 25, 2022

delete nwb export - rename ephys module -> ephys_acute
Thinh Nguyen
committed

Commits on Mar 14, 2022

Update __init__.py
Thinh Nguyen
authored

Commits on Mar 16, 2022

bugfix
Thinh Nguyen
authored

Commits on Mar 25, 2022

configurable paramset_idx for auto ClusteringTask generation
Thinh Nguyen
authored

Commits on May 12, 2022

Update openephys.py
Thinh Nguyen
committed

Commits on May 20, 2022

update openephys loader - handle new open ephys format
Thinh Nguyen
committed
handles new probe naming in latest Open Ephys format
Thinh Nguyen
committed

Commits on May 23, 2022

Apply suggestions from code review

Thinh Nguyen
and
dimitri-yatsenko
authored
added assertion - safeguard against failed loading of continuous.dat
Thinh Nguyen
committed
Merge branch 'no-curation' of https://github.com/ttngu207/element-array-ephys into no-curation
Thinh Nguyen
committed
Merge remote-tracking branch 'upstream/run_kilosort' into no-curation
Thinh Nguyen
committed
Update requirements.txt
Thinh Nguyen
committed
kilosort2 also as part of the contents for ClusteringMethod
Thinh Nguyen
committed

Commits on May 24, 2022

bugfix for LFP electrode mapping
Thinh Nguyen
committed

Commits on May 26, 2022

update open ephys loader to handle "STREAM" in latest format
Thinh Nguyen
committed

Commits on Jun 1, 2022

added loading of electrode location for new openephys format
Thinh Nguyen
committed

Commits on Jun 3, 2022

more robust loading of openephys format
Thinh Nguyen
committed

Commits on Jun 7, 2022

improve logic for running kilosort modules in a resumable fashion
Thinh Nguyen
committed
code cleanup, minor bugfix
Thinh Nguyen
committed

Commits on Jun 9, 2022

bugfix, match new implementation for openephys
Thinh Nguyen
committed

Commits on Jun 30, 2022

improve kilosort calls, handle spaces in paths
Thinh Nguyen
authored
remove space escaping character
Thinh Nguyen
authored

Commits on Jul 19, 2022

improve error message
Thinh Nguyen
committed
Merge branch 'no-curation' of https://github.com/ttngu207/element-array-ephys into no-curation
Thinh Nguyen
committed

Commits on Jul 25, 2022

implement data compression using mtscomp for openephys and spikeglx neuropixels data
Thinh Nguyen
committed

Commits on Aug 12, 2022

garbage collect openephys objects
Thinh Nguyen
committed

Commits on Aug 15, 2022

garbage collect openephys objects
Thinh Nguyen
committed

Commits on Aug 30, 2022

Update kilosort_triggering.py
Thinh Nguyen
committed

Commits on Sep 2, 2022

bugfix in running catgt
Thinh Nguyen
committed
enable catgt run
Thinh Nguyen
committed
bugfix
Thinh Nguyen
committed
bugfix
Thinh Nguyen
committed
missing_ok arg only available in python 3.8+
Thinh Nguyen
committed
bugfix - path search for catgt output
Thinh Nguyen
committed
improve error handling
Thinh Nguyen
committed

Commits on Sep 14, 2022

Update kilosort_triggering.py
Thinh Nguyen
committed