Skip to content

Commit

Permalink
Merge branch 'deepinterp' of github.com:alejoe91/spikeinterface into …
Browse files Browse the repository at this point in the history 
…deepinterp
  • Loading branch information
@alejoe91
alejoe91 committed Sep 5, 2023
2 parents aa30c6f + 6a36156 commit 9c312e9
Show file tree
Hide file tree
Showing 198 changed files with 6,007 additions and 6,802 deletions.
4 changes: 3 additions & 1 deletion .github/workflows/caches_cron_job.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
with:
path: ${{ github.workspace }}/test_env
key: ${{ runner.os }}-venv-${{ steps.dependencies.outputs.hash }}-${{ steps.date.outputs.date }}
lookup-only: 'true' # Avoids downloading the data, saving behavior is not affected.
- name: Cache found?
run: echo "Cache-hit == ${{steps.cache-venv.outputs.cache-hit == 'true'}}"
- name: Create the virtual environment to be cached
Expand Down Expand Up @@ -64,6 +65,7 @@ jobs:
with:
path: ~/spikeinterface_datasets
key: ${{ runner.os }}-datasets-${{ steps.repo_hash.outputs.dataset_hash }}
lookup-only: 'true' # Avoids downloading the data, saving behavior is not affected.
- name: Cache found?
run: echo "Cache-hit == ${{steps.cache-datasets.outputs.cache-hit == 'true'}}"
- name: Installing datalad and git-annex
Expand All @@ -88,7 +90,7 @@ jobs:
run: |
cd $HOME
pwd
du -hs spikeinterface_datasets
du -hs spikeinterface_datasets # Should show the size of ephy_testing_data
cd spikeinterface_datasets
pwd
ls -lh # Should show ephy_testing_data
Expand Down
23 changes: 20 additions & 3 deletions .github/workflows/streaming-extractor-test.yml
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
name: Test streaming extractors

on: workflow_dispatch
on:
pull_request:
types: [synchronize, opened, reopened]
branches:
- main

concurrency: # Cancel previous workflows on the same pull request
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down Expand Up @@ -28,14 +32,27 @@ jobs:
- run: git fetch --prune --unshallow --tags
- name: Install openblas
run: sudo apt install libopenblas-dev # Necessary for ROS3 support
- name: Install package and streaming extractor dependencies
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@v35
- name: Module changes
id: modules-changed
run: |
pip install -e .[test_core,streaming_extractors]
for file in ${{ steps.changed-files.outputs.all_changed_files }}; do
if [[ $file == *"/nwbextractors.py" || $file == *"/iblstreamingrecording.py"* ]]; then
echo "Streaming files changed changed"
echo "STREAMING_CHANGED=true" >> $GITHUB_OUTPUT
fi
done
- name: Install package and streaming extractor dependencies
if: ${{ steps.modules-changed.outputs.STREAMING_CHANGED == 'true' }}
run: pip install -e .[test_core,streaming_extractors]
# Temporary disabled because of complicated error with path
# - name: Install h5py with ROS3 support and test it works
# run: |
# pip uninstall -y h5py
# conda install -c conda-forge "h5py>=3.2"
# python -c "import h5py; assert 'ros3' in h5py.registered_drivers(), f'ros3 suppport not available, failed to install'"
- name: run tests
if: steps.modules-changed.outputs.STREAMING_CHANGED == 'true'
run: pytest -m "streaming_extractors and not ros3_test" -vv -ra
3 changes: 2 additions & 1 deletion doc/api.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -269,13 +269,14 @@ spikeinterface.widgets
.. autofunction:: plot_amplitudes
.. autofunction:: plot_autocorrelograms
.. autofunction:: plot_crosscorrelograms
.. autofunction:: plot_motion
.. autofunction:: plot_quality_metrics
.. autofunction:: plot_sorting_summary
.. autofunction:: plot_spike_locations
.. autofunction:: plot_spikes_on_traces
.. autofunction:: plot_template_metrics
.. autofunction:: plot_template_similarity
.. autofunction:: plot_timeseries
.. autofunction:: plot_traces
.. autofunction:: plot_unit_depths
.. autofunction:: plot_unit_locations
.. autofunction:: plot_unit_summary
Expand Down
14 changes: 7 additions & 7 deletions doc/how_to/analyse_neuropixels.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ the ipywydgets interactive ploter
.. code:: python
%matplotlib widget
si.plot_timeseries({'filter':rec1, 'cmr': rec4}, backend='ipywidgets')
si.plot_traces({'filter':rec1, 'cmr': rec4}, backend='ipywidgets')
Note that using this ipywydgets make possible to explore diffrents
preprocessing chain wihtout to save the entire file to disk. Everything
Expand All @@ -276,9 +276,9 @@ is lazy, so you can change the previsous cell (parameters, step order,
# here we use static plot using matplotlib backend
fig, axs = plt.subplots(ncols=3, figsize=(20, 10))
si.plot_timeseries(rec1, backend='matplotlib', clim=(-50, 50), ax=axs[0])
si.plot_timeseries(rec4, backend='matplotlib', clim=(-50, 50), ax=axs[1])
si.plot_timeseries(rec, backend='matplotlib', clim=(-50, 50), ax=axs[2])
si.plot_traces(rec1, backend='matplotlib', clim=(-50, 50), ax=axs[0])
si.plot_traces(rec4, backend='matplotlib', clim=(-50, 50), ax=axs[1])
si.plot_traces(rec, backend='matplotlib', clim=(-50, 50), ax=axs[2])
for i, label in enumerate(('filter', 'cmr', 'final')):
axs[i].set_title(label)
Expand All @@ -292,7 +292,7 @@ is lazy, so you can change the previsous cell (parameters, step order,
# plot some channels
fig, ax = plt.subplots(figsize=(20, 10))
some_chans = rec.channel_ids[[100, 150, 200, ]]
si.plot_timeseries({'filter':rec1, 'cmr': rec4}, backend='matplotlib', mode='line', ax=ax, channel_ids=some_chans)
si.plot_traces({'filter':rec1, 'cmr': rec4}, backend='matplotlib', mode='line', ax=ax, channel_ids=some_chans)
Expand Down Expand Up @@ -426,7 +426,7 @@ Let’s use here the ``locally_exclusive`` method for detection and the
job_kwargs = dict(n_jobs=40, chunk_duration='1s', progress_bar=True)
peaks = detect_peaks(rec, method='locally_exclusive', noise_levels=noise_levels_int16,
detect_threshold=5, local_radius_um=50., **job_kwargs)
detect_threshold=5, radius_um=50., **job_kwargs)
peaks
Expand All @@ -451,7 +451,7 @@ Let’s use here the ``locally_exclusive`` method for detection and the
from spikeinterface.sortingcomponents.peak_localization import localize_peaks
peak_locations = localize_peaks(rec, peaks, method='center_of_mass', local_radius_um=50., **job_kwargs)
peak_locations = localize_peaks(rec, peaks, method='center_of_mass', radius_um=50., **job_kwargs)
Expand Down
227 changes: 14 additions & 213 deletions doc/how_to/get_started.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ and the raster plots.

.. code:: ipython3
w_ts = sw.plot_timeseries(recording, time_range=(0, 5))
w_ts = sw.plot_traces(recording, time_range=(0, 5))
w_rs = sw.plot_rasters(sorting_true, time_range=(0, 5))
Expand Down Expand Up @@ -266,7 +266,7 @@ available parameters are dictionaries and can be accessed with:
'clustering': {},
'detection': {'detect_threshold': 5, 'peak_sign': 'neg'},
'filtering': {'dtype': 'float32'},
'general': {'local_radius_um': 100, 'ms_after': 2, 'ms_before': 2},
'general': {'radius_um': 100, 'ms_after': 2, 'ms_before': 2},
'job_kwargs': {},
'localization': {},
'matching': {},
Expand Down Expand Up @@ -497,218 +497,19 @@ accomodate the duration:
qm = sqm.compute_quality_metrics(we_TDC, qm_params=qm_params)
display(qm)
.. parsed-literal::
.. raw:: html

<div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>num_spikes</th>
<th>firing_rate</th>
<th>presence_ratio</th>
<th>snr</th>
<th>isi_violations_ratio</th>
<th>isi_violations_count</th>
<th>rp_contamination</th>
<th>rp_violations</th>
<th>sliding_rp_violation</th>
<th>amplitude_cutoff</th>
<th>amplitude_median</th>
<th>drift_ptp</th>
<th>drift_std</th>
<th>drift_mad</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>30</td>
<td>3.0</td>
<td>0.9</td>
<td>27.258799</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.200717</td>
<td>307.199036</td>
<td>1.313088</td>
<td>0.492143</td>
<td>0.476104</td>
</tr>
<tr>
<th>1</th>
<td>51</td>
<td>5.1</td>
<td>1.0</td>
<td>24.213808</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.500000</td>
<td>274.444977</td>
<td>0.934371</td>
<td>0.325045</td>
<td>0.216362</td>
</tr>
<tr>
<th>2</th>
<td>53</td>
<td>5.3</td>
<td>0.9</td>
<td>24.229277</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.500000</td>
<td>270.204590</td>
<td>0.901922</td>
<td>0.392344</td>
<td>0.372247</td>
</tr>
<tr>
<th>3</th>
<td>50</td>
<td>5.0</td>
<td>1.0</td>
<td>27.080778</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.500000</td>
<td>312.545715</td>
<td>0.598991</td>
<td>0.225554</td>
<td>0.185147</td>
</tr>
<tr>
<th>4</th>
<td>36</td>
<td>3.6</td>
<td>1.0</td>
<td>9.544292</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.207231</td>
<td>107.953278</td>
<td>1.913661</td>
<td>0.659317</td>
<td>0.507955</td>
</tr>
<tr>
<th>5</th>
<td>42</td>
<td>4.2</td>
<td>1.0</td>
<td>13.283191</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.204838</td>
<td>151.833191</td>
<td>0.671453</td>
<td>0.231825</td>
<td>0.156004</td>
</tr>
<tr>
<th>6</th>
<td>48</td>
<td>4.8</td>
<td>1.0</td>
<td>8.319447</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>NaN</td>
<td>0.500000</td>
<td>91.358444</td>
<td>2.391275</td>
<td>0.885580</td>
<td>0.772367</td>
</tr>
<tr>
<th>7</th>
<td>193</td>
<td>19.3</td>
<td>1.0</td>
<td>8.690839</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>0.155</td>
<td>0.500000</td>
<td>103.491577</td>
<td>0.710640</td>
<td>0.300565</td>
<td>0.316645</td>
</tr>
<tr>
<th>8</th>
<td>129</td>
<td>12.9</td>
<td>1.0</td>
<td>11.167040</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>0.310</td>
<td>0.500000</td>
<td>128.252319</td>
<td>0.985251</td>
<td>0.375529</td>
<td>0.301622</td>
</tr>
<tr>
<th>9</th>
<td>110</td>
<td>11.0</td>
<td>1.0</td>
<td>8.377251</td>
<td>0.0</td>
<td>0</td>
<td>0.0</td>
<td>0</td>
<td>0.270</td>
<td>0.203415</td>
<td>98.207291</td>
<td>1.386857</td>
<td>0.526532</td>
<td>0.410644</td>
</tr>
</tbody>
</table>
</div>
id num_spikes firing_rate presence_ratio snr isi_violations_ratio isi_violations_count rp_contamination rp_violations sliding_rp_violation amplitude_cutoff amplitude_median drift_ptp drift_std drift_mad
0 30 3.0 0.9 27.258799 0.0 0 0.0 0 NaN 0.200717 307.199036 1.313088 0.492143 0.476104
1 51 5.1 1.0 24.213808 0.0 0 0.0 0 NaN 0.500000 274.444977 0.934371 0.325045 0.216362
2 53 5.3 0.9 24.229277 0.0 0 0.0 0 NaN 0.500000 270.204590 0.901922 0.392344 0.372247
3 50 5.0 1.0 27.080778 0.0 0 0.0 0 NaN 0.500000 312.545715 0.598991 0.225554 0.185147
4 36 3.6 1.0 9.544292 0.0 0 0.0 0 NaN 0.207231 107.953278 1.913661 0.659317 0.507955
5 42 4.2 1.0 13.283191 0.0 0 0.0 0 NaN 0.204838 151.833191 0.671453 0.231825 0.156004
6 48 4.8 1.0 8.319447 0.0 0 0.0 0 NaN 0.500000 91.358444 2.391275 0.885580 0.772367
7 193 19.3 1.0 8.690839 0.0 0 0.0 0 0.155 0.500000 103.491577 0.710640 0.300565 0.316645
8 129 12.9 1.0 11.167040 0.0 0 0.0 0 0.310 0.500000 128.252319 0.985251 0.375529 0.301622
9 110 11.0 1.0 8.377251 0.0 0 0.0 0 0.270 0.203415 98.207291 1.386857 0.526532 0.410644
Quality metrics are also extensions (and become part of the waveform
Expand Down
4 changes: 2 additions & 2 deletions doc/how_to/handle_drift.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -118,10 +118,10 @@ to load them later.
'peak_sign': 'neg',
'detect_threshold': 8.0,
'exclude_sweep_ms': 0.1,
'local_radius_um': 50},
'radius_um': 50},
'select_kwargs': None,
'localize_peaks_kwargs': {'method': 'grid_convolution',
'local_radius_um': 30.0,
'radius_um': 30.0,
'upsampling_um': 3.0,
'sigma_um': array([ 5. , 12.5, 20. ]),
'sigma_ms': 0.25,
Expand Down
Loading

0 comments on commit 9c312e9

Please sign in to comment.