From 4e8ba0533ad29bd8ac4b7fb4c37de181fefe783b Mon Sep 17 00:00:00 2001 From: chrishalcrow <57948917+chrishalcrow@users.noreply.github.com> Date: Thu, 11 Jul 2024 09:43:22 +0100 Subject: [PATCH 1/8] Everything but repeated titles --- doc/how_to/load_your_data_into_sorting.rst | 6 ++-- doc/how_to/viewers.rst | 2 +- doc/index.rst | 7 ++--- doc/modules/core.rst | 8 ++++-- doc/modules/curation.rst | 10 +++---- doc/modules/exporters.rst | 2 +- doc/modules/postprocessing.rst | 2 -- doc/modules/preprocessing.rst | 2 +- doc/modules/qualitymetrics/references.rst | 28 ------------------- doc/modules/qualitymetrics/synchrony.rst | 2 +- doc/modules/sorters.rst | 2 ++ doc/references.rst | 6 ++-- doc/releases/0.100.0.rst | 3 +- src/spikeinterface/core/sortinganalyzer.py | 14 +++++----- .../qualitymetrics/misc_metrics.py | 4 +-- .../quality_metric_calculator.py | 2 +- 16 files changed, 36 insertions(+), 64 deletions(-) diff --git a/doc/how_to/load_your_data_into_sorting.rst b/doc/how_to/load_your_data_into_sorting.rst index 21c4460c5a..4e434ecb7a 100644 --- a/doc/how_to/load_your_data_into_sorting.rst +++ b/doc/how_to/load_your_data_into_sorting.rst @@ -18,7 +18,7 @@ to take your existing data and load it as a SpikeInterface :code:`Sorting` objec Reading a standard spike sorting format into a :code:`Sorting` -------------------------------------------------------------- +-------------------------------------------------------------- For most spike sorting output formats the :code:`Sorting` is automatically generated. For example one could do @@ -36,7 +36,7 @@ current formats see :ref:`compatible_formats`. Loading your own data into a :code:`Sorting` -------------------------------------------- +-------------------------------------------- This :code:`Sorting` contains important information about your spike trains including: @@ -115,7 +115,7 @@ using :code:`Neo.SpikeTrain`'s. Loading multisegment data into a :code:`Sorting` ------------------------------------------------ +------------------------------------------------ One of the great advantages of SpikeInterface :code:`Sorting` objects is that they can also handle multisegment recordings and sortings (e.g. you have a baseline, stimulus, post-stimulus). The diff --git a/doc/how_to/viewers.rst b/doc/how_to/viewers.rst index c7574961bd..9486755c85 100644 --- a/doc/how_to/viewers.rst +++ b/doc/how_to/viewers.rst @@ -11,7 +11,7 @@ There are several ways to plot signals (raw, preprocessed) and spikes. 4. You can use the :code:`phy` software -spikeinterface.widgets +spikeinterface-widgets ---------------------- The easiest way to visualize :code:`spikeinterface` objects is to use the :code:`widgets` module for plotting. diff --git a/doc/index.rst b/doc/index.rst index 96dea055cb..3fd779d89f 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -15,12 +15,11 @@ state-of-the-art spike sorters, post-process and curate the output, compute qual .. warning:: - **New features under construction!** 🚧🚧🚧: after the 0.100.0 release (and related bug fixes), the next release will contain a major API improvement: the :code:`SortingAnalyzer`. To read more about this, checkout the `enhancement proposal `_. - Please refer to the stable documentation `here `_. - Learn how to update your code `here `_ and read more about the - :code:`SortingAnalyzer` `here `_. + Please refer to the `stable documentation `_. + Learn how to :ref:`update your code here ` and read more about the + :ref:`SortingAnalyzer here `. diff --git a/doc/modules/core.rst b/doc/modules/core.rst index f8f410018b..66c023b5be 100644 --- a/doc/modules/core.rst +++ b/doc/modules/core.rst @@ -22,6 +22,7 @@ All classes support: * multiple segments, where each segment is a contiguous piece of data (recording, sorting, events). .. _core-recording: + Recording --------- @@ -163,6 +164,7 @@ Internally, any sorting object can construct 2 internal caches: time, like for extracting amplitudes from a recording. .. _core-sorting-analyzer: + SortingAnalyzer --------------- @@ -207,7 +209,7 @@ Now we will create a :code:`SortingAnalyzer` called :code:`sorting_analyzer`. The :py:class:`~spikeinterface.core.SortingAnalyzer` by default is defined *in memory*, but it can be saved at any time (or upon instantiation) to one of the following backends: -* | :code:`zarr`: the sorting analyzer is saved to a `Zarr `_ folder, and each extension is a Zarr group. This is the recommended backend, since Zarr files can be written to/read from the cloud and compression is applied. +* | :code:`zarr`: the sorting analyzer is saved to a `Zarr `__ folder, and each extension is a Zarr group. This is the recommended backend, since Zarr files can be written to/read from the cloud and compression is applied. * | :code:`binary_folder`: the sorting analyzer is saved to a folder, and each extension creates a sub-folder. The extension data are saved to either :code:`npy` (for arrays), :code:`csv` (for dataframes), or :code:`pickle` (for everything else). If the sorting analyzer is in memory, the :code:`SortingAnalyzer.save_as` function can be used to save it @@ -555,7 +557,7 @@ re-instantiate the object from scratch (this is true for all objects except in-m The :code:`save()` function allows to easily store SI objects to a folder on disk. :py:class:`~spikeinterface.core.BaseRecording` objects are stored in binary (.raw) or -`Zarr `_ (.zarr) format and +`Zarr `__ (.zarr) format and :py:class:`~spikeinterface.core.BaseSorting` and :py:class:`~spikeinterface.core.BaseSnippets` object in numpy (.npz) format. With the actual data, the :code:`save()` function also stores the provenance dictionary and all the properties and annotations associated to the object. @@ -909,7 +911,7 @@ The :py:class:`~spikeinterface.core.WaveformExtractor` allows us to: * extract waveforms * sub-sample spikes for waveform extraction * compute templates (i.e. average extracellular waveforms) with different modes -* save waveforms in a folder (in numpy / `Zarr `_) for easy retrieval +* save waveforms in a folder (in numpy / `Zarr `__) for easy retrieval * save sparse waveforms or *sparsify* dense waveforms * select units and associated waveforms diff --git a/doc/modules/curation.rst b/doc/modules/curation.rst index 46fdcc6d65..45e6fb9ae8 100644 --- a/doc/modules/curation.rst +++ b/doc/modules/curation.rst @@ -65,12 +65,12 @@ This format has two part: * "merged_unit_groups" * "removed_units" -Here is the description of the format with a simple example: +Here is the description of the format with a simple example (the first part of the +format is the definition; the second part of the format is manual action): .. code-block:: json { - # the first part of the format is the definitation "format_version": "1", "unit_ids": [ "u1", @@ -91,7 +91,7 @@ Here is the description of the format with a simple example: "MUA", "artifact" ], - "exclusive": true + "exclusive": "true" }, "putative_type": { "label_options": [ @@ -100,10 +100,10 @@ Here is the description of the format with a simple example: "pyramidal", "mitral" ], - "exclusive": false + "exclusive": "false" } }, - # the second part of the format is manual action + "manual_labels": [ { "unit_id": "u1", diff --git a/doc/modules/exporters.rst b/doc/modules/exporters.rst index 6ccee2246c..580a21f74d 100644 --- a/doc/modules/exporters.rst +++ b/doc/modules/exporters.rst @@ -15,7 +15,7 @@ results. **Note** : :py:func:`~spikeinterface.exporters.export_to_phy` speed and the size of the folder will highly depend on the sparsity of the :code:`SortingAnalyzer` itself or the external specified sparsity. The Phy viewer enables one to explore PCA projections, spike amplitudes, waveforms and quality of spike sorting results. -So if these pieces of information have already been computed as extensions (see :ref:`analyzer_extensions`), +So if these pieces of information have already been computed as extensions (see :ref:`Extensions as AnalyzerExtensions`), then exporting to Phy should be fast (and the user has better control of the parameters for the extensions). If not pre-computed, then the required extensions (e.g., :code:`spike_amplitudes`, :code:`principal_components`) can be computed directly at export time. diff --git a/doc/modules/postprocessing.rst b/doc/modules/postprocessing.rst index 9aad8568f4..dfa4c8b517 100644 --- a/doc/modules/postprocessing.rst +++ b/doc/modules/postprocessing.rst @@ -68,11 +68,9 @@ both `random_spikes` and `waveforms`. We say that `principal_components` is a ch two. Other extensions, like `isi_histograms`, don't depend on anything. It has no children and no parents. The parent/child relationships of all the extensions currently defined in spikeinterface can be found in this diagram: -| .. figure:: ../images/parent_child.svg :alt: Parent child relationships for the extensions in spikeinterface :align: center -| If you try to calculate a child before calculating a parent, an error will be thrown. Further, when a parent is recalculated we delete its children. Why? Consider calculating :code:`principal_components`. This depends on random selection of spikes chosen diff --git a/doc/modules/preprocessing.rst b/doc/modules/preprocessing.rst index 38a7da995b..0b71b49d3f 100644 --- a/doc/modules/preprocessing.rst +++ b/doc/modules/preprocessing.rst @@ -32,7 +32,7 @@ These two preprocessors will not compute anything at instantiation, but the comp traces = recording_cmr.get_traces(start_frame=100_000, end_frame=200_000) -Some internal sorters (see :ref:`si_based`) can work directly on these preprocessed objects so there is no need to +Some internal sorters (see :ref:`Intertnal Sorters`) can work directly on these preprocessed objects so there is no need to save the object: .. code-block:: python diff --git a/doc/modules/qualitymetrics/references.rst b/doc/modules/qualitymetrics/references.rst index f5236cff66..72829b345a 100644 --- a/doc/modules/qualitymetrics/references.rst +++ b/doc/modules/qualitymetrics/references.rst @@ -1,30 +1,2 @@ References ---------- - -.. [Buzsáki] Buzsáki, György, and Kenji Mizuseki. “The Log-Dynamic Brain: How Skewed Distributions Affect Network Operations.” Nature reviews. Neuroscience 15.4 (2014): 264–278. Web. - -.. [Chung] Chung, Jason E et al. “A Fully Automated Approach to Spike Sorting.” Neuron (Cambridge, Mass.) 95.6 (2017): 1381–1394.e6. Web. - -.. [Harris] Kenneth D Harris, Hajime Hirase, Xavier Leinekugel, Darrell A Henze, and Gy ̈orgy Buzs ́aki. Temporal interaction between single spikes and complex spike bursts in hippocampal pyramidal cells. Neuron (Cambridge, Mass.), 32(1):141–149, 2001. - -.. [Hill] Hill, Daniel N., Samar B. Mehta, and David Kleinfeld. “Quality Metrics to Accompany Spike Sorting of Extracellular Signals.” The Journal of neuroscience 31.24 (2011): 8699–8705. Web. - -.. [Hruschka] Hruschka, E.R., de Castro, L.N., Campello R.J.G.B. "Evolutionary algorithms for clustering gene-expression data." Fourth IEEE International Conference on Data Mining (ICDM'04) 2004, pp 403-406. - -.. [Gruen] Sonja Grün, Moshe Abeles, and Markus Diesmann. Impact of higher-order correlations on coincidence distributions of massively parallel data. In International School on Neural Networks, Initiated by IIASS and EMFCSC, volume 5286, 96–114. Springer, 2007. - -.. [IBL] International Brain Laboratory. “Spike sorting pipeline for the International Brain Laboratory”. 4 May 2022. - -.. [Jackson] Jadin Jackson, Neil Schmitzer-Torbert, K.D. Harris, and A.D. Redish. Quantitative assessment of extracellular multichannel recording quality using measures of cluster separation. Soc Neurosci Abstr, 518, 01 2005. - -.. [Lemon] R. Lemon. Methods for neuronal recording in conscious animals. IBRO Handbook Series, 4:56–60, 1984. - -.. [Llobet] Llobet Victor, Wyngaard Aurélien and Barbour Boris. “Automatic post-processing and merging of multiple spike-sorting analyses with Lussac“. BioRxiv (2022). - -.. [Pouzat] Pouzat Christophe, Mazor Ofer and Laurent Gilles. “Using noise signature to optimize spike-sorting and to assess neuronal classification quality“. Journal of Neuroscience Methods (2002). - -.. [Rousseeuw] Peter J Rousseeuw. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20(C):53–65, 1987. - -.. [Schmitzer-Torbert] Schmitzer-Torbert, Neil, and A. David Redish. “Neuronal Activity in the Rodent Dorsal Striatum in Sequential Navigation: Separation of Spatial and Reward Responses on the Multiple T Task.” Journal of neurophysiology 91.5 (2004): 2259–2272. Web. - -.. [Siegle] Siegle, Joshua H. et al. “Survey of Spiking in the Mouse Visual System Reveals Functional Hierarchy.” Nature (London) 592.7852 (2021): 86–. Web. diff --git a/doc/modules/qualitymetrics/synchrony.rst b/doc/modules/qualitymetrics/synchrony.rst index 41c92dd99e..d244fd0c0f 100644 --- a/doc/modules/qualitymetrics/synchrony.rst +++ b/doc/modules/qualitymetrics/synchrony.rst @@ -48,4 +48,4 @@ References Literature ---------- -Based on concepts described in [Gruen]_ +Based on concepts described in [Grün]_ diff --git a/doc/modules/sorters.rst b/doc/modules/sorters.rst index d8a4708236..2a440f9c0a 100644 --- a/doc/modules/sorters.rst +++ b/doc/modules/sorters.rst @@ -473,6 +473,8 @@ Here is the list of external sorters accessible using the run_sorter wrapper: * **HDSort** :code:`run_sorter(sorter_name='hdsort')` * **YASS** :code:`run_sorter(sorter_name='yass')` +Intertnal Sorters +----------------- Here a list of internal sorter based on `spikeinterface.sortingcomponents`; they are totally experimental for now: diff --git a/doc/references.rst b/doc/references.rst index 5fbcbecb63..ce4672a9ca 100644 --- a/doc/references.rst +++ b/doc/references.rst @@ -58,7 +58,7 @@ important for your research: - :code:`rp_violation` [Llobet]_ - :code:`sd_ratio` [Pouzat]_ - :code:`snr` [Lemon]_ [Jackson]_ -- :code:`synchrony` [Grun]_ +- :code:`synchrony` [Grün]_ If you use the :code:`qualitymetrics.pca_metrics` module, i.e. you use the :code:`compute_pc_metrics()` method, please include the citations for the :code:`metric_names` that were particularly @@ -78,7 +78,7 @@ References .. [Buccino] `SpikeInterface, a unified framework for spike sorting. 2020. `_ -.. [Buzsaki] `The Log-Dynamic Brain: How Skewed Distributions Affect Network Operations. 2014. `_ +.. [Buzsáki] `The Log-Dynamic Brain: How Skewed Distributions Affect Network Operations. 2014. `_ .. [Chaure] `A novel and fully automatic spike-sorting implementation with variable number of features. 2018. `_ @@ -88,7 +88,7 @@ References .. [Garcia] `A Modular Implementation to Handle and Benchmark Drift Correction for High-Density Extracellular Recordings. 2024. `_ -.. [Grun] `Impact of higher-order correlations on coincidence distributions of massively parallel data. 2007. `_ +.. [Grün] `Impact of higher-order correlations on coincidence distributions of massively parallel data. 2007. `_ .. [Harris] `Temporal interaction between single spikes and complex spike bursts in hippocampal pyramidal cells. 2001. `_ diff --git a/doc/releases/0.100.0.rst b/doc/releases/0.100.0.rst index d39b5569da..6bd8a33989 100644 --- a/doc/releases/0.100.0.rst +++ b/doc/releases/0.100.0.rst @@ -24,8 +24,7 @@ Main changes: core: * Add `Templates` class (#1982) -* Use python methods instead of parsing and eleminate try-except in to_dict -(#2157) +* Use python methods instead of parsing and eleminate try-except in to_dict (#2157) * `WaveformExtractor.is_extension` --> `has_extension` (#2158) * Speed improvement to `get_empty_units()` (#2173) * Allow precomputing spike trains (#2175) diff --git a/src/spikeinterface/core/sortinganalyzer.py b/src/spikeinterface/core/sortinganalyzer.py index fc20029ce6..af3b502f0f 100644 --- a/src/spikeinterface/core/sortinganalyzer.py +++ b/src/spikeinterface/core/sortinganalyzer.py @@ -718,7 +718,7 @@ def save_as(self, format="memory", folder=None) -> "SortingAnalyzer": def select_units(self, unit_ids, format="memory", folder=None) -> "SortingAnalyzer": """ - This method is equivalent to `save_as()`but with a subset of units. + This method is equivalent to `save_as()` but with a subset of units. Filters units by creating a new sorting analyzer object in a new folder. Extensions are also updated to filter the selected unit ids. @@ -873,11 +873,10 @@ def compute(self, input, save=True, extension_params=None, verbose=False, **kwar ---------- input : str or dict or list The extensions to compute, which can be passed as: - * a string: compute one extension. Additional parameters can be passed as key word arguments. * a dict: compute several extensions. The keys are the extension names and the values are dictiopnaries with the extension parameters. * a list: compute several extensions. The list contains the extension names. Additional parameters can be passed with the extension_params - argument. + argument. save : bool, default: True If True the extension is saved to disk (only if sorting analyzer format is not "memory") extension_params : dict or None, default: None @@ -906,10 +905,11 @@ def compute(self, input, save=True, extension_params=None, verbose=False, **kwar Compute two extensions with an input list specifying custom parameters for one (the other will use default parameters): - >>> analyzer.compute( - ["random_spikes", "waveforms"], - extension_params={"waveforms":{"ms_before":1.5, "ms_after", "2.5"}} - ) + >>> analyzer.compute(\ +["random_spikes", "waveforms"],\ +extension_params={"waveforms":{"ms_before":1.5, "ms_after": "2.5"}}\ +) + """ if isinstance(input, str): return self.compute_one_extension(extension_name=input, save=save, verbose=verbose, **kwargs) diff --git a/src/spikeinterface/qualitymetrics/misc_metrics.py b/src/spikeinterface/qualitymetrics/misc_metrics.py index 433c04d248..24165da5b3 100644 --- a/src/spikeinterface/qualitymetrics/misc_metrics.py +++ b/src/spikeinterface/qualitymetrics/misc_metrics.py @@ -534,7 +534,7 @@ def get_synchrony_counts(spikes, synchrony_sizes, all_unit_ids): References ---------- - Based on concepts described in [Gruen]_ + Based on concepts described in [Grün]_ This code was adapted from `Elephant - Electrophysiology Analysis Toolkit `_ """ @@ -581,7 +581,7 @@ def compute_synchrony_metrics(sorting_analyzer, synchrony_sizes=(2, 4, 8), unit_ References ---------- - Based on concepts described in [Gruen]_ + Based on concepts described in [Grün]_ This code was adapted from `Elephant - Electrophysiology Analysis Toolkit `_ """ assert min(synchrony_sizes) > 1, "Synchrony sizes must be greater than 1" diff --git a/src/spikeinterface/qualitymetrics/quality_metric_calculator.py b/src/spikeinterface/qualitymetrics/quality_metric_calculator.py index 5a9191a256..f3eecb20bf 100644 --- a/src/spikeinterface/qualitymetrics/quality_metric_calculator.py +++ b/src/spikeinterface/qualitymetrics/quality_metric_calculator.py @@ -19,7 +19,7 @@ class ComputeQualityMetrics(AnalyzerExtension): """ - Compute quality metrics on sorting_. + Compute quality metrics on a `sorting_analyzer`. Parameters ---------- From cb8032aff756e29d9d7e3696d92940411cdd9c62 Mon Sep 17 00:00:00 2001 From: chrishalcrow <57948917+chrishalcrow@users.noreply.github.com> Date: Fri, 12 Jul 2024 08:09:08 +0100 Subject: [PATCH 2/8] Add autosectionlabel_prefix_document --- doc/conf.py | 2 ++ doc/index.rst | 5 ++--- doc/modules/core.rst | 2 -- examples/tutorials/README.rst | 2 +- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/doc/conf.py b/doc/conf.py index 4373ec3c36..46f9ae5933 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -74,6 +74,8 @@ "IPython.sphinxext.ipython_console_highlighting" ] +autosectionlabel_prefix_document = True + numpydoc_show_class_members = False diff --git a/doc/index.rst b/doc/index.rst index 3fd779d89f..6a13b12607 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -12,14 +12,13 @@ SpikeInterface is a Python module to analyze extracellular electrophysiology dat With a few lines of code, SpikeInterface enables you to load and pre-process the recording, run several state-of-the-art spike sorters, post-process and curate the output, compute quality metrics, and visualize the results. - .. warning:: a major API improvement: the :code:`SortingAnalyzer`. To read more about this, checkout the `enhancement proposal `_. Please refer to the `stable documentation `_. - Learn how to :ref:`update your code here ` and read more about the - :ref:`SortingAnalyzer here `. + Learn how to :ref:`update your code here ` and read more about the + :ref:`SortingAnalyzer here `. diff --git a/doc/modules/core.rst b/doc/modules/core.rst index 66c023b5be..d45115851f 100644 --- a/doc/modules/core.rst +++ b/doc/modules/core.rst @@ -163,8 +163,6 @@ Internally, any sorting object can construct 2 internal caches: 2. a unique numpy.array with structured dtype aka "spikes vector". This is useful for processing by small chunks of time, like for extracting amplitudes from a recording. -.. _core-sorting-analyzer: - SortingAnalyzer --------------- diff --git a/examples/tutorials/README.rst b/examples/tutorials/README.rst index 0697f0ee60..3f7dd10ecc 100644 --- a/examples/tutorials/README.rst +++ b/examples/tutorials/README.rst @@ -3,7 +3,7 @@ Tutorials Longer form tutorials about using spikeinterface. Many of these are downloadable as notebooks or python scripts so that you can "code along" to the tutorials. -If you're new to SpikeInterface, we recommend trying out the :ref:`Quickstart tutorial` first. +If you're new to SpikeInterface, we recommend trying out the :ref:`get_started/quickstart:Quickstart tutorial` first. Updating from legacy -------------------- From 8fea18adbfd82a8045a0eaaefe6a7fa06bf347d1 Mon Sep 17 00:00:00 2001 From: chrishalcrow <57948917+chrishalcrow@users.noreply.github.com> Date: Fri, 12 Jul 2024 08:40:07 +0100 Subject: [PATCH 3/8] Update generation docs --- doc/api.rst | 75 +++++++++---------- doc/get_started/install_sorters.rst | 2 +- doc/get_started/installation.rst | 2 +- doc/modules/core.rst | 2 +- doc/modules/exporters.rst | 2 +- doc/modules/postprocessing.rst | 4 +- doc/modules/preprocessing.rst | 2 +- src/spikeinterface/core/generate.py | 2 + .../generation/drifting_generator.py | 2 + 9 files changed, 48 insertions(+), 45 deletions(-) diff --git a/doc/api.rst b/doc/api.rst index 3e825084e7..40648f8410 100644 --- a/doc/api.rst +++ b/doc/api.rst @@ -338,59 +338,58 @@ spikeinterface.curation spikeinterface.generation ------------------------- +.. currentmodule:: spikeinterface.generation + Core ~~~~ -.. automodule:: spikeinterface.generation - - .. autofunction:: generate_recording - .. autofunction:: generate_sorting - .. autofunction:: generate_snippets - .. autofunction:: generate_templates - .. autofunction:: generate_recording_by_size - .. autofunction:: generate_ground_truth_recording - .. autofunction:: add_synchrony_to_sorting - .. autofunction:: synthesize_random_firings - .. autofunction:: inject_some_duplicate_units - .. autofunction:: inject_some_split_units - .. autofunction:: synthetize_spike_train_bad_isi - .. autofunction:: inject_templates - .. autofunction:: noise_generator_recording - .. autoclass:: InjectTemplatesRecording - .. autoclass:: NoiseGeneratorRecording + + +.. autofunction:: generate_recording +.. autofunction:: generate_sorting +.. autofunction:: generate_snippets +.. autofunction:: generate_templates +.. autofunction:: generate_recording_by_size +.. autofunction:: generate_ground_truth_recording +.. autofunction:: add_synchrony_to_sorting +.. autofunction:: synthesize_random_firings +.. autofunction:: inject_some_duplicate_units +.. autofunction:: inject_some_split_units +.. autofunction:: synthetize_spike_train_bad_isi +.. autofunction:: inject_templates +.. autofunction:: noise_generator_recording +.. autoclass:: InjectTemplatesRecording +.. autoclass:: NoiseGeneratorRecording Drift ~~~~~ -.. automodule:: spikeinterface.generation - .. autofunction:: generate_drifting_recording - .. autofunction:: generate_displacement_vector - .. autofunction:: make_one_displacement_vector - .. autofunction:: make_linear_displacement - .. autofunction:: move_dense_templates - .. autofunction:: interpolate_templates - .. autoclass:: DriftingTemplates - .. autoclass:: InjectDriftingTemplatesRecording +.. autofunction:: generate_drifting_recording +.. autofunction:: generate_displacement_vector +.. autofunction:: make_one_displacement_vector +.. autofunction:: make_linear_displacement +.. autofunction:: move_dense_templates +.. autofunction:: interpolate_templates +.. autoclass:: DriftingTemplates +.. autoclass:: InjectDriftingTemplatesRecording Hybrid ~~~~~~ -.. automodule:: spikeinterface.generation - .. autofunction:: generate_hybrid_recording - .. autofunction:: estimate_templates_from_recording - .. autofunction:: select_templates - .. autofunction:: scale_template_to_range - .. autofunction:: relocate_templates - .. autofunction:: fetch_template_object_from_database - .. autofunction:: fetch_templates_database_info - .. autofunction:: list_available_datasets_in_template_database - .. autofunction:: query_templates_from_database +.. autofunction:: generate_hybrid_recording +.. autofunction:: estimate_templates_from_recording +.. autofunction:: select_templates +.. autofunction:: scale_template_to_range +.. autofunction:: relocate_templates +.. autofunction:: fetch_template_object_from_database +.. autofunction:: fetch_templates_database_info +.. autofunction:: list_available_datasets_in_template_database +.. autofunction:: query_templates_from_database Noise ~~~~~ -.. automodule:: spikeinterface.generation - .. autofunction:: generate_noise +.. autofunction:: generate_noise spikeinterface.sortingcomponents diff --git a/doc/get_started/install_sorters.rst b/doc/get_started/install_sorters.rst index c1666352b1..12233784fd 100644 --- a/doc/get_started/install_sorters.rst +++ b/doc/get_started/install_sorters.rst @@ -27,7 +27,7 @@ sorters to retrieve installation instructions for other operating systems. We use **pip** to install packages, but **conda** should also work in many cases. Some novel spike sorting algorithms are implemented directly in SpikeInterface using the -:py:mod:`spikeinterface.sortingcomponents` module. Checkout the :ref:`SpikeInterface-based spike sorters` section of this page +:py:mod:`spikeinterface.sortingcomponents` module. Checkout the :ref:`get_started/install_sorters:SpikeInterface-based spike sorters` section of this page for more information! If you experience installation problems please directly contact the authors of these tools or write on the diff --git a/doc/get_started/installation.rst b/doc/get_started/installation.rst index aae9cdf63c..182ce67b94 100644 --- a/doc/get_started/installation.rst +++ b/doc/get_started/installation.rst @@ -102,4 +102,4 @@ Sub-modules have more dependencies, so you should also install: All external spike sorters can be either run inside containers (Docker or Singularity - see :ref:`containerizedsorters`) -or must be installed independently (see :ref:`Installing Spike Sorters`). +or must be installed independently (see :ref:`get_started/install_sorters:Installing Spike Sorters`). diff --git a/doc/modules/core.rst b/doc/modules/core.rst index d45115851f..ef82fcbdbf 100644 --- a/doc/modules/core.rst +++ b/doc/modules/core.rst @@ -178,7 +178,7 @@ to perform further analysis, such as calculating :code:`waveforms` and :code:`te Importantly, the :py:class:`~spikeinterface.core.SortingAnalyzer` handles the *sparsity* and the physical *scaling*. Sparsity defines the channels on which waveforms and templates are calculated using, for example, a -physical distance from the channel with the largest peak amplitude (see the :ref:`Sparsity` section). Scaling, set by +physical distance from the channel with the largest peak amplitude (see the :ref:`modules/core:Sparsity` section). Scaling, set by the :code:`return_scaled` argument, determines whether the data is converted from integer values to :math:`\mu V` or not. By default, :code:`return_scaled` is true and all processed data voltage values are in :math:`\mu V` (e.g., waveforms, templates, spike amplitudes, etc.). diff --git a/doc/modules/exporters.rst b/doc/modules/exporters.rst index 580a21f74d..7f8eeeb19e 100644 --- a/doc/modules/exporters.rst +++ b/doc/modules/exporters.rst @@ -15,7 +15,7 @@ results. **Note** : :py:func:`~spikeinterface.exporters.export_to_phy` speed and the size of the folder will highly depend on the sparsity of the :code:`SortingAnalyzer` itself or the external specified sparsity. The Phy viewer enables one to explore PCA projections, spike amplitudes, waveforms and quality of spike sorting results. -So if these pieces of information have already been computed as extensions (see :ref:`Extensions as AnalyzerExtensions`), +So if these pieces of information have already been computed as extensions (see :ref:`modules/postprocessing:Extensions as AnalyzerExtensions`), then exporting to Phy should be fast (and the user has better control of the parameters for the extensions). If not pre-computed, then the required extensions (e.g., :code:`spike_amplitudes`, :code:`principal_components`) can be computed directly at export time. diff --git a/doc/modules/postprocessing.rst b/doc/modules/postprocessing.rst index dfa4c8b517..7737986bdf 100644 --- a/doc/modules/postprocessing.rst +++ b/doc/modules/postprocessing.rst @@ -59,7 +59,7 @@ To check what extensions spikeinterface can calculate, you can use the :code:`ge >>> ['random_spikes', 'waveforms', 'templates', 'noise_levels', 'amplitude_scalings', 'correlograms', 'isi_histograms', 'principal_components', 'spike_amplitudes', 'spike_locations', 'template_metrics', 'template_similarity', 'unit_locations', 'quality_metrics'] -There is detailed documentation about each extension :ref:`below`. +There is detailed documentation about each extension :ref:`below`. Each extension comes from a different module. To use the :code:`postprocessing` extensions, you'll need to have the `postprocessing` module loaded. @@ -310,7 +310,7 @@ By default, the following metrics are computed: The units of :code:`recovery_slope` and :code:`repolarization_slope` depend on the input. Voltages are based on the units of the template. By default this is :math:`\mu V` but can be the raw output from the recording device (this depends on the -:code:`return_scaled` parameter, read more here: :ref:`core-sorting-analyzer`). +:code:`return_scaled` parameter, read more here: :ref:`modules/core:SortingAnalyzer`). Distances are in :math:`\mu m` and times are in seconds. So, for example, if the templates are in units of :math:`\mu V` then: :code:`repolarization_slope` is in :math:`mV / s`; :code:`peak_to_trough_ratio` is in :math:`\mu m` and the diff --git a/doc/modules/preprocessing.rst b/doc/modules/preprocessing.rst index 0b71b49d3f..4fbadd3ab1 100644 --- a/doc/modules/preprocessing.rst +++ b/doc/modules/preprocessing.rst @@ -32,7 +32,7 @@ These two preprocessors will not compute anything at instantiation, but the comp traces = recording_cmr.get_traces(start_frame=100_000, end_frame=200_000) -Some internal sorters (see :ref:`Intertnal Sorters`) can work directly on these preprocessed objects so there is no need to +Some internal sorters (see :ref:`modules/sorters:Intertnal Sorters`) can work directly on these preprocessed objects so there is no need to save the object: .. code-block:: python diff --git a/src/spikeinterface/core/generate.py b/src/spikeinterface/core/generate.py index 6ce94114c4..db57d028f7 100644 --- a/src/spikeinterface/core/generate.py +++ b/src/spikeinterface/core/generate.py @@ -1677,6 +1677,7 @@ class InjectTemplatesRecording(BaseRecording): templates: np.ndarray[n_units, n_samples, n_channels] or np.ndarray[n_units, n_samples, n_oversampling] Array containing the templates to inject for all the units. Shape can be: + * (num_units, num_samples, num_channels): standard case * (num_units, num_samples, num_channels, upsample_factor): case with oversample template to introduce sampling jitter. nbefore: list[int] | int | None, default: None @@ -2057,6 +2058,7 @@ def generate_ground_truth_recording( The templates of units. If None they are generated. Shape can be: + * (num_units, num_samples, num_channels): standard case * (num_units, num_samples, num_channels, upsample_factor): case with oversample template to introduce jitter. ms_before: float, default: 1.5 diff --git a/src/spikeinterface/generation/drifting_generator.py b/src/spikeinterface/generation/drifting_generator.py index a0e8ece37e..b439c57c52 100644 --- a/src/spikeinterface/generation/drifting_generator.py +++ b/src/spikeinterface/generation/drifting_generator.py @@ -339,11 +339,13 @@ def generate_drifting_recording( Same for both recordings. extra_infos: If extra_outputs=True, then return also a dict that contain various information like: + * displacement_vectors * displacement_sampling_frequency * unit_locations * displacement_unit_factor * unit_displacements + This can be helpfull for motion benchmark. """ From f187e4c122d7001e58f97a37a9af60966450931b Mon Sep 17 00:00:00 2001 From: chrishalcrow Date: Fri, 12 Jul 2024 09:03:19 +0100 Subject: [PATCH 4/8] Warnings from linux run --- doc/modules/widgets.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/modules/widgets.rst b/doc/modules/widgets.rst index 4d69867d83..5bf0658e99 100644 --- a/doc/modules/widgets.rst +++ b/doc/modules/widgets.rst @@ -30,8 +30,8 @@ backends can be installed with: pip install spikeinterface[widgets] -matplotlib -^^^^^^^^^^ +Install matplotlib +^^^^^^^^^^^^^^^^^^ The :code:`matplotlib` backend (default) uses the :code:`matplotlib` package to generate static figures. @@ -41,8 +41,8 @@ To install it, run: pip install matplotlib -ipywidgets -^^^^^^^^^^ +Install ipywidgets +^^^^^^^^^^^^^^^^^^ The :code:`ipywidgets` backend allows users to interact with the plot, for example, by selecting units or scrolling through a time series. @@ -62,8 +62,8 @@ To enable interactive widgets in your notebook, add and run a cell with: .. _sorting_view: -sortingview -^^^^^^^^^^^ +Install sortingview +^^^^^^^^^^^^^^^^^^^ The :code:`sortingview` backend generates web-based and shareable links that can be viewed in the browser. @@ -89,8 +89,8 @@ Finally, if you wish to set up another cloud provider, follow the instruction fr `kachery-cloud `_ package ("Using your own storage bucket"). -ephyviewer -^^^^^^^^^^ +Install ephyviewer +^^^^^^^^^^^^^^^^^^ This backend is Qt based with PyQt5, PyQt6 or PySide6 support. Qt is sometimes tedious to install. From 86ac4f1e4adf8467fd91b8a9d211fc995306c9df Mon Sep 17 00:00:00 2001 From: chrishalcrow Date: Fri, 12 Jul 2024 09:05:41 +0100 Subject: [PATCH 5/8] Delete QM references, use main references page instead --- doc/modules/qualitymetrics/references.rst | 2 -- 1 file changed, 2 deletions(-) delete mode 100644 doc/modules/qualitymetrics/references.rst diff --git a/doc/modules/qualitymetrics/references.rst b/doc/modules/qualitymetrics/references.rst deleted file mode 100644 index 72829b345a..0000000000 --- a/doc/modules/qualitymetrics/references.rst +++ /dev/null @@ -1,2 +0,0 @@ -References ----------- From f9932319a323021f550911f3380dd7db5db5a184 Mon Sep 17 00:00:00 2001 From: Chris Halcrow <57948917+chrishalcrow@users.noreply.github.com> Date: Fri, 12 Jul 2024 09:17:25 +0100 Subject: [PATCH 6/8] Apply suggestions from code review Undo a `.` -> `_` and delete redundant label --- doc/how_to/viewers.rst | 2 +- doc/modules/core.rst | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/how_to/viewers.rst b/doc/how_to/viewers.rst index 9486755c85..c7574961bd 100644 --- a/doc/how_to/viewers.rst +++ b/doc/how_to/viewers.rst @@ -11,7 +11,7 @@ There are several ways to plot signals (raw, preprocessed) and spikes. 4. You can use the :code:`phy` software -spikeinterface-widgets +spikeinterface.widgets ---------------------- The easiest way to visualize :code:`spikeinterface` objects is to use the :code:`widgets` module for plotting. diff --git a/doc/modules/core.rst b/doc/modules/core.rst index ef82fcbdbf..7f8620a261 100644 --- a/doc/modules/core.rst +++ b/doc/modules/core.rst @@ -21,7 +21,6 @@ All classes support: * data on-demand (lazy loading) * multiple segments, where each segment is a contiguous piece of data (recording, sorting, events). -.. _core-recording: Recording --------- From d70129d193223e8db8a7dfebc51ad3468e3a915e Mon Sep 17 00:00:00 2001 From: chrishalcrow <57948917+chrishalcrow@users.noreply.github.com> Date: Fri, 12 Jul 2024 11:40:50 +0100 Subject: [PATCH 7/8] delete reference link --- doc/modules/qualitymetrics.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/doc/modules/qualitymetrics.rst b/doc/modules/qualitymetrics.rst index f5f3581c31..04a302c597 100644 --- a/doc/modules/qualitymetrics.rst +++ b/doc/modules/qualitymetrics.rst @@ -67,12 +67,3 @@ This code snippet shows how to compute quality metrics (with or without principa For more information about quality metrics, check out this excellent `documentation `_ from the Allen Institute. - - -Quality Metrics References --------------------------- - -.. toctree:: - :maxdepth: 1 - - qualitymetrics/references From 4823068f50274e67bbaf4f9a628c56a2f3558c14 Mon Sep 17 00:00:00 2001 From: chrishalcrow <57948917+chrishalcrow@users.noreply.github.com> Date: Fri, 12 Jul 2024 12:00:42 +0100 Subject: [PATCH 8/8] Ignore generate_ python file --- doc/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/conf.py b/doc/conf.py index 46f9ae5933..e3d58ca8f2 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -130,7 +130,7 @@ '../examples/tutorials/widgets', ]), 'within_subsection_order': FileNameSortKey, - 'ignore_pattern': '/generate_', + 'ignore_pattern': '/generate_*', 'nested_sections': False, 'copyfile_regex': r'.*\.rst|.*\.png|.*\.svg' }