Use edfio for reading and writing edf files

.github/workflows/release.yml

@@ -47,7 +47,7 @@ jobs:
  - uses: actions/setup-python@v4
  name: Install Python
  with:
- python-version: 3.8
+ python-version: 3.9
 
  - name: Build sdist
  run: |

.pre-commit-config.yaml

@@ -24,6 +24,7 @@ repos:
  hooks:
  - id: mypy
  exclude: ^sleepecg/test/|^examples
+ args: [--python-version=3.9]
  additional_dependencies:
  - types-PyYAML
  - types-requests

CHANGELOG.md

@@ -1,4 +1,6 @@
 ## [UNRELEASED] - YYYY-MM-DD
+### Changed
+- Use [edfio](https://github.com/the-siesta-group/edfio) for reading and writing EDF files ([#195](https://github.com/cbrnr/sleepecg/pull/195) by [Florian Hofer](https://github.com/hofaflo))
 
 ## [0.5.5] - 2023-06-01
 ### Changed

CONTRIBUTING.md

@@ -35,7 +35,7 @@ If another PR is merged while you are working on something, a merge conflict may
 
 
 ## Development environment
-Make sure to use Python 3.8. You might want to [create a virtual environment](https://docs.python.org/3/library/venv.html#creating-virtual-environments) instead of working your main environment. In the root of the local clone of your fork, install SleepECG as follows:
+Make sure to use Python 3.9. You might want to [create a virtual environment](https://docs.python.org/3/library/venv.html#creating-virtual-environments) instead of working your main environment. In the root of the local clone of your fork, install SleepECG as follows:
 
 ```
 pip install -e ".[dev]"

README.md

@@ -52,18 +52,16 @@ The following example detects heartbeats in a short ECG (a one-dimensional NumPy
 
 ```python
 import numpy as np
-from scipy.datasets import electrocardiogram
-from sleepecg import detect_heartbeats
+from sleepecg import detect_heartbeats, get_toy_ecg
 
-ecg = electrocardiogram() # 5 min of ECG data at 360 Hz
-fs = 360 # sampling frequency
+ecg, fs = get_toy_ecg() # 5 min of ECG data at 360 Hz
 beats = detect_heartbeats(ecg, fs) # indices of detected heartbeats
 ```
 
 
 ### Dependencies
 
-SleepECG requires Python ≥ 3.8 and the following packages:
+SleepECG requires Python ≥ 3.9 and the following packages:
 
 - [numpy](https://numpy.org/) ≥ 1.20.0
 - [requests](https://requests.readthedocs.io/en/latest/) ≥ 2.25.0
@@ -73,9 +71,9 @@ SleepECG requires Python ≥ 3.8 and the following packages:
 
 Optional dependencies provide additional features:
 
+- [edfio](https://github.com/the-siesta-group/edfio/) ≥ 0.1.1 (read data from [MESA](https://sleepdata.org/datasets/mesa) and [SHHS](https://sleepdata.org/datasets/shhs))
 - [joblib](https://joblib.readthedocs.io/en/latest/) ≥ 1.0.0 (parallelized feature extraction)
 - [matplotlib](https://matplotlib.org/) ≥ 3.5.0 (plot ECG time courses, hypnograms, and confusion matrices)
-- [mne](https://mne.tools/stable/index.html) ≥ 1.0.0 (read data from [MESA](https://sleepdata.org/datasets/mesa) and [SHHS](https://sleepdata.org/datasets/shhs))
 - [numba](https://numba.pydata.org/) ≥ 0.55.0 (JIT-compiled heartbeat detector)
 - [tensorflow](https://www.tensorflow.org/) ≥ 2.7.0 (sleep stage classification with Keras models)
 - [wfdb](https://github.com/MIT-LCP/wfdb-python/) ≥ 3.4.0 (read data from [SLPDB](https://physionet.org/content/slpdb), [MITDB](https://physionet.org/content/mitdb), and [LTDB](https://physionet.org/content/ltdb))

docs/api/heartbeat_detection.md

@@ -3,3 +3,4 @@
 ::: sleepecg.compare_heartbeats
 ::: sleepecg.detect_heartbeats
 ::: sleepecg.rri_similarity
+::: sleepecg.get_toy_ecg

docs/examples.md

@@ -86,27 +86,27 @@ plt.show()
 
 ## Sleep staging custom data
 
-This example requires `mne` and `tensorflow` packages. In addition, it uses an example file [`sleep.edf`](https://osf.io/download/mx7av/), which contains ECG data for a whole night. Download and save this file in your working directory before running this example.
+This example requires `edfio` and `tensorflow` packages. In addition, it uses an example file [`sleep.edf`](https://osf.io/download/mx7av/), which contains ECG data for a whole night. Download and save this file in your working directory before running this example.
 
 ```python
-from datetime import datetime, timezone
+from datetime import datetime
 
-from mne.io import read_raw_edf
+from edfio import read_edf
 
 import sleepecg
 
 # load dataset
-raw = read_raw_edf("sleep.edf", include="ECG")
-raw.set_channel_types({"ECG": "ecg"})
-fs = raw.info["sfreq"]
+edf = read_edf("sleep.edf")
 
 # crop dataset (we only want data for the sleep duration)
-start = datetime(2023, 3, 1, 23, 0, 0, tzinfo=timezone.utc)
-stop = datetime(2023, 3, 2, 6, 0, 0, tzinfo=timezone.utc)
-raw.crop((start - raw.info["meas_date"]).seconds, (stop - raw.info["meas_date"]).seconds)
-
-# get ECG time series as 1D NumPy array
-ecg = raw.get_data().squeeze()
+start = datetime(2023, 3, 1, 23, 0, 0)
+stop = datetime(2023, 3, 2, 6, 0, 0)
+rec_start = datetime.combine(edf.startdate, edf.starttime)
+edf.slice_between_seconds((start - rec_start).seconds, (stop - rec_start).seconds)
+
+# get ECG time series and sampling frequency
+ecg = edf.get_signal("ECG").data
+fs = edf.get_signal("ECG").sampling_frequency
 
 # detect heartbeats
 beats = sleepecg.detect_heartbeats(ecg, fs)

docs/heartbeat_detection.md

@@ -19,11 +19,9 @@ Let's detect heartbeats in a short electrocardiogram:
 
 ```python
 import numpy as np
-from scipy.misc import electrocardiogram
-from sleepecg import detect_heartbeats
+from sleepecg import detect_heartbeats, get_toy_ecg
 
-ecg = electrocardiogram() # 5 min of ECG data at 360 Hz
-fs = 360
+ecg, fs = get_toy_ecg() # 5 min of ECG data at 360 Hz
 beats = detect_heartbeats(ecg, fs)
 ```
 

docs/index.md

@@ -12,7 +12,7 @@ Documentation for SleepECG is available on [Read the Docs](https://sleepecg.read
 Check out the [changelog](https://github.com/cbrnr/sleepecg/blob/main/CHANGELOG.md) to learn what we added, changed, or fixed.
 
 ## Dependencies
-SleepECG requires Python ≥ 3.8 and the following packages:
+SleepECG requires Python ≥ 3.9 and the following packages:
 
 - [numpy](https://numpy.org/) ≥ 1.20.0
 - [requests](https://requests.readthedocs.io/en/latest/) ≥ 2.25.0
@@ -22,9 +22,9 @@ SleepECG requires Python ≥ 3.8 and the following packages:
 
 Optional dependencies provide additional features:
 
+- [edfio](https://github.com/the-siesta-group/edfio/) ≥ 0.1.1 (read data from [MESA](https://sleepdata.org/datasets/mesa) and [SHHS](https://sleepdata.org/datasets/shhs))
 - [joblib](https://joblib.readthedocs.io/en/latest/) ≥ 1.0.0 (parallelized feature extraction)
 - [matplotlib](https://matplotlib.org/) ≥ 3.5.0 (plot ECG time courses, hypnograms, and confusion matrices)
-- [mne](https://mne.tools/stable/index.html) ≥ 1.0.0 (read data from [MESA](https://sleepdata.org/datasets/mesa) and [SHHS](https://sleepdata.org/datasets/shhs))
 - [numba](https://numba.pydata.org/) ≥ 0.55.0 (JIT-compiled heartbeat detector)
 - [tensorflow](https://www.tensorflow.org/) ≥ 2.7.0 (sleep stage classification with Keras models)
 - [wfdb](https://github.com/MIT-LCP/wfdb-python/) ≥ 3.4.0 (read data from [SLPDB](https://physionet.org/content/slpdb), [MITDB](https://physionet.org/content/mitdb), and [LTDB](https://physionet.org/content/ltdb))

docs/plot.md

@@ -5,12 +5,10 @@ If [Matplotlib](https://matplotlib.org/) is installed, SleepECG can create usefu
 The function [`sleepecg.plot_ecg()`][sleepecg.plot_ecg] plots the time course of an ECG signal, optionally with one or more markers (such as detected heart beats). The following example demonstrates this functionality with toy data:
 
 ```python
-from scipy.misc import electrocardiogram
 import sleepecg
 
-fs = 360
-ecg = electrocardiogram()[:10 * fs]
-beats = sleepecg.detect_heartbeats(ecg, fs)
+ecg, fs = sleepecg.get_toy_ecg() # 5 min of ECG data at 360 Hz
+beats = sleepecg.detect_heartbeats(ecg[:10 * fs], fs)
 
 sleepecg.plot_ecg(ecg, fs, correct=beats, bad=beats + 7)
 ```
@@ -22,11 +20,9 @@ In this example, we plotted two different annotations, `beats` (the detected hea
 Similarly, a [`sleepecg.ECGRecord`][sleepecg.ECGRecord] can be visualized with its [`sleepecg.ECGRecord.plot()`][sleepecg.ECGRecord.plot] method:
 
 ```python
-from scipy.misc import electrocardiogram
 import sleepecg
 
-fs = 360
-ecg = electrocardiogram()[:10 * fs]
+ecg, fs = sleepecg.get_toy_ecg() # 5 min of ECG data at 360 Hz
 beats = sleepecg.detect_heartbeats(ecg, fs)
 
 record = sleepecg.ECGRecord(

pyproject.toml

@@ -10,11 +10,10 @@ authors = [
  {name = "Florian Hofer", email = "[email protected]"},
  {name = "Clemens Brunner", email = "[email protected]"},
 ]
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 classifiers = [
  "License :: OSI Approved :: BSD License",
  "Programming Language :: Python :: 3",
- "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
@@ -33,25 +32,24 @@ dynamic = ["version", "readme"]
 
 [project.optional-dependencies]
 full = [ # complete package functionality
+ "edfio >= 0.1.1",
  "joblib >= 1.0.0",
  "matplotlib >= 3.5.0",
- "mne >= 1.0.0",
  "numba >= 0.55.0; python_version < '3.12'",
  "tensorflow >= 2.7.0; python_version < '3.12'",
  "wfdb >= 3.4.0",
 ]
 
 dev = [ # everything needed for development
  "black >= 22.6.0",
+ "edfio >= 0.1.1",
  "joblib >= 1.0.0",
  "matplotlib >= 3.5.0",
  "mkdocs-material >= 8.4.0",
  "mkdocstrings-python >= 0.7.1",
- "mne >= 1.0.0",
  "mypy >= 0.991",
  "numba >= 0.55.0; python_version < '3.12'",
  "pre-commit >= 2.13.0",
- "pyEDFlib >= 0.1.25",
  "pytest >= 6.2.0",
  "ruff >= 0.0.263",
  "setuptools >= 56.0.0",
@@ -65,9 +63,8 @@ doc = [ # RTD uses this when building the documentation
 ]
 
 cibw = [ # cibuildwheel uses this for running the test suite
- "mne >= 1.0.0",
+ "edfio >= 0.1.1",
  "numba >= 0.55.0; python_version < '3.12'",
- "pyEDFlib >= 0.1.25",
  "wfdb >= 3.4.0",
 ]
 
@@ -109,7 +106,6 @@ pretty = true
 markers = ["c_extension"]
 filterwarnings = [
  "error",
- 'ignore:.*sample_rate:DeprecationWarning:pyedflib',
  'ignore:.*pkg_resources:DeprecationWarning',
  'ignore:.*datetime.datetime.utcfromtimestamp\(\):DeprecationWarning',
 ]

sleepecg/__init__.py

@@ -15,5 +15,6 @@
 from sleepecg.heartbeats import compare_heartbeats, detect_heartbeats, rri_similarity
 from sleepecg.io import * # noqa: F403
 from sleepecg.plot import plot_ecg, plot_hypnogram
+from sleepecg.utils import get_toy_ecg
 
 __version__ = "0.6.0-dev"

sleepecg/config.py

@@ -4,8 +4,6 @@
 
 """Functions for getting and setting configuration values."""
 
-from __future__ import annotations
-
 from pathlib import Path
 from typing import Any, Optional
 

sleepecg/heartbeats.py

@@ -4,8 +4,6 @@
 
 """Heartbeat detection and detector evaluation."""
 
-from __future__ import annotations
-
 import warnings
 from typing import NamedTuple
 
@@ -83,10 +81,8 @@ def detect_heartbeats(ecg: np.ndarray, fs: float, backend: str = "c") -> np.ndar
  --------
  Detect heartbeats in a short electrocardiogram:
 
- >>> from scipy.misc import electrocardiogram
- >>> from sleepecg import detect_heartbeats
- >>> ecg = electrocardiogram() # 5 min of ECG data at 360 Hz
- >>> fs = 360
+ >>> from sleepecg import detect_heartbeats, get_toy_ecg
+ >>> ecg, fs = get_toy_ecg() # 5 min of ECG data at 360 Hz
  >>> heartbeats = detect_heartbeats(ecg, fs)
  >>> print(f"{len(heartbeats)} heartbeats detected")
  478 heartbeats detected

sleepecg/io/sleep_readers.py

@@ -202,7 +202,7 @@ def read_mesa(
  SleepRecord
  Each element in the generator is of type `SleepRecord`.
  """
- from mne.io import read_raw_edf
+ from edfio import read_edf
 
  DB_SLUG = "mesa"
  ANNOTATION_DIRNAME = "polysomnography/annotations-events-nsrr"
@@ -330,11 +330,9 @@ def read_mesa(
  checksums[edf_filename],
  )
 
- rec = read_raw_edf(edf_filepath, verbose=False)
- ecg = rec.get_data("EKG").ravel()
- fs = rec.info["sfreq"]
- heartbeat_indices = detect_heartbeats(ecg, fs)
- heartbeat_times = heartbeat_indices / fs
+ ecg = read_edf(edf_filepath).get_signal("EKG")
+ heartbeat_indices = detect_heartbeats(ecg.data, ecg.sampling_frequency)
+ heartbeat_times = heartbeat_indices / ecg.sampling_frequency
  np.save(heartbeats_file, heartbeat_times)
 
  if not edf_was_available and not keep_edfs:
@@ -513,7 +511,7 @@ def read_shhs(
  SleepRecord
  Each element in the generator is of type `SleepRecord`.
  """
- from mne.io import read_raw_edf
+ from edfio import read_edf
 
  DB_SLUG = "shhs"
  ANNOTATION_DIRNAME = "polysomnography/annotations-events-nsrr"
@@ -643,11 +641,9 @@ def read_shhs(
  checksums[edf_filename],
  )
 
- rec = read_raw_edf(edf_filepath, verbose=False)
- ecg = rec.get_data("ECG").ravel()
- fs = rec.info["sfreq"]
- heartbeat_indices = detect_heartbeats(ecg, fs)
- heartbeat_times = heartbeat_indices / fs
+ ecg = read_edf(edf_filepath).get_signal("ECG")
+ heartbeat_indices = detect_heartbeats(ecg.data, ecg.sampling_frequency)
+ heartbeat_times = heartbeat_indices / ecg.sampling_frequency
 
  heartbeats_file.parent.mkdir(parents=True, exist_ok=True)
  np.save(heartbeats_file, heartbeat_times)

sleepecg/plot.py

@@ -4,8 +4,6 @@
 
 """Plotting functions."""
 
-from __future__ import annotations
-
 from itertools import cycle
 from typing import TYPE_CHECKING, Optional
 

sleepecg/test/test_heartbeat_detection.py

@@ -8,18 +8,12 @@
 
 import numpy as np
 import pytest
-
-try:
- from scipy.datasets import electrocardiogram # SciPy ≥ 1.10
-except ImportError:
- from scipy.misc import electrocardiogram # SciPy < 1.10
 from scipy.signal import resample_poly
 
-from sleepecg import compare_heartbeats, detect_heartbeats
+from sleepecg import compare_heartbeats, detect_heartbeats, get_toy_ecg
 
 pytestmark = pytest.mark.c_extension
-ecg = electrocardiogram()
-fs = 360
+ecg, fs = get_toy_ecg()
 y_true = detect_heartbeats(ecg, fs) # 478 true peaks
 
 

sleepecg/test/test_sleep_readers.py

@@ -10,24 +10,17 @@
 from pathlib import Path
 
 import numpy as np
+from edfio import Edf, EdfSignal
 
-try:
- from scipy.datasets import electrocardiogram # SciPy ≥ 1.10
-except ImportError:
- from scipy.misc import electrocardiogram # SciPy < 1.10
-from pyedflib import highlevel
-
-from sleepecg import SleepStage, read_mesa, read_shhs, read_slpdb
+from sleepecg import SleepStage, get_toy_ecg, read_mesa, read_shhs, read_slpdb
 from sleepecg.io.sleep_readers import Gender
 
 
 def _dummy_nsrr_edf(filename: str, hours: float, ecg_channel: str):
- ECG_FS = 360
- ecg_5_min = electrocardiogram()
+ ecg_5_min, fs = get_toy_ecg()
  seconds = int(hours * 60 * 60)
- ecg = np.tile(ecg_5_min, int(np.ceil(seconds / 300)))[np.newaxis, : seconds * ECG_FS]
- signal_headers = highlevel.make_signal_headers([ecg_channel], sample_frequency=ECG_FS)
- highlevel.write_edf(filename, ecg, signal_headers)
+ ecg = np.tile(ecg_5_min, int(np.ceil(seconds / 300)))[: seconds * fs]
+ Edf([EdfSignal(ecg, fs, label=ecg_channel)]).write(filename)
 
 
 def _dummy_nsrr_xml(filename: str, hours: float, random_state: int):