Remove eofs (#1621)

### What kind of change does this PR introduce? * The `eofs` library has been removed from the optional dependencies. * The empirical orthogonal functionality (`xclim.sdba.properties.first_eof` and associated test) has been removed from the code base. * CI builds no longer test `eofs` functionality. ### Does this PR introduce a breaking change? Yes. `eofs` has been completely removed since this violated its GPLv3 license. Associated functionality is reproducible by users ### Other information: Should we be offering users a note about how to migrate? I feel like it's not necessary.
Ouranosinc · Jan 25, 2024 · 36d49e9 · 36d49e9
2 parents a569e91 + 5dfa324
commit 36d49e9
Show file tree

Hide file tree

Showing 8 changed files with 23 additions and 134 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -115,10 +115,10 @@ jobs:
     strategy:
       matrix:
         include:
-          - tox-env: py38-coverage-eofs
+          - tox-env: py38-coverage
             python-version: "3.8"
             markers: -m 'not slow'
-          - tox-env: py39-coverage-sbck-eofs
+          - tox-env: py39-coverage-sbck
             python-version: "3.9"
             markers: -m 'not slow'
           - tox-env: py310-coverage  # No markers -- includes slow tests

diff --git a/CHANGES.rst b/CHANGES.rst
@@ -24,6 +24,7 @@ Breaking changes
 * `bump2version` has been replaced with `bump-my-version` to bump the version number using configurations set in the `pyproject.toml` file. (:issue:`1557`, :pull:`1569`).
 * `xclim`'s units registry and units formatting are now extended from `cf-xarray`. The exponent sign "^" is now never added in the ``units`` attribute. For example, square meters are given as "m2" instead of "m^2" by xclim, both are still accepted as input. (:issue:`1010`, :pull:`1590`).
 * `yamale` is now listed as a core dependency (was previously listed in the `dev` installation recipe). (:issue:`1595`, :pull:`1596`).
+* Due to a licensing limitation, the calculation of empirical orthogonal function  based on `eofs` (``xclim.sdba.properties.first_eof``) has been removed from `xclim`. (:issue:`1620`, :pull:`1621`).
 
 Bug fixes
 ^^^^^^^^^

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -11,7 +11,7 @@ To install `xclim` via `pip`, run this command in your terminal:
 
 .. code-block:: shell
 
-    $ pip install xclim
+    $ python -m pip install xclim
 
 If you don't have `pip`_ installed, this `Python installation guide`_ can guide you through the process.
 
@@ -55,7 +55,7 @@ Both of these libraries are available on PyPI and conda-forge:
 
 .. code-block:: shell
 
-    $ pip install flox clisops
+    $ python -m pip install flox clisops
     # Or, alternatively:
     $ conda install -c conda-forge flox clisops
 
@@ -70,7 +70,7 @@ For convenience, these libraries can be installed alongside `xclim` using the fo
 
 .. code-block:: shell
 
-    $ pip install -r requirements_upstream.txt
+    $ python -m pip install -r requirements_upstream.txt
 
 Or, alternatively:
 
@@ -105,17 +105,8 @@ Afterwards, `SBCK` can be installed from PyPI using `pip`:
 
 .. code-block:: shell
 
-    $ pip install SBCK
+    $ python -m pip install pybind11 sbck
 
-Another experimental function :py:indicator:`xclim.sdba.property.first_eof` makes use of the `eofs`_ library, which is available on both PyPI and conda-forge:
-
-.. code-block:: shell
-
-    $ pip install eofs
-    # or alternatively,
-    $ conda install -c conda-forge eofs
-
-.. _eofs: https://ajdawson.github.io/eofs/
 .. _SBCK: https://github.com/yrobink/SBCK
 .. _Eigen3: https://eigen.tuxfamily.org/index.php
 
@@ -145,7 +136,7 @@ Once you have extracted a copy of the source, you can install it with pip:
 
 .. code-block:: shell
 
-    $ pip install -e ".[dev]"
+    $ python -m pip install -e ".[dev]"
 
 Alternatively, you can also install a local development copy via `flit`_:
 
@@ -160,10 +151,10 @@ Alternatively, you can also install a local development copy via `flit`_:
 Creating a Conda environment
 ----------------------------
 
-To create a conda environment including `xclim`'s dependencies and several optional libraries (notably: `clisops`, `eigen`, `eofs`, and `flox`) and development dependencies, run the following command from within your cloned repo:
+To create a conda environment including `xclim`'s dependencies and several optional libraries (notably: `clisops`, `eigen`, `sbck`, and `flox`) and development dependencies, run the following command from within your cloned repo:
 
 .. code-block:: console
 
     $ conda env create -n my_xclim_env python=3.8 --file=environment.yml
     $ conda activate my_xclim_env
-    (my_xclim_env) $ pip install -e .
+    (my_xclim_env) $ python -m pip install -e .
diff --git a/docs/references.bib b/docs/references.bib
@@ -1835,26 +1835,6 @@ @article{roy_extremeprecip_2023
 	year = {2023},
 }
 
-@article{dawson_eofs_2016,
-	title = {eofs: {A} {Library} for {EOF} {Analysis} of {Meteorological}, {Oceanographic}, and {Climate} {Data}},
-	volume = {4},
-	issn = {2049-9647},
-	shorttitle = {eofs},
-	url = {https://openresearchsoftware.metajnl.com/article/10.5334/jors.122/},
-	doi = {10.5334/jors.122},
-	abstract = {Article: eofs: A Library for EOF Analysis of Meteorological, Oceanographic, and Climate Data},
-	language = {eng},
-	number = {1},
-	urldate = {2022-11-11},
-	journal = {Journal of Open Research Software},
-	author = {Dawson, Andrew},
-	month = apr,
-	year = {2016},
-	note = {Number: 1
-Publisher: Ubiquity Press},
-	pages = {e14},
-}
-
 @article{francois_multivariate_2020,
 	title = {Multivariate bias corrections of climate simulations: which benefits for which losses?},
 	volume = {11},

diff --git a/environment.yml b/environment.yml
@@ -27,7 +27,6 @@ dependencies:
   - xarray >=2022.06.0,<2023.11.0
   - yamale
   # Extras
-  - eofs
   - flox
   # Testing and development dependencies
   - black >=22.12

diff --git a/tests/test_sdba/test_properties.py b/tests/test_sdba/test_properties.py
@@ -515,19 +515,3 @@ def test_get_measure(self, open_dataset):
 
         meas = sdba.properties.var.get_measure()(sim_var, ref_var)
         np.testing.assert_allclose(meas, [0.408327], rtol=1e-3)
-
-
-class TestEOF:
-    def test_first_eof(self, open_dataset):
-        pytest.importorskip("eofs")
-        sim = (
-            open_dataset("NRCANdaily/nrcan_canada_daily_tasmax_1990.nc")
-            .tasmax.isel(lon=slice(0, 10), lat=slice(50, 60))
-            .load()
-        )
-
-        out = sdba.properties.first_eof(sim)
-        np.testing.assert_allclose(
-            [out.mean(), out.max()], [0.099976, 0.103867], rtol=1e-5
-        )
-        assert (out.isnull() == sim.isnull().any("time")).all()
diff --git a/tox.ini b/tox.ini
@@ -5,7 +5,6 @@ env_list =
     docs
     notebooks_doctests
     offline-prefetch
-;    opt-slow
     py38
     py39-upstream-doctest
     py310
@@ -58,8 +57,10 @@ allowlist_externals =
 # Requires tox-conda compatible with [email protected]
 ;[testenv:conda]
 ;description = Run tests with pytest under {basepython} (Anaconda distribution)
+;commands_pre =
 ;conda_channels = conda-forge
 ;conda_env = environment-dev.yml
+;deps =
 ;extras =
 
 [testenv:notebooks_doctests{-coverage,}]
@@ -68,15 +69,6 @@ commands =
     pytest --no-cov --nbval --dist=loadscope --rootdir=tests/ --ignore=docs/notebooks/example.ipynb docs/notebooks
     pytest --rootdir=tests/ --xdoctest xclim
 
-# Requires tox-conda compatible with [email protected]
-;[testenv:opt-{slow,not_slow}]
-;description = Run tests with optional requirements (SBCK (experimental), eofs) and pytest under {basepython} (Anaconda distribution)
-;conda_env = environment-dev.yml
-;commands =
-;    pip check
-;    !slow: pytest xclim -m "not slow" --durations=10
-;    slow: pytest xclim --durations=10
-
 [testenv:offline{-prefetch,}{-coverage,}]
 description = Run tests with pytest under {basepython}, preventing socket connections (except for unix sockets for async support)
 commands:
@@ -112,7 +104,6 @@ deps =
     numba: llvmlite==0.42.0rc1
     coverage: coveralls
     upstream: -rrequirements_upstream.txt
-    eofs: eofs
     sbck: pybind11
 install_command = python -m pip install --no-user {opts} {packages}
 download = True

diff --git a/xclim/sdba/properties.py b/xclim/sdba/properties.py
@@ -28,7 +28,6 @@
 
 from .base import Grouper, map_groups
 from .nbutils import _pairwise_haversine_and_bins
-from .processing import jitter_under_thresh
 from .utils import _pairwise_spearman, copy_all_attrs
 
 
@@ -1202,73 +1201,17 @@ def _bin_corr(corr, distance):
 )
 
 
-def _first_eof(da: xr.DataArray, *, dims=None, kind="+", thresh="1 mm/d", group="time"):
-    """First Empirical Orthogonal Function.
+def first_eof():
+    """EOF Statistical Property (function removed).
 
-    Through principal component analysis (PCA), compute the predominant empirical orthogonal function.
-    The temporal dimension is reduced. The Eof is multiplied by the sign of its mean to ensure coherent
-    signs as much as possible. Needs the eofs package to run. Based on an idea from :cite:p:`vrac_multivariate_2018`,
-    using an implementation from :cite:p:`dawson_eofs_2016`.
-
-    Parameters
-    ----------
-    da: xr.DataArray
-      Data.
-    dims: sequence of string, optional
-      Name of the spatial dimensions. If None (default), all dimensions except "time" are used.
-    kind : {'+', '*'}
-      Variable "kind". If multiplicative, the zero values are set to
-      very small values and the PCA is performed over the logarithm of the data.
-    thresh: str
-      If kind is multiplicative, this is the "zero" threshold passed to
-      :py:func:`xclim.sdba.processing.jitter_under_thresh`.
-    group: str
-      Useless for now.
-
-    Returns
-    -------
-    xr.DataArray, [dimensionless]
-      First empirical orthogonal function
+    Warnings
+    --------
+    Due to a licensing issue, eofs-based functionality has been permanently removed.
+    Please excuse the inconvenience.
+    For more information, see: https://github.com/Ouranosinc/xclim/issues/1620
     """
-    try:
-        from eofs.standard import Eof
-    except ImportError as err:
-        raise ValueError(
-            "The `first_eof` property requires the `eofs` package"
-            ", which is an optional dependency of xclim."
-        ) from err
-
-    if dims is None:
-        dims = [d for d in da.dims if d != "time"]
-
-    if kind == "*":
-        da = np.log(jitter_under_thresh(da, thresh=thresh))
-
-    da = da - da.mean("time")
-
-    def _get_eof(d):
-        # Remove slices where everything is nan
-        d = d[~np.isnan(d).all(axis=tuple(range(1, d.ndim)))]
-        solver = Eof(d, center=False)
-        eof = solver.eofs(neofs=1).squeeze()
-        return eof * np.sign(np.nanmean(eof))
-
-    out = xr.apply_ufunc(
-        _get_eof,
-        da,
-        input_core_dims=[["time", *dims]],
-        output_core_dims=[dims],
-        dask="parallelized",
-        vectorize=True,
-        output_dtypes=[float],
-        dask_gufunc_kwargs={"allow_rechunk": True},
+    raise RuntimeError(
+        "Due to a licensing issue, eofs-based functionality has been permanently removed. "
+        "Please excuse the inconvenience. "
+        "For more information, see: https://github.com/Ouranosinc/xclim/issues/1620"
     )
-    return out.assign_attrs(units="")
-
-
-first_eof = StatisticalProperty(
-    identifier="first_eof",
-    aspect="spatial",
-    compute=_first_eof,
-    allowed_groups=["group"],
-)