diff --git a/README.md b/README.md index bb7e30e..a9bdf35 100644 --- a/README.md +++ b/README.md @@ -20,9 +20,11 @@ DOI

-* Set up a testing **experiment** for your earthquake forecasts using authoritative data sources and benchmarks. +* Set up a testing **experiment** for your earthquake forecasts using authoritative data sources + and benchmarks. * **Encapsulate** the complete experiment's definition and rules in a couple of lines. -* **Reproduce**, **reuse**, and **share** forecasting experiments from you, other researchers and institutions. +* **Reproduce**, **reuse**, and **share** forecasting experiments from you, other researchers + and institutions. # Table of Contents @@ -34,12 +36,14 @@ * [Contributing](#contributing) * [License](#license) - # Installing floatCSEP -The core of `floatCSEP` is built around the `pyCSEP` package (https://github.com/sceccode/pycsep), which itself contains the core dependencies. +The core of `floatCSEP` is built around the `pyCSEP` +package (https://github.com/sceccode/pycsep), which itself contains the core dependencies. -The simplest way to install `floatCSEP`, is by creating a `conda` environment (https://conda.io - checkout Anaconda or Miniconda) and install `pyCSEP` from `conda-forge` +The simplest way to install `floatCSEP`, is by creating a `conda` +environment (https://conda.io - checkout Anaconda or Miniconda) and install `pyCSEP` +from `conda-forge` ``` conda env create -n $NAME @@ -48,21 +52,29 @@ conda install -c conda-forge pycsep ``` Clone and install the floatCSEP source code using `pip` + ``` git clone https://github.com/cseptesting/floatcsep cd floatcsep pip install . ``` -Please read the [Installation](https://floatcsep.readthedocs.io/en/latest/intro/installation.html) documentation for detailed instructions and additional installation methods. +Please read +the [Installation](https://floatcsep.readthedocs.io/en/latest/intro/installation.html) +documentation for detailed instructions and additional installation methods. # Run an Experiment -Using the command terminal, navigate to an example experiment in `floatcsep/examples/` and type +Using the command terminal, navigate to an example experiment in ``floatcsep/tutorials/`` and +type + ``` floatcsep run config.yml ``` -A runtime directory will be created in a `results` folder. The experiment results can be visualized in `results/report.md`. **Check out the experiment, models and tests definition in the examples**! + +A runtime directory will be created in a `results` folder. The experiment results can be +visualized in `results/report.md`. **Check out the experiment, models and tests definition in +the tutorials**! # Important Links @@ -71,7 +83,6 @@ A runtime directory will be created in a `results` folder. The experiment result * `pyCSEP` [Github](https://github.com/sceccode/pycsep) * `pyCSEP` [Documentation](https://docs.cseptesting.org/) - # Roadmap and Known Issues * Add report customization @@ -80,21 +91,27 @@ A runtime directory will be created in a `results` folder. The experiment result * Implement spatial database and HDF5 experiment storage feature * Set up task paralellization - # Contributing -We encourage all types of contributions, from reporting bugs, suggesting enhancements, adding new features and more. Please refer to the [Contribution Guidelines](https://github.com/cseptesting/floatcsep/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/cseptesting/floatcsep/blob/main/CODE_OF_CONDUCT.md) for more information +We encourage all types of contributions, from reporting bugs, suggesting enhancements, adding +new features and more. Please refer to +the [Contribution Guidelines](https://github.com/cseptesting/floatcsep/blob/main/CONTRIBUTING.md) +and the [Code of Conduct](https://github.com/cseptesting/floatcsep/blob/main/CODE_OF_CONDUCT.md) +for more information # License -The `floatCSEP` (as well as `pyCSEP`) software is distributed under the BSD 3-Clause open-source license. Please see the [license file](https://github.com/cseptesting/floatcsep/blob/main/LICENSE) for more information. +The `floatCSEP` (as well as `pyCSEP`) software is distributed under the BSD 3-Clause open-source +license. Please see +the [license file](https://github.com/cseptesting/floatcsep/blob/main/LICENSE) for more +information. ## Support
-| | | -|:---|:---| +| | | +|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
This research was supported by the Statewide California Earthquake Center. SCEC is funded by NSF Cooperative Agreement EAR-2225216 and USGS Cooperative Agreement G24AC00072-00.
|
The work in this repository has received funding from the European Union’s Horizon research and innovation programme under grant agreements No.s 101058518 and 821115 of the projects GeoInquire and RISE.
|
diff --git a/docs/index.rst b/docs/index.rst index 0982c6b..0410bc1 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -46,7 +46,7 @@ floatCSEP: Floating Experiments :height: 48px .. |tutorials| image:: https://img.icons8.com/nolan/64/checklist.png - :target: examples/case_a.html + :target: tutorials/case_a.html :height: 48px @@ -56,14 +56,14 @@ Quickstart +--------------------------------------------------+-------------------------------------+ | |start| **Get Started** | |tutorials| **Tutorials** | | | | -| |learn1| **Forecasting Concepts** | - :ref:`example_a` | -| | - :ref:`example_b` | -| |learn2| **Testing Theory** | - :ref:`example_c` | -| | - :ref:`example_d` | -| |experiment| **Floating Experiments** | - :ref:`example_e` | -| | - :ref:`example_f` | -| |api| **API Reference** | - :ref:`example_g` | -| | - :ref:`example_h` | +| |learn1| **Forecasting Concepts** | - :ref:`case_a` | +| | - :ref:`case_b` | +| |learn2| **Testing Theory** | - :ref:`case_c` | +| | - :ref:`case_d` | +| |experiment| **Floating Experiments** | - :ref:`case_e` | +| | - :ref:`case_f` | +| |api| **API Reference** | - :ref:`case_g` | +| | - :ref:`case_h` | +--------------------------------------------------+-------------------------------------+ What is floatCSEP @@ -84,7 +84,7 @@ Goals Running ------- -Start using **floatCSEP** by `installing `_ the latest version and running the ``examples`` with simply: +Start using **floatCSEP** by `installing `_ the latest version and running the ``tutorials`` with simply: .. code-block:: @@ -146,14 +146,14 @@ Collaborators :maxdepth: 1 :caption: Tutorial Experiments - examples/case_a.rst - examples/case_b.rst - examples/case_c.rst - examples/case_d.rst - examples/case_e.rst - examples/case_f.rst - examples/case_g.rst - examples/case_h.rst + tutorials/case_a.rst + tutorials/case_b.rst + tutorials/case_c.rst + tutorials/case_d.rst + tutorials/case_e.rst + tutorials/case_f.rst + tutorials/case_g.rst + tutorials/case_h.rst .. toctree:: :hidden: diff --git a/docs/intro/forecasting_experiments.rst b/docs/intro/forecasting_experiments.rst index ac0817a..1b9e6ac 100644 --- a/docs/intro/forecasting_experiments.rst +++ b/docs/intro/forecasting_experiments.rst @@ -2,4 +2,4 @@ Forecasting Experiments ======================= -TBI \ No newline at end of file +TBW \ No newline at end of file diff --git a/docs/intro/installation.rst b/docs/intro/installation.rst index 4cdbeca..5d53517 100644 --- a/docs/intro/installation.rst +++ b/docs/intro/installation.rst @@ -12,15 +12,12 @@ Installing the latest version Using ``conda`` ~~~~~~~~~~~~~~~ -The core of `floatCSEP` is built around the `pyCSEP` package (https://github.com/sceccode/pycsep), which itself contains the core dependencies. +To install **floatCSEP**, first a ``conda`` manager should be installed (https://conda.io). Checkout `Anaconda`, `Miniconda` or `Miniforge` (recommended). Once installed, create an environment with: -The simplest way to install `floatCSEP`, is by creating a `conda` environment (https://conda.io - checkout Anaconda or Miniconda) and install `pyCSEP` from `conda-forge` + .. code-block:: sh - .. code-block:: console - - $ conda env create -n $NAME - $ conda activate $NAME - $ conda install -c conda-forge pycsep + $ conda env create -n csep_env + $ conda activate csep_env Then, clone and install the floatCSEP source code using ``pip`` @@ -30,27 +27,50 @@ Then, clone and install the floatCSEP source code using ``pip`` cd floatcsep pip install . -Using ``apt`` and ``pip`` -~~~~~~~~~~~~~~~~~~~~~~~~~ -To install from ``pip``, we require to install the binary dependencies of ``pyCSEP`` (see `Installing pyCSEP `_} +Using ``pip`` +~~~~~~~~~~~~~ -Then, install the ``pycsep`` latest version +To install using the ``pip`` manager, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatcsep** latest version then can be installed as: .. code-block:: - git clone https://github.com/SCECcode/pycsep - cd pycsep - python -m virtualenv venv - source venv/bin/activate - pip install -e .[all] + cd .. + git clone https://github.com/cseptesting/floatcsep + cd floatcsep + pip install . + +Installing the stable version +----------------------------- + + +Using ``conda`` +~~~~~~~~~~~~~~~ + +To install **floatCSEP**, first a ``conda`` manager should be installed (https://conda.io). Checkout `Anaconda`, `Miniconda` or `Miniforge` (recommended). Once installed, create an environment with: -and the ``floatcsep`` latest version + .. code-block:: sh + + $ conda env create -n csep_env + $ conda activate csep_env + +Then, clone and install the floatCSEP source code using ``pip`` + + .. code-block:: console + + git clone https://github.com/cseptesting/floatcsep + cd floatcsep + pip install . + + +Using ``pip`` +~~~~~~~~~~~~~ + +To install using the ``pip`` manager, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatcsep** latest version can then be installed as: .. code-block:: cd .. git clone https://github.com/cseptesting/floatcsep cd floatcsep - pip install .[all] - + pip install . \ No newline at end of file diff --git a/docs/examples/case_a.rst b/docs/tutorials/case_a.rst similarity index 81% rename from docs/examples/case_a.rst rename to docs/tutorials/case_a.rst index 4fe7f9a..45dd200 100644 --- a/docs/examples/case_a.rst +++ b/docs/tutorials/case_a.rst @@ -1,4 +1,4 @@ -.. _example_a: +.. _case_a: A - Testing a Simple Model ========================== @@ -9,7 +9,7 @@ The following example shows the definition of a testing experiment of a single * .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_a`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_a`` and type: .. code-block:: console @@ -25,7 +25,7 @@ The following example shows the definition of a testing experiment of a single * Experiment Components --------------------- -The source code can be found in the ``examples/case_a`` folder or in `GitHub `_. The directory structure of the experiment is: +The source code can be found in the ``tutorials/case_a`` folder or in `GitHub `_. The directory structure of the experiment is: :: @@ -38,18 +38,18 @@ The source code can be found in the ``examples/case_a`` folder or in `GitHub `_. The input structure of the experiment is: +The source code can be found in the ``tutorials/case_b`` folder or in `GitHub `_. The input structure of the experiment is: :: @@ -53,14 +53,14 @@ Configuration In this example, the time, region and catalog specifications are written in the ``config.yml`` file. -.. literalinclude:: ../../examples/case_b/config.yml - :caption: examples/case_b/config.yml +.. literalinclude:: ../../tutorials/case_b/config.yml + :caption: tutorials/case_b/config.yml :language: yaml :lines: 3-15 whereas the models' and tests' configurations are referred to external files for better readability -.. literalinclude:: ../../examples/case_b/config.yml +.. literalinclude:: ../../tutorials/case_b/config.yml :language: yaml :lines: 17-18 @@ -69,17 +69,17 @@ Models ~~~~~~ The model configuration is now set in the ``models.yml`` file, where a list of model names specify their file paths. - .. literalinclude:: ../../examples/case_b/models.yml - :caption: examples/case_b/models.yml + .. literalinclude:: ../../tutorials/case_b/models.yml + :caption: tutorials/case_b/models.yml :language: yaml Evaluations ~~~~~~~~~~~ The evaluations are defined in the ``tests.yml`` file as a list of evaluation names, with their functions and plots (see :doc:`pycsep:concepts/evaluations`). In this example, we use the N-, M-, S- and CL-consistency tests, along with the comparison T-test. - .. literalinclude:: ../../examples/case_b/tests.yml + .. literalinclude:: ../../tutorials/case_b/tests.yml :language: yaml - :caption: examples/case_b/tests.yml + :caption: tutorials/case_b/tests.yml .. note:: Plotting keyword arguments can be set in the ``plot_kwargs`` option - see :func:`~csep.utils.plots.plot_poisson_consistency_test` and :func:`~csep.utils.plots.plot_comparison_test` -. @@ -91,7 +91,7 @@ Running the experiment ---------------------- -The experiment can be run by simply navigating to the ``examples/case_b`` folder in the terminal an type. +The experiment can be run by simply navigating to the ``tutorials/case_b`` folder in the terminal an type. .. code-block:: console diff --git a/docs/examples/case_c.rst b/docs/tutorials/case_c.rst similarity index 84% rename from docs/examples/case_c.rst rename to docs/tutorials/case_c.rst index 2236cf3..d6b74d2 100644 --- a/docs/examples/case_c.rst +++ b/docs/tutorials/case_c.rst @@ -1,4 +1,4 @@ -.. _example_c: +.. _case_c: C - Multiple Time Windows ========================= @@ -9,7 +9,7 @@ The following example shows an experiment with **multiple time windows**. .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_c`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_c`` and type: .. code-block:: console @@ -25,7 +25,7 @@ The following example shows an experiment with **multiple time windows**. Experiment Components --------------------- -The source code can be found in the ``examples/case_c`` folder or in `GitHub `_. The input structure of the experiment is: +The source code can be found in the ``tutorials/case_c`` folder or in `GitHub `_. The input structure of the experiment is: :: @@ -49,8 +49,8 @@ Time The time configuration now sets a sequence of time intervals between the start and end dates. - .. literalinclude:: ../../examples/case_c/config.yml - :caption: examples/case_c/config.yml + .. literalinclude:: ../../tutorials/case_c/config.yml + :caption: tutorials/case_c/config.yml :language: yaml :lines: 3-7 @@ -66,9 +66,9 @@ Evaluations ~~~~~~~~~~~ The experiment's evaluations are defined in ``tests.yml``, which can now include temporal evaluations (see :func:`~floatcsep.utils.helpers.sequential_likelihood`, :func:`~floatcsep.utils.helpers.sequential_information_gain`, :func:`~floatcsep.utils.helpers.plot_sequential_likelihood`). - .. literalinclude:: ../../examples/case_c/tests.yml + .. literalinclude:: ../../tutorials/case_c/tests.yml :language: yaml - :caption: examples/case_c/tests.yml + :caption: tutorials/case_c/tests.yml .. note:: diff --git a/docs/examples/case_d.rst b/docs/tutorials/case_d.rst similarity index 84% rename from docs/examples/case_d.rst rename to docs/tutorials/case_d.rst index fbeb505..3fdc51e 100644 --- a/docs/examples/case_d.rst +++ b/docs/tutorials/case_d.rst @@ -1,4 +1,4 @@ -.. _example_d: +.. _case_d: D - Catalog and Model Queries ============================= @@ -11,7 +11,7 @@ The following example shows an experiment whose forecasts are **retrieved from a .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_d`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_d`` and type: .. code-block:: console @@ -26,7 +26,7 @@ The following example shows an experiment whose forecasts are **retrieved from a Experiment Components --------------------- -The source code can be found in the ``examples/case_d`` folder or in `GitHub `_. The **initial** input structure of the experiment is: +The source code can be found in the ``tutorials/case_d`` folder or in `GitHub `_. The **initial** input structure of the experiment is: :: @@ -65,8 +65,8 @@ Catalog The ``catalog`` inset from ``config.yml`` now makes reference to a catalog query function, in this case :func:`~pycsep.query_gcmt`. - .. literalinclude:: ../../examples/case_d/config.yml - :caption: examples/case_d/config.yml + .. literalinclude:: ../../tutorials/case_d/config.yml + :caption: tutorials/case_d/config.yml :language: yaml :lines: 14-14 @@ -80,8 +80,8 @@ Models ~~~~~~ The model configuration is set in ``models.yml``. - .. literalinclude:: ../../examples/case_d/models.yml - :caption: examples/case_d/models.yml + .. literalinclude:: ../../tutorials/case_d/models.yml + :caption: tutorials/case_d/models.yml :language: yaml * The option ``zenodo_id`` makes reference to the zenodo **record id**. The model ``team`` is found in https://zenodo.org/record/6289795, whereas the model ``wheel`` in https://zenodo.org/record/6255575. @@ -104,7 +104,7 @@ Models Running the experiment ---------------------- - The experiment can be run by simply navigating to the ``examples/case_d`` folder in the terminal and typing. + The experiment can be run by simply navigating to the ``tutorials/case_d`` folder in the terminal and typing. .. code-block:: console diff --git a/docs/examples/case_e.rst b/docs/tutorials/case_e.rst similarity index 74% rename from docs/examples/case_e.rst rename to docs/tutorials/case_e.rst index 208246a..061e6e0 100644 --- a/docs/examples/case_e.rst +++ b/docs/tutorials/case_e.rst @@ -1,15 +1,15 @@ -.. _example_e: +.. _case_e: E - A Time-Independent Experiment ================================= -This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous examples. +This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous tutorials. .. currentmodule:: floatcsep .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_e`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_e`` and type: .. code-block:: console @@ -25,7 +25,7 @@ This example shows how to run a realistic testing experiment (based on https://d Experiment Components --------------------- -The source code can be found in the ``examples/case_e`` folder or in `GitHub `_. The input structure of the experiment is: +The source code can be found in the ``tutorials/case_e`` folder or in `GitHub `_. The input structure of the experiment is: :: @@ -51,8 +51,8 @@ Time The time configuration is manifested in the ``time-config`` inset. - .. literalinclude:: ../../examples/case_e/config.yml - :caption: examples/case_e/config.yml + .. literalinclude:: ../../tutorials/case_e/config.yml + :caption: tutorials/case_e/config.yml :language: yaml :lines: 3-7 @@ -61,8 +61,8 @@ Region The testing region is the official Italy CSEP Region obtained from :obj:`csep.core.regions.italy_csep_region`. - .. literalinclude:: ../../examples/case_e/config.yml - :caption: examples/case_e/config.yml + .. literalinclude:: ../../tutorials/case_e/config.yml + :caption: tutorials/case_e/config.yml :language: yaml :lines: 9-15 @@ -72,8 +72,8 @@ Catalog The catalog is obtained from an authoritative source, namely the Bollettino Sismico Italiano (http://terremoti.ingv.it/en/bsi ), using the function :func:`csep.query_bsi` - .. literalinclude:: ../../examples/case_e/config.yml - :caption: examples/case_e/config.yml + .. literalinclude:: ../../tutorials/case_e/config.yml + :caption: tutorials/case_e/config.yml :language: yaml :lines: 17-17 @@ -81,8 +81,8 @@ Models ~~~~~~ The models are set in ``models.yml``. For simplicity, there are only three of the nineteen models originally submitted to the Italy Experiment. - .. literalinclude:: ../../examples/case_e/models.yml - :caption: examples/case_e/models.yml + .. literalinclude:: ../../tutorials/case_e/models.yml + :caption: tutorials/case_e/models.yml :language: yaml The ``.xml`` format is automatically detected and parsed by ``floatcsep`` readers. @@ -100,17 +100,17 @@ Post-Process Additional options for post-processing can set using the ``postprocess`` option. Here, we customize the forecasts plotting with: - .. literalinclude:: ../../examples/case_e/config.yml + .. literalinclude:: ../../tutorials/case_e/config.yml :language: yaml :lines: 21-34 - The forecasts are plotted and stored in ``examples/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plotting options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top of those plots. + The forecasts are plotted and stored in ``tutorials/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plotting options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top of those plots. Running the experiment ---------------------- - The experiment can be run by navigating to the ``examples/case_e`` folder in the terminal and typing. + The experiment can be run by navigating to the ``tutorials/case_e`` folder in the terminal and typing. .. code-block:: console diff --git a/docs/examples/case_f.rst b/docs/tutorials/case_f.rst similarity index 82% rename from docs/examples/case_f.rst rename to docs/tutorials/case_f.rst index c31dc24..26438a3 100644 --- a/docs/examples/case_f.rst +++ b/docs/tutorials/case_f.rst @@ -1,4 +1,4 @@ -.. _example_f: +.. _case_f: F - Testing Catalog-Based Forecasts =================================== @@ -9,7 +9,7 @@ This example shows how set up an experiment with a **time-dependent** model, who .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_f`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_f`` and type: .. code-block:: console @@ -25,7 +25,7 @@ Experiment Components --------------------- -The source files can be found in the ``examples/case_e`` folder or in `GitHub `_. The experiment structure is as follows: +The source files can be found in the ``tutorials/case_e`` folder or in `GitHub `_. The experiment structure is as follows: :: @@ -43,13 +43,13 @@ The source files can be found in the ``examples/case_e`` folder or in `GitHub < * The model to be evaluated (``etas``) is a collection of daily forecasts from ``2016-11-14`` until ``2016-11-21``. .. important:: - The forecasts must be located in a folder ``forecasts`` inside the model folder. This is meant for consistency with models based on source codes (see subsequent examples). + The forecasts must be located in a folder ``forecasts`` inside the model folder. This is meant for consistency with models based on source codes (see subsequent tutorials). Model ----- -The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (`doi:10.1785/0220200231 `_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent examples. +The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (`doi:10.1785/0220200231 `_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent tutorials. @@ -62,8 +62,8 @@ Time The configuration is analogous to time-independent models with multiple time-windows (e.g., case C) with the exception that a ``horizon`` could be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``. - .. literalinclude:: ../../examples/case_f/config.yml - :caption: examples/case_f/config.yml + .. literalinclude:: ../../tutorials/case_f/config.yml + :caption: tutorials/case_f/config.yml :language: yaml :lines: 3-7 @@ -84,8 +84,8 @@ Models Some additional arguments should be passed to a **time-dependent** model, such as its class ('td' for time-dependent) and the number of simulations. - .. literalinclude:: ../../examples/case_f/models.yml - :caption: examples/case_f/config.yml + .. literalinclude:: ../../tutorials/case_f/models.yml + :caption: tutorials/case_f/config.yml :language: yaml :lines: 1-4 @@ -101,7 +101,7 @@ Tests With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used. - .. literalinclude:: ../../examples/case_f/tests.yml + .. literalinclude:: ../../tutorials/case_f/tests.yml :language: yaml .. note:: @@ -111,7 +111,7 @@ Tests Running the experiment ---------------------- - The experiment can be run by simply navigating to the ``examples/case_h`` folder in the terminal and typing. + The experiment can be run by simply navigating to the ``tutorials/case_h`` folder in the terminal and typing. .. code-block:: console diff --git a/docs/examples/case_g.rst b/docs/tutorials/case_g.rst similarity index 87% rename from docs/examples/case_g.rst rename to docs/tutorials/case_g.rst index 9529e40..1edfaa7 100644 --- a/docs/examples/case_g.rst +++ b/docs/tutorials/case_g.rst @@ -1,4 +1,4 @@ -.. _example_g: +.. _case_g: G - Testing a Time-Dependent Model ================================== @@ -7,7 +7,7 @@ Here, we set up a time-dependent model from its **source code** for an experimen .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_g`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_g`` and type: .. code-block:: console @@ -70,8 +70,8 @@ The experiment's complexity increases from time-independent to dependent mostly 1. The **input data** is, at the very least, a catalog filtered until the forecast beginning. The catalog will be automatically allocated by ``floatcsep`` prior to each model's run (e.g., a single forecast run) in the `{model}/input` folder. It is stored in the ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`). - .. literalinclude:: ../../examples/case_g/catalog.csv - :caption: examples/case_g/catalog.csv + .. literalinclude:: ../../tutorials/case_g/catalog.csv + :caption: tutorials/case_g/catalog.csv :lines: 1-2 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model are the forecast ``start_date`` and ``end_date``, which will be modified dynamically during an experiment with multiple time-windows. The experiment system will access `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as (not limited to) ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility. @@ -116,8 +116,8 @@ Time The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td`` - .. literalinclude:: ../../examples/case_g/config.yml - :caption: examples/case_g/config.yml + .. literalinclude:: ../../tutorials/case_g/config.yml + :caption: tutorials/case_g/config.yml :language: yaml :lines: 3-7 @@ -131,8 +131,8 @@ Models Additional arguments should be passed to time-independent models. - .. literalinclude:: ../../examples/case_g/models.yml - :caption: examples/case_g/models.yml + .. literalinclude:: ../../tutorials/case_g/models.yml + :caption: tutorials/case_g/models.yml :language: yaml :lines: 1-7 @@ -146,7 +146,7 @@ Models 4. The ``build`` option defines the style of container within which the model will be placed. Currently in **floatCSEP**, only the python module ``venv``, the package manager ``conda`` and the containerization manager ``Docker`` are currently supported. .. important:: - For these examples, we use ``venv`` sub-environments, but we recommend ``Docker`` to set up real experiments. + For these tutorials, we use ``venv`` sub-environments, but we recommend ``Docker`` to set up real experiments. Tests @@ -155,8 +155,8 @@ Tests With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used. - .. literalinclude:: ../../examples/case_g/tests.yml - :caption: examples/case_g/tests.yml + .. literalinclude:: ../../tutorials/case_g/tests.yml + :caption: tutorials/case_g/tests.yml :language: yaml .. note:: @@ -168,8 +168,8 @@ Custom Post-Process Additional to the default :func:`~floatcsep.postprocess.plot_handler.plot_results`, :func:`~floatcsep.postprocess.plot_handler.plot_catalogs`, :func:`~floatcsep.postprocess.plot_handler.plot_forecasts` functions, a custom plotting function(s) can be set within the ``postprocess`` configuration - .. literalinclude:: ../../examples/case_g/config.yml - :caption: examples/case_g/config.yml + .. literalinclude:: ../../tutorials/case_g/config.yml + :caption: tutorials/case_g/config.yml :language: yaml :lines: 22-23 @@ -181,20 +181,20 @@ Custom Post-Process The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument - .. literalinclude:: ../../examples/case_g/custom_plot_script.py - :caption: examples/case_g/custom_plot_script.py + .. literalinclude:: ../../tutorials/case_g/custom_plot_script.py + :caption: tutorials/case_g/custom_plot_script.py :language: python :lines: 6-13 - In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment data in runtime. + In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``tutorials/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment data in runtime. Running the experiment ---------------------- - The experiment can be run by simply navigating to the ``examples/case_g`` folder in the terminal and typing. + The experiment can be run by simply navigating to the ``tutorials/case_g`` folder in the terminal and typing. .. code-block:: console diff --git a/docs/examples/case_h.rst b/docs/tutorials/case_h.rst similarity index 77% rename from docs/examples/case_h.rst rename to docs/tutorials/case_h.rst index 6f3592e..c9e6637 100644 --- a/docs/examples/case_h.rst +++ b/docs/tutorials/case_h.rst @@ -1,4 +1,4 @@ -.. _example_h: +.. _case_h: H - A time-dependent experiment =============================== @@ -7,7 +7,7 @@ Here, we run an experiment that access, containerize and execute multiple **time .. admonition:: **TL; DR** - In a terminal, navigate to ``floatcsep/examples/case_h`` and type: + In a terminal, navigate to ``floatcsep/tutorials/case_h`` and type: .. code-block:: console @@ -59,22 +59,22 @@ Configuration Models ~~~~~~ -As in :ref:`Tutorial G`, each **Model** requires to build and execute a source code to generate forecasts. The instructions for each model are located within ``models.yml``, which we further explain here: +As in :ref:`Tutorial G`, each **Model** requires to build and execute a source code to generate forecasts. The instructions for each model are located within ``models.yml``, which we further explain here: .. note:: The ``models.yml`` will define how to interface **floatCSEP** to each Model, implying that a Model should be developed, or adapted to ensure the interface requirements specified below. 1. The repository URL of each model and their specific versions (e.g., commit hash, tag, release) are specified as: - .. literalinclude:: ../../examples/case_h/models.yml - :caption: examples/case_h/models.yml + .. literalinclude:: ../../tutorials/case_h/models.yml + :caption: tutorials/case_h/models.yml :language: yaml :lines: 1-3, 11-13, 21-23 2. A ``path`` needs to be indicated for each model, to both download the repository contents therein and from where the source code will be executed. - .. literalinclude:: ../../examples/case_h/models.yml - :caption: examples/case_h/models.yml + .. literalinclude:: ../../tutorials/case_h/models.yml + :caption: tutorials/case_h/models.yml :language: yaml :lines: 1-4 :emphasize-lines: 4 @@ -85,8 +85,8 @@ As in :ref:`Tutorial G`, each **Model** requires to build and execute 2. There is some flexibility to interface **floatCSEP** with a model. For instance, a different `filepath` can be set for the argument file: - .. literalinclude:: ../../examples/case_h/models.yml - :caption: examples/case_h/models.yml + .. literalinclude:: ../../tutorials/case_h/models.yml + :caption: tutorials/case_h/models.yml :language: yaml :lines: 5 :lineno-match: @@ -116,18 +116,18 @@ As in :ref:`Tutorial G`, each **Model** requires to build and execute 4. The ``func`` entry indicates how the models are invoked from a shell terminal. - .. literalinclude:: ../../examples/case_h/models.yml - :caption: examples/case_h/models.yml + .. literalinclude:: ../../tutorials/case_h/models.yml + :caption: tutorials/case_h/models.yml :language: yaml :lines: 1,6,11,15,21,25 .. important:: - Please refer to :ref:`Tutorial G` for example of how to set up ``func`` for the model and interface it to **floatCSEP**. + Please refer to :ref:`Tutorial G` for example of how to set up ``func`` for the model and interface it to **floatCSEP**. 5. A prefix for the resulting forecast filepaths can be specified beforehand for each model. Without specifying this, the default is ``{model_name}`` (e.g., `etas`, `Poisson Mock`, `Negbinom Mock`). - .. literalinclude:: ../../examples/case_h/models.yml - :caption: examples/case_h/models.yml + .. literalinclude:: ../../tutorials/case_h/models.yml + :caption: tutorials/case_h/models.yml :language: yaml :lines: 21, 26 @@ -141,8 +141,8 @@ As in :ref:`Tutorial G`, each **Model** requires to build and execute 6. Additional function arguments can be passed to the model with the entry ``func_kwargs``. We perhaps noted that both Poisson Mock and Negbinom Mock use the same source code. With ``func_kwargs`` a different subclass can be defined for the same source code (in this case, a Negative-Binomial number distribution instead of Poisson). - .. literalinclude:: ../../examples/case_h/models.yml - :caption: examples/case_h/models.yml + .. literalinclude:: ../../tutorials/case_h/models.yml + :caption: tutorials/case_h/models.yml :language: yaml :lines: 11,17-20,21,27-31 @@ -152,8 +152,8 @@ Time The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td`` - .. literalinclude:: ../../examples/case_h/config.yml - :caption: examples/case_h/config.yml + .. literalinclude:: ../../tutorials/case_h/config.yml + :caption: tutorials/case_h/config.yml :language: yaml :lines: 3-7 @@ -169,8 +169,8 @@ Tests With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used. - .. literalinclude:: ../../examples/case_h/tests.yml - :caption: examples/case_h/tests.yml + .. literalinclude:: ../../tutorials/case_h/tests.yml + :caption: tutorials/case_h/tests.yml :language: yaml .. note:: @@ -182,8 +182,8 @@ Custom Post-Process A custom reporting function can be set within the ``postprocess`` configuration to replace the :func:`~floatcsep.postprocess.reporting.generate_report`: - .. literalinclude:: ../../examples/case_h/config.yml - :caption: examples/case_h/config.yml + .. literalinclude:: ../../tutorials/case_h/config.yml + :caption: tutorials/case_h/config.yml :language: yaml :lines: 22-23 @@ -195,17 +195,17 @@ Custom Post-Process The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument - .. literalinclude:: ../../examples/case_h/custom_report.py + .. literalinclude:: ../../tutorials/case_h/custom_report.py :language: yaml :lines: 5-11 - In this way, the report function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_h/custom_report.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts. + In this way, the report function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``tutorials/case_h/custom_report.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts. Running the experiment ---------------------- - The experiment can be run by simply navigating to the ``examples/case_h`` folder in the terminal and typing. + The experiment can be run by simply navigating to the ``tutorials/case_h`` folder in the terminal and typing. .. code-block:: console diff --git a/pyproject.toml b/pyproject.toml index ef57c31..1c9d0f9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,6 +4,7 @@ build-backend = "setuptools.build_meta" [tool.pytest.ini_options] addopts = "--cov=floatcsep" +norecursedirs = ["tutorials", "tests/artifacts"] testpaths = [ "tests", ] diff --git a/tests/artifacts/models/template/Dockerfile b/tests/artifacts/models/template/Dockerfile index dd5f778..b20e588 100644 --- a/tests/artifacts/models/template/Dockerfile +++ b/tests/artifacts/models/template/Dockerfile @@ -40,7 +40,7 @@ WORKDIR $MODEL_PATH COPY --chown=$USER_UID:$USER_GID {relpaths_host} $MODEL_PATH ## Ideally, the folder/files to be copied should be explicited -# COPY --chown=$USER_UID:$USER_GID source examples tests docs parameters setup $MODEL_PATH +# COPY --chown=$USER_UID:$USER_GID source tutorials tests docs parameters setup $MODEL_PATH # ______________________________ # 4- Build the model source code @@ -49,7 +49,7 @@ COPY --chown=$USER_UID:$USER_GID {relpaths_host} $MODEL_PATH ## Provide a set of instructions to build the model inside the docker image, and run the unit tests RUN setup && run_tests -## See examples in +## See tutorials in #### Python (https://git.gfz-potsdam.de/csep-group/rise_italy_experiment/models/mockup_py) #### R (https://git.gfz-potsdam.de/csep-group/rise_italy_experiment/models/mockup_R) diff --git a/tests/integration/test_model_interface.py b/tests/integration/test_model_interface.py index 318a708..ae2c0d8 100644 --- a/tests/integration/test_model_interface.py +++ b/tests/integration/test_model_interface.py @@ -139,7 +139,7 @@ def setUpClass(cls) -> None: cls._dir = os.path.join(path, "../", "artifacts", "models") cls._alm_fn = os.path.join( path, - "../../examples", + "../../tutorials", "case_e", "models", "gulia-wiemer.ALM.italy.10yr.2010-01-01.xml", @@ -201,7 +201,7 @@ def test_forecast_ti_from_hdf5(self): def tearDownClass(cls) -> None: alm_db = os.path.join( cls._path, - "../../examples", + "../../tutorials", "case_e", "models", "gulia-wiemer.ALM.italy.10yr.2010-01-01.hdf5", diff --git a/tests/qa/test_data.py b/tests/qa/test_data.py index 8441d87..c2db68f 100644 --- a/tests/qa/test_data.py +++ b/tests/qa/test_data.py @@ -10,14 +10,14 @@ class DataTest(unittest.TestCase): @staticmethod def get_runpath(case): return os.path.abspath( - os.path.join(__file__, "../../..", "examples", f"case_{case}", f"config.yml") + os.path.join(__file__, "../../..", "tutorials", f"case_{case}", f"config.yml") ) @staticmethod def get_rerunpath(case): return os.path.abspath( os.path.join( - __file__, "../../..", "examples", f"case_{case}", "results", f"repr_config.yml" + __file__, "../../..", "tutorials", f"case_{case}", "results", f"repr_config.yml" ) ) diff --git a/tests/unit/test_readers.py b/tests/unit/test_readers.py index a9e4bf8..f4935a1 100644 --- a/tests/unit/test_readers.py +++ b/tests/unit/test_readers.py @@ -71,7 +71,7 @@ def test_parse_csv_qtree(self): def test_parse_xml(self): fname = os.path.join( self._path, - "../../examples", + "../../tutorials", "case_e", "models", "gulia-wiemer.ALM.italy.10yr.2010-01-01.xml", diff --git a/examples/case_a/best_model.dat b/tutorials/case_a/best_model.dat similarity index 100% rename from examples/case_a/best_model.dat rename to tutorials/case_a/best_model.dat diff --git a/examples/case_a/case_a.py b/tutorials/case_a/case_a.py similarity index 100% rename from examples/case_a/case_a.py rename to tutorials/case_a/case_a.py diff --git a/examples/case_a/catalog.csep b/tutorials/case_a/catalog.csep similarity index 100% rename from examples/case_a/catalog.csep rename to tutorials/case_a/catalog.csep diff --git a/examples/case_a/config.yml b/tutorials/case_a/config.yml similarity index 100% rename from examples/case_a/config.yml rename to tutorials/case_a/config.yml diff --git a/examples/case_a/region.txt b/tutorials/case_a/region.txt similarity index 100% rename from examples/case_a/region.txt rename to tutorials/case_a/region.txt diff --git a/examples/case_b/case_b.py b/tutorials/case_b/case_b.py similarity index 100% rename from examples/case_b/case_b.py rename to tutorials/case_b/case_b.py diff --git a/examples/case_b/catalog.json b/tutorials/case_b/catalog.json similarity index 100% rename from examples/case_b/catalog.json rename to tutorials/case_b/catalog.json diff --git a/examples/case_b/config.yml b/tutorials/case_b/config.yml similarity index 100% rename from examples/case_b/config.yml rename to tutorials/case_b/config.yml diff --git a/examples/case_b/models.yml b/tutorials/case_b/models.yml similarity index 100% rename from examples/case_b/models.yml rename to tutorials/case_b/models.yml diff --git a/examples/case_b/models/model_a.csv b/tutorials/case_b/models/model_a.csv similarity index 100% rename from examples/case_b/models/model_a.csv rename to tutorials/case_b/models/model_a.csv diff --git a/examples/case_b/models/model_b.csv b/tutorials/case_b/models/model_b.csv similarity index 100% rename from examples/case_b/models/model_b.csv rename to tutorials/case_b/models/model_b.csv diff --git a/examples/case_b/models/model_c.csv b/tutorials/case_b/models/model_c.csv similarity index 100% rename from examples/case_b/models/model_c.csv rename to tutorials/case_b/models/model_c.csv diff --git a/examples/case_b/models/model_d.csv b/tutorials/case_b/models/model_d.csv similarity index 100% rename from examples/case_b/models/model_d.csv rename to tutorials/case_b/models/model_d.csv diff --git a/examples/case_b/region.txt b/tutorials/case_b/region.txt similarity index 100% rename from examples/case_b/region.txt rename to tutorials/case_b/region.txt diff --git a/examples/case_b/tests.yml b/tutorials/case_b/tests.yml similarity index 100% rename from examples/case_b/tests.yml rename to tutorials/case_b/tests.yml diff --git a/examples/case_c/case_c.py b/tutorials/case_c/case_c.py similarity index 100% rename from examples/case_c/case_c.py rename to tutorials/case_c/case_c.py diff --git a/examples/case_c/catalog.json b/tutorials/case_c/catalog.json similarity index 100% rename from examples/case_c/catalog.json rename to tutorials/case_c/catalog.json diff --git a/examples/case_c/config.yml b/tutorials/case_c/config.yml similarity index 100% rename from examples/case_c/config.yml rename to tutorials/case_c/config.yml diff --git a/examples/case_c/models.yml b/tutorials/case_c/models.yml similarity index 100% rename from examples/case_c/models.yml rename to tutorials/case_c/models.yml diff --git a/examples/case_c/models/model_a.csv b/tutorials/case_c/models/model_a.csv similarity index 100% rename from examples/case_c/models/model_a.csv rename to tutorials/case_c/models/model_a.csv diff --git a/examples/case_c/models/model_b.csv b/tutorials/case_c/models/model_b.csv similarity index 100% rename from examples/case_c/models/model_b.csv rename to tutorials/case_c/models/model_b.csv diff --git a/examples/case_c/models/model_c.csv b/tutorials/case_c/models/model_c.csv similarity index 100% rename from examples/case_c/models/model_c.csv rename to tutorials/case_c/models/model_c.csv diff --git a/examples/case_c/models/model_d.csv b/tutorials/case_c/models/model_d.csv similarity index 100% rename from examples/case_c/models/model_d.csv rename to tutorials/case_c/models/model_d.csv diff --git a/examples/case_c/region.txt b/tutorials/case_c/region.txt similarity index 100% rename from examples/case_c/region.txt rename to tutorials/case_c/region.txt diff --git a/examples/case_c/tests.yml b/tutorials/case_c/tests.yml similarity index 100% rename from examples/case_c/tests.yml rename to tutorials/case_c/tests.yml diff --git a/examples/case_d/config.yml b/tutorials/case_d/config.yml similarity index 100% rename from examples/case_d/config.yml rename to tutorials/case_d/config.yml diff --git a/examples/case_d/models.yml b/tutorials/case_d/models.yml similarity index 100% rename from examples/case_d/models.yml rename to tutorials/case_d/models.yml diff --git a/examples/case_d/tests.yml b/tutorials/case_d/tests.yml similarity index 100% rename from examples/case_d/tests.yml rename to tutorials/case_d/tests.yml diff --git a/examples/case_e/case_e.py b/tutorials/case_e/case_e.py similarity index 100% rename from examples/case_e/case_e.py rename to tutorials/case_e/case_e.py diff --git a/examples/case_e/catalog.json b/tutorials/case_e/catalog.json similarity index 100% rename from examples/case_e/catalog.json rename to tutorials/case_e/catalog.json diff --git a/examples/case_e/config.yml b/tutorials/case_e/config.yml similarity index 100% rename from examples/case_e/config.yml rename to tutorials/case_e/config.yml diff --git a/examples/case_e/models.yml b/tutorials/case_e/models.yml similarity index 100% rename from examples/case_e/models.yml rename to tutorials/case_e/models.yml diff --git a/examples/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml b/tutorials/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml similarity index 100% rename from examples/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml rename to tutorials/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml diff --git a/examples/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml b/tutorials/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml similarity index 100% rename from examples/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml rename to tutorials/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml diff --git a/examples/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml b/tutorials/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml similarity index 100% rename from examples/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml rename to tutorials/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml diff --git a/examples/case_e/tests.yml b/tutorials/case_e/tests.yml similarity index 100% rename from examples/case_e/tests.yml rename to tutorials/case_e/tests.yml diff --git a/examples/case_f/catalog.json b/tutorials/case_f/catalog.json similarity index 100% rename from examples/case_f/catalog.json rename to tutorials/case_f/catalog.json diff --git a/examples/case_f/config.yml b/tutorials/case_f/config.yml similarity index 100% rename from examples/case_f/config.yml rename to tutorials/case_f/config.yml diff --git a/examples/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv diff --git a/examples/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv diff --git a/examples/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv diff --git a/examples/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv diff --git a/examples/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv diff --git a/examples/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv diff --git a/examples/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv similarity index 100% rename from examples/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv rename to tutorials/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv diff --git a/examples/case_f/models.yml b/tutorials/case_f/models.yml similarity index 100% rename from examples/case_f/models.yml rename to tutorials/case_f/models.yml diff --git a/examples/case_f/tests.yml b/tutorials/case_f/tests.yml similarity index 100% rename from examples/case_f/tests.yml rename to tutorials/case_f/tests.yml diff --git a/examples/case_g/catalog.csv b/tutorials/case_g/catalog.csv similarity index 100% rename from examples/case_g/catalog.csv rename to tutorials/case_g/catalog.csv diff --git a/examples/case_g/config.yml b/tutorials/case_g/config.yml similarity index 100% rename from examples/case_g/config.yml rename to tutorials/case_g/config.yml diff --git a/examples/case_g/custom_plot_script.py b/tutorials/case_g/custom_plot_script.py similarity index 100% rename from examples/case_g/custom_plot_script.py rename to tutorials/case_g/custom_plot_script.py diff --git a/examples/case_g/models.yml b/tutorials/case_g/models.yml similarity index 100% rename from examples/case_g/models.yml rename to tutorials/case_g/models.yml diff --git a/examples/case_g/pymock/Dockerfile b/tutorials/case_g/pymock/Dockerfile similarity index 100% rename from examples/case_g/pymock/Dockerfile rename to tutorials/case_g/pymock/Dockerfile diff --git a/examples/case_g/pymock/README.md b/tutorials/case_g/pymock/README.md similarity index 100% rename from examples/case_g/pymock/README.md rename to tutorials/case_g/pymock/README.md diff --git a/examples/case_g/pymock/input/args.txt b/tutorials/case_g/pymock/input/args.txt similarity index 100% rename from examples/case_g/pymock/input/args.txt rename to tutorials/case_g/pymock/input/args.txt diff --git a/examples/case_g/pymock/pymock/__init__.py b/tutorials/case_g/pymock/pymock/__init__.py similarity index 100% rename from examples/case_g/pymock/pymock/__init__.py rename to tutorials/case_g/pymock/pymock/__init__.py diff --git a/examples/case_g/pymock/pymock/libs.py b/tutorials/case_g/pymock/pymock/libs.py similarity index 100% rename from examples/case_g/pymock/pymock/libs.py rename to tutorials/case_g/pymock/pymock/libs.py diff --git a/examples/case_g/pymock/pymock/main.py b/tutorials/case_g/pymock/pymock/main.py similarity index 100% rename from examples/case_g/pymock/pymock/main.py rename to tutorials/case_g/pymock/pymock/main.py diff --git a/examples/case_g/pymock/pyproject.toml b/tutorials/case_g/pymock/pyproject.toml similarity index 100% rename from examples/case_g/pymock/pyproject.toml rename to tutorials/case_g/pymock/pyproject.toml diff --git a/examples/case_g/pymock/requirements.txt b/tutorials/case_g/pymock/requirements.txt similarity index 100% rename from examples/case_g/pymock/requirements.txt rename to tutorials/case_g/pymock/requirements.txt diff --git a/examples/case_g/pymock/run.py b/tutorials/case_g/pymock/run.py similarity index 100% rename from examples/case_g/pymock/run.py rename to tutorials/case_g/pymock/run.py diff --git a/examples/case_g/pymock/setup.cfg b/tutorials/case_g/pymock/setup.cfg similarity index 100% rename from examples/case_g/pymock/setup.cfg rename to tutorials/case_g/pymock/setup.cfg diff --git a/examples/case_g/pymock/setup.py b/tutorials/case_g/pymock/setup.py similarity index 100% rename from examples/case_g/pymock/setup.py rename to tutorials/case_g/pymock/setup.py diff --git a/examples/case_g/tests.yml b/tutorials/case_g/tests.yml similarity index 100% rename from examples/case_g/tests.yml rename to tutorials/case_g/tests.yml diff --git a/examples/case_h/catalog.csv b/tutorials/case_h/catalog.csv similarity index 100% rename from examples/case_h/catalog.csv rename to tutorials/case_h/catalog.csv diff --git a/examples/case_h/config.yml b/tutorials/case_h/config.yml similarity index 100% rename from examples/case_h/config.yml rename to tutorials/case_h/config.yml diff --git a/examples/case_h/custom_report.py b/tutorials/case_h/custom_report.py similarity index 100% rename from examples/case_h/custom_report.py rename to tutorials/case_h/custom_report.py diff --git a/examples/case_h/models.yml b/tutorials/case_h/models.yml similarity index 100% rename from examples/case_h/models.yml rename to tutorials/case_h/models.yml diff --git a/examples/case_h/tests.yml b/tutorials/case_h/tests.yml similarity index 100% rename from examples/case_h/tests.yml rename to tutorials/case_h/tests.yml