Skip to content

Commit

Permalink
Merge pull request #68 from AutoResearch/docs/workflow-docs
Browse files Browse the repository at this point in the history
docs: add introduction to workflow
  • Loading branch information
hollandjg authored Jan 24, 2024
2 parents 054fd69 + 4a85f11 commit 45e4b42
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 0 deletions.
36 changes: 36 additions & 0 deletions docs/cli/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Command Line Interface

Different parts of AutoRA experiments can require very different computational resources. For instance:

- The theorist and experimentalist might require training or use of neural networks, and benefit from high
performance computing (HPC) resources for short bursts – minutes or hours.
- The experiment runner might post an experiment using a service like "Prolific" and poll every few minutes for
hours, days or week until the experimental data are gathered.

Running the experiment runner with the same resources as the theorist and experimentalist in this case would be
wasteful, and may be prohibitively expensive.

To solve this problem, AutoRA comes with a command line interface (CLI). This can be used with HPC schedulers like
[SLURM](https://slurm.schedmd.com/) to run different steps in the cycle with different resources.

You can use the CLI if the following conditions are true:

1. Every part of `s` can be successfully [pickled](https://docs.python.org/3/library/pickle.html).
2. You can write each step of your experiment as a single importable function which operates on a state and returns
a state:
```python
from example.lib import initial_state, experimentalist, experiment_runner, theorist
s = initial_state()
for i in range(3):
s = experimentalist(s)
s = experiment_runner(s)
s = theorist(s)
```

Contents of this section:

- ["Basic Usage"](./basic-usage): Basic usage of the CLI
- ["Usage with Cylc workflow manager"](./cylc-pip): Example using the Cylc workflow manager (which
handles cyclical processes)
- ["Usage with Cylc workflow manager and Slurm"](./cylc-slurm-pip): Example using the Cylc
workflow manager and the SLURM scheduler with different resources for each step in the cycle.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ nav:
- Cycle:
- Functional: 'cycle/Linear and Cyclical Workflows using Functions and States.ipynb'
- "Command Line Interface":
- Home: "cli/index.md"
- "Basic Usage": "cli/basic-usage/README.md"
- "Using Cylc": "cli/cylc-pip/README.md"
- "Using Cylc and Slurm": "cli/cylc-slurm-pip/README.md"
Expand Down

0 comments on commit 45e4b42

Please sign in to comment.