add cli to readme

README.md

@@ -46,6 +46,7 @@ You can install ARLBench using `pip`:
 pip install arlbench
 ```
 
+[!WARNING]
 If you want to use envpool environments (not currently supported for Mac!), instead choose:
 ```bash
 pip install arlbench[envpool]
@@ -71,20 +72,38 @@ Then you can install the benchmark. For the base version, use:
 ```bash
 make install
 ```
-
+[!WARNING]
 For the envpool functionality (not available on Mac!), instead use:
 ```bash
 make install-envpool
 ```
 </details>
 
+[!CAUTION]
+Windows is currently not supported and also not tested. We recommend using the Linux subsytem if you're on a Windows machine.
+
 ## Quickstart
 
 Here are the two ways you can use ARLBench: via the command line or as an environment. To see them in action, take a look at our [examples](https://github.com/automl/arlbench/tree/main/examples).
 
 ### Use the CLI
 
-TODO: where does this even exist?
+We provide a command line script for black-box configuration in ARLBench. Simply run:
+```bash
+python run_arlbench.py
+```
+
+You can use the hydra command line syntax to override some of the configuration like this:
+```bash
+python run_arlbench.py algorithm=ppo
+```
+
+Or run multiple different versions after one another:
+```bash
+python run_arlbench.py -m autorl.seed=0,1,2,3,4
+```
+
+We recommend you create your own custom config files if using the CLI. Our [examples](https://github.com/automl/arlbench/tree/main/examples) can show you how these can look.
 
 ### Use the AutoRL environment
 

run_arlbench.py

@@ -0,0 +1,65 @@
+"""Console script for arlbench."""
+
+from __future__ import annotations
+
+import warnings
+warnings.filterwarnings("ignore")
+import csv
+import logging
+import sys
+import traceback
+
+import hydra
+import jax
+from arlbench.arlbench import run_arlbench
+from omegaconf import DictConfig
+
+
+@hydra.main(version_base=None, config_path="examples/configs", config_name="base")
+def execute(cfg: DictConfig):
+ """Helper function for nice logging and error handling."""
+ logging.basicConfig(
+ filename="job.log", format="%(asctime)s %(message)s", filemode="w"
+ )
+ logger = logging.getLogger()
+ logger.setLevel(logging.INFO)
+
+ if cfg.jax_enable_x64:
+ logger.info("Enabling x64 support for JAX.")
+ jax.config.update("jax_enable_x64", True)
+ try:
+ return run(cfg, logger)
+ except Exception:
+ traceback.print_exc(file=sys.stderr)
+ raise
+
+
+def run(cfg: DictConfig, logger: logging.Logger):
+ """Console script for arlbench."""
+
+ # check if file done exists and if so, return
+ try:
+ with open("./done.txt") as f:
+ logger.info("Job already done, returning.")
+
+ with open("./performance.csv") as pf:
+ csvreader = csv.reader(pf)
+ performance = next(csvreader)
+ performance = float(performance[0])
+ return performance
+ except FileNotFoundError:
+ pass
+
+ objectives = run_arlbench(cfg, logger=logger)
+ logger.info(f"Returned objectives: {objectives}")
+
+ with open("./performance.csv", "w+") as f:
+ f.write(str(objectives))
+ with open("./done.txt", "w+") as f:
+ f.write("yes")
+
+ return objectives
+
+
+if __name__ == "__main__":
+ sys.exit(execute()) # pragma: no cover