Move benchmark code from nox to bm_runner (#361)

* move benchmark code from nox to bm_runner

* fix benchmark.yml

* move bm_runner.py

* keep benchmark python version consistent

* fix for asv with latest conda

* flake 8, remove old code

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs changes

* fix sperf

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update benchmark README

* address review comments

* address review comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

.flake8

@@ -35,3 +35,4 @@ exclude =
     build
     esmf_regrid/__init__.py
     esmf_regrid/tests/results
+    benchmarks/*

.github/workflows/benchmark.yml

@@ -73,9 +73,7 @@ jobs:
       - name: Benchmark script
         run: |
           if ${{ github.event_name != 'pull_request' }}; then export COMPARE="HEAD~"; else export COMPARE="origin/${{ github.base_ref }}"; fi;
-          nox --session=tests --install-only
-          export DATA_GEN_PYTHON=$(realpath $(find .nox -path "*tests/bin/python"))
-          nox --session="benchmarks(branch)" -- "${COMPARE}"
+          python benchmarks/bm_runner.py branch ${COMPARE}
 
       - name: Archive ASV results
         uses: actions/upload-artifact@v4

CHANGELOG.md

@@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## [Unreleased]
+
+### Changed
+
+- [PR#361](https://github.com/SciTools-incubator/iris-esmf-regrid/pull/361)
+  Moved the code for running benchmarks to `bm_runner.py` in line with iris
+  benchmarks.
+  [@stephenworsley](https://github.com/stephenworsley)
+
 ## [0.9] - 2023-11-03
 
 ### Added

benchmarks/README.md

@@ -16,16 +16,27 @@ raising a ❌ failure.
 installed, as well as Nox (see
 [Benchmark environments](#benchmark-environments)).
 
-[iris-esmf-regrid's noxfile](../noxfile.py) includes a `benchmarks` session
-that provides conveniences for setting up before benchmarking, and can also
-replicate the CI run locally. See the session docstring for detail.
+The benchmark runner ([bm_runner.py](./bm_runner.py)) provides conveniences for
+common benchmark setup and run tasks, including replicating the automated 
+overnight run locally. See `python bm_runner.py --help` for detail.
+
+A significant portion of benchmark run time is environment management. Run-time
+can be reduced by placing the benchmark environment on the same file system as
+your
+[Conda package cache](https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#specify-pkg-directories),
+if it is not already. You can achieve this by either:
+
+- Temporarily reconfiguring `delegated_env_commands` and `delegated_env_parent` 
+  in [asv.conf.json](asv.conf.json) to reference a location on the same file
+  system as the Conda package cache.
+- Moving your Iris repo to the same file system as the Conda package cache.
 
 ### Environment variables
 
 * `DATA_GEN_PYTHON` - required - path to a Python executable that can be
 used to generate benchmark test objects/files; see
-[Data generation](#data-generation). The Nox session sets this automatically,
-but will defer to any value already set in the shell.
+[Data generation](#data-generation). The benchmark runner sets this 
+automatically, but will defer to any value already set in the shell.
 * `BENCHMARK_DATA` - optional - path to a directory for benchmark synthetic
 test data, which the benchmark scripts will create if it doesn't already
 exist. Defaults to `<root>/benchmarks/.data/` if not set. Note that some of
@@ -34,7 +45,7 @@ plan accordingly.
 * `ON_DEMAND_BENCHMARKS` - optional - when set (to any value): benchmarks
 decorated with `@on_demand_benchmark` are included in the ASV run. Usually
 coupled with the ASV `--bench` argument to only run the benchmark(s) of
-interest. Is set during the Nox `sperf` session.
+interest. Is set during the benchmark runner `sperf` sub-commands.
 
 ### Reducing run time
 

benchmarks/asv_delegated_conda.py

@@ -189,6 +189,11 @@ def copy_asv_files(src_parent: Path, dst_parent: Path) -> None:
             # Record new environment information in properties.
             self._update_info()
 
+    def _run_conda(self, args, env=None):
+        # TODO: remove after airspeed-velocity/asv#1397 is merged and released.
+        args = ["--yes" if arg == "--force" else arg for arg in args]
+        return super()._run_conda(args, env)
+
     def checkout_project(self, repo: Repo, commit_hash: str) -> None:
         """Check out the working tree of the project at given commit hash."""
         super().checkout_project(repo, commit_hash)