enh(#319): Cmdline shortforms

- Update all doc anchors to not use 'ln-sierra-' and '-' as separators in general; much better/easier to read if you use '/'. 'ln-sierra' is not needed either--when trying to link with SIERRA docs from a downstream project in sphinx, you can define the unique prefix you want to use anyway. - Fix annoying OSX issue with some python packages being found in the system for a DIFFERENT python version, and the rest being found appropriately. Putting everything in a venv seems overkill when you also control the environment via github actions runner, but is needed here.
jharwell · Nov 20, 2024 · a6567d3 · a6567d3
1 parent 41176a2
commit a6567d3
Show file tree

Hide file tree

Showing 89 changed files with 621 additions and 384 deletions.
diff --git a/.github/actions/sample-project-setup/action.yml b/.github/actions/sample-project-setup/action.yml
@@ -130,7 +130,7 @@ runs:
         # 2023/1/18: You still need install these even after installing stuff
         # with apt AND if the sysem python matches your current python. I don't
         # know why.
-        pip3 install empy \
+        python3 -m pip install empy \
                      rospkg \
                      wheel \
                      pyparsing \

diff --git a/.github/actions/sierra-setup/action.yml b/.github/actions/sierra-setup/action.yml
@@ -68,8 +68,14 @@ runs:
       run: |
         cd docs && make man && cd ..
 
-        python3 -m pip install .
-        python3 -m pip install .[devel]
+        # 2024-11-13 [JRH]: This works around an issue with msgpec failing to
+        # compile from source on OSX because of recent default compiler changes
+        # on OSX 13. Once a new release of mspspec is released with the fix,
+        # this should go away.
+        # python3 -m pip install git+https://github.com/jcrist/msgspec.git@main
+        python3 -m pip install nox coverage
+        # python3 -m pip install .
+        # python3 -m pip install .[devel]
 
-        which sierra-cli
-        sierra-cli --version
+        # which sierra-cli
+        # sierra-cli --version
diff --git a/.github/workflows/core.yml b/.github/workflows/core.yml
@@ -44,7 +44,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/core-integration-tests.sh -f ${{ matrix.function }}
+          command: nox -s core_integration -p ${{ matrix.python-version }} -- -f ${{ matrix.function }}
 
       - uses: actions/upload-artifact@v3
         with:
@@ -94,7 +94,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/${{ matrix.platform }}-integration-tests.sh -f bc_bivar_sanity_test
+          command: nox -s plugin_${{ matrix.platform }}_integration -p ${{ matrix.python-version }} -- -f bc_bivar_sanity_test
 
 
       - uses: actions/upload-artifact@v3
@@ -151,7 +151,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/${{ matrix.platform }}-integration-tests.sh -f stage1_bivar_test
+          command: nox -s plugin_${{ matrix.platform }}_integration -p ${{ matrix.python-version }} -- -f stage1_bivar_test
 
       - uses: actions/upload-artifact@v3
         with:
@@ -193,7 +193,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/argos-integration-tests.sh -f stage3_univar_test
+          command: nox -s plugin_argos_integration -p ${{ matrix.python-version }} -- -f stage3_univar_test
 
       - uses: actions/upload-artifact@v3
         with:
@@ -235,7 +235,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/argos-integration-tests.sh -f stage3_bivar_test
+          command: nox -s plugin_argos_integration -p ${{ matrix.python-version }} -- -f stage3_bivar_test
 
       - uses: actions/upload-artifact@v3
         with:
@@ -277,7 +277,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/argos-integration-tests.sh -f stage4_univar_test
+          command: nox -s plugin_argos_integration -p ${{ matrix.python-version }} -- -f stage4_univar_test
 
       - uses: actions/upload-artifact@v3
         with:
@@ -318,7 +318,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/argos-integration-tests.sh -f stage5_univar_test
+          command: nox -s plugin_argos_integration -p ${{ matrix.python-version }} -- -f stage5_univar_test
 
       - uses: actions/upload-artifact@v3
         with:
@@ -361,7 +361,7 @@ jobs:
           timeout_minutes: 3600
           max_attempts: 3
           shell: bash
-          command: ./scripts/argos-integration-tests.sh -f stage5_bivar_test
+          command: nox -s plugin_argos_integration -p ${{ matrix.python-version }} -- -f stage5_bivar_test
 
       - uses: actions/upload-artifact@v3
         with:

diff --git a/.github/workflows/coverage-top.yml b/.github/workflows/coverage-top.yml
@@ -87,9 +87,6 @@ jobs:
           # ls -al ci-*-Linux-${{ matrix.python-version }}-coverage/.* || true
           # ls -al ci-*-Linux-${{ matrix.python-version }}-coverage/* || true
 
-          echo HERE1
-          find . -name "ci-*-Linux-${{ matrix.python-version }}-coverage*"
-          echo HERE2
           coverage \
             combine \
             $(find . -name "ci-*-Linux-${{ matrix.python-version }}-coverage*")

diff --git a/.github/workflows/plugin.yml b/.github/workflows/plugin.yml
@@ -79,7 +79,7 @@ jobs:
         with:
           timeout_minutes: 3600
           max_attempts: 3
-          command: ./scripts/argos-integration-tests.sh -f ${{ matrix.function }} -e hpc.local
+          command: nox -s plugin_argos_integration -p ${{ matrix.python-version }} -- -f ${{ matrix.function }} -e hpc.local
 
       - uses: actions/upload-artifact@v4
         with:
@@ -117,7 +117,10 @@ jobs:
 
       - name: Integration tests
         run: |
-          ./scripts/ros1robot-integration-tests.sh -f ${{ matrix.function }} -e hpc.local
+          nox \
+          -s plugin_ros1robot_integration \
+          -p ${{ matrix.python-version }} \
+          -- -f ${{ matrix.function }} -e hpc.local
 
       - uses: actions/upload-artifact@v4
         with:
@@ -157,7 +160,10 @@ jobs:
 
       - name: Integration tests
         run: |
-          ./scripts/ros1gazebo-integration-tests.sh -f ${{ matrix.function }} -e hpc.local
+          nox \
+          -s plugin_ros1gazebo_integration \
+          -p ${{ matrix.python-version }} \
+          -- -f ${{ matrix.function }} -e hpc.local
 
       - uses: actions/upload-artifact@v4
         with:
@@ -211,7 +217,10 @@ jobs:
       - name: Integration Tests
         if: matrix.platform != ''
         run: |
-          ./scripts/${{ matrix.platform }}-integration-tests.sh -f stage2_univar_test -e ${{ matrix.exec-env }}
+          nox \
+          -s plugin_${{ matrix.platform }}_integration \
+          -p ${{ matrix.python-version }} \
+           -- -f stage2_univar_test -e ${{ matrix.exec-env }}
 
       - uses: actions/upload-artifact@v4
         with:

diff --git a/docs/index.rst b/docs/index.rst
@@ -3,6 +3,8 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
+.. _main:
+
 ===========================================================================
 SIERRA (reSearch pIpEline for Reproducibility, Reusability, and Automation)
 ===========================================================================

diff --git a/docs/man/sierra-cli.rst b/docs/man/sierra-cli.rst
@@ -1,4 +1,4 @@
-.. _ln-sierra-usage-cli:
+.. _usage/cli:
 
 ######################
 Command Line Interface
@@ -12,7 +12,7 @@ If an option is given more than once, the last such occurrence is used.
 
 See also :manpage:`sierra-examples`.
 
-:seealso:`ln-sierra-examples`
+:seealso:`sierra-examples`
 
 
 ===========

diff --git a/docs/src/api.rst b/docs/src/api.rst
@@ -1,3 +1,5 @@
+.. _api:
+
 =============
 API Reference
 =============

diff --git a/docs/src/contributing.rst b/docs/src/contributing.rst
@@ -1,4 +1,4 @@
-.. _ln-sierra-contributing:
+.. _contributing:
 
 ============
 Contributing
@@ -11,8 +11,8 @@ All types of contributions are welcome: bugfixes, adding unit/integration tests,
 etc. If you're only have a little bit of time and/or are new to SIERRA, looking
 at the issues is a good place to look for places to contribute. If you have more
 time and/or want to give back to the community in a bigger way, see
-:ref:`ln-sierra-roadmap` for some big picture ideas about where things might be
-going, and help shape the future!
+:ref:`roadmap` for some big picture ideas about where things might be going, and
+help shape the future!
 
 Mechanics
 =========
@@ -70,7 +70,7 @@ repository.
     - ``models/`` - Model interfaces.
 
     - ``pipeline/`` - Core pipline code in 5 stages (see
-      :ref:`ln-sierra-usage-pipeline`).
+      :ref:`usage/pipeline`).
 
     - ``ros1/`` - Common :term:`ROS1` bindings.
 

diff --git a/docs/src/description.rst b/docs/src/description.rst
@@ -9,7 +9,6 @@ SIERRA is a command line tool and plugin framework for:
 - Improving the reproducibility of scientific research, particular in AI.
 
 It supports a wide range of platforms, execution environments, and experiment
-input/output formats--see :ref:`ln-sierra-support-matrix` for details.  A short
-overview of the research automation SIERRA provides is here:
-:ref:`ln-sierra-usage-pipeline`. To get started, see
-:ref:`ln-sierra-getting-started`.
+input/output formats--see :ref:`support-matrix` for details.  A short overview
+of the research automation SIERRA provides is here: :ref:`usage/pipeline`. To
+get started, see :ref:`getting-started`.
diff --git a/docs/src/faq.rst b/docs/src/faq.rst
@@ -1,4 +1,4 @@
-.. _ln-sierra-faq:
+.. _faq:
 
 ===
 FAQ
@@ -16,14 +16,14 @@ FAQ
    A: No! Experiment execution happens in stage 2, and graph generation happens
    in stage4. If you just want to add/remove lines from a graph, change the
    title, formatting, etc., you can just tell SIERRA to re-run stage 4 via
-   ``--pipeline 4``. See :ref:`ln-sierra-usage-pipeline` for more details about
+   ``--pipeline 4``. See :ref:`usage/pipeline` for more details about
    pipeline stages.
 
 #. Q: How to I resume an experiment which got killed part of the way through by
    an HPC scheduler for exceeding its job time limit ?
 
    A: Run SIERRA just as you did before, but add ``--exec-resume``, which will
-   tell SIERRA to pick up where it left off. See :ref:`ln-sierra-usage-cli` for
+   tell SIERRA to pick up where it left off. See :ref:`usage/cli` for
    the full cmdline reference.
 
 #. Q: How do I run a non-default set of pipeline stages, such as {3,4}?
@@ -63,7 +63,7 @@ FAQ
 #. Q: SIERRA fails to run on my HPC environment?
 
    A: The most likely reason is that you don't have the necessary environment
-   variables set up--see :ref:`ln-sierra-exec-env-hpc` for details on what is
+   variables set up--see :ref:`plugins/exec-env/hpc` for details on what is
    required.
 
 #. Q: SIERRA doesn't generate any graphs during stage4/the graph I'm interested
@@ -95,7 +95,7 @@ FAQ
    stages. The files generated in stages {3,4,5} are generated `from` the
    earlier results, so it is OK to overwrite those. However, if you are sure you
    want to overwrite stuff, you can pass ``--exp-overwrite`` to disable this
-   behavior during stages {1,2}. See also :ref:`ln-sierra-philosophy`.
+   behavior during stages {1,2}. See also :ref:`philosophy`.
 
 #. Q: I need to apply very precise/nuanced configuration to experiments that is
    too specific to be easily captured in a :term:`Batch Criteria` or other
@@ -107,7 +107,7 @@ FAQ
    tags, modifying attributes. This is a good way to apply tricky configuration
    which doesn't really fit anywhere else, or to try out some "quick and dirty"
    changes to see if they do what you want before codifying them with a python
-   class (see :ref:`ln-sierra-tutorials-project-main-config` for details on how
+   class (see :ref:`tutorials/project/main-config` for details on how
    to do that).
 
 #. Q: SIERRA can't find a module it should be able to find via