refactor(#304): XML experiment definition

docs/index.rst

@@ -39,6 +39,7 @@ SIERRA: Automation for the Scientific Method and Agent-Based Research
    src/philosophy.rst
    src/faq.rst
    src/contributing.rst
+   src/roadmap.rst
    src/glossary.rst
    src/api.rst
 

docs/src/roadmap.rst

@@ -1,13 +1,11 @@
 .. _ln-sierra-roadmap:
 
-==============
-SIERRA Roadmap
-==============
+===================
+Development Roadmap
+===================
 
 This page shows a brief overview of some of the ways I think SIERRA could be
-improved. Big picture stuff, not "add more unit tests".  Basically a place to
-put stuff I want to add someday so I don't forget.
-
+improved. Big picture stuff, not "add more unit tests".
 
 Supporting ROS2
 ===============

docs/src/tutorials/plugin/platform_plugin.rst

@@ -38,8 +38,7 @@ additional files can be included.
 
       import implements
 
-      from sierra.core import xml
-      from sierra.core.experiment import bindings
+      from sierra.core.experiment import bindings, xml, definition
 
       @implements.implements(bindings.IParsedCmdlineConfigurer)
       class CmdlineParserGenerator():
@@ -91,8 +90,8 @@ additional files can be included.
         :class:`~sierra.core.experiment.bindings.IExpRunConfigurer`.
         """
 
-      def population_size_from_pickle(def: tp.Union[xml.XMLAttrChangeSet,
-                                                    xml.XMLTagAddList]) -> int:
+      def population_size_from_pickle(def: tp.Union[xml.AttrChangeSet,
+                                                    xml.TagAddList]) -> int:
         """
         During stage 5, there is no way for SIERRA to know how many robots
         were used in a cross-platform way, because different platforms can
@@ -107,7 +106,7 @@ additional files can be included.
             if path == ".//system" and attr == "size":
                 return int(value)
 
-      def population_size_from_def(exp_def: xml.XMLLuigi) -> int:
+      def population_size_from_def(exp_def: definition.XMLExpDef) -> int:
         """
         During stage 2, on some platforms (e.g., ROS) you need to be able to
         extract the # of robots that will be used for a given
@@ -221,6 +220,8 @@ additional files can be included.
 
    .. code-block:: python
 
+      from sierra.core.experiment import definition
+
       class PlatformExpDefGenerator():
           """
           Create an experiment definition from the
@@ -244,7 +245,7 @@ additional files can be included.
                        **kwargs) -> None:
               pass
 
-          def generate(self) -> XMLLuigi:
+          def generate(self) -> definition.XMLExpDef:
               pass
 
        class PlatformExpRunDefUniqueGenerator:

docs/src/tutorials/project/new_bc.rst

@@ -33,7 +33,7 @@ following to get it to work with SIERRA as a :term:`Batch Criteria`:
    it within), produce a list of sets, where each set is all changes that need
    to be made to the ``.xml`` template file in order to set the value of your
    variable to something. Each change is a
-   :class:`~sierra.core.xml.XMLAttrChange` object, that takes the
+   :class:`~sierra.core.experiment.xml.AttrChange` object, that takes the
    following arguments in its constructor:
 
    #. XPath search path for the **parent** of the attribute that you want to
@@ -48,8 +48,8 @@ following to get it to work with SIERRA as a :term:`Batch Criteria`:
    tags that need to be removed from the ``.xml`` template file in order to
    set the value of your variable to something.
 
-   Each change is a :class:`~sierra.core.xml.XMLTagRm` object that takes
-   the following arguments in its constructor:
+   Each change is a :class:`~sierra.core.experiment.xml.TagRm` object that
+   takes the following arguments in its constructor:
 
    #. XPath search path for the **parent** of the tag that you want to
       remove.
@@ -60,8 +60,8 @@ following to get it to work with SIERRA as a :term:`Batch Criteria`:
    passed during initialization, generate a list of sets, where each set is all
    tags that need to be added to the ``.xml`` template file.
 
-   Each change is a :class:`~sierra.core.xml.XMLTagAdd` object that takes
-   the following arguments in its constructor:
+   Each change is a :class:`~sierra.core.experiment.xml.TagAdd` object that
+   takes the following arguments in its constructor:
 
    #. XPath search path for the **parent** of the tag that you want to
       add.

docs/src/usage/environment.rst

@@ -0,0 +1,104 @@
+.. _ln-usage-env-vars:
+
+Environment Variables
+=====================
+
+.. envvar:: SIERRA_PLUGIN_PATH
+
+   Used for locating :term:`plugins <Plugin>`. The directory `containing` a
+   plugin directory outside the SIERRA source tree must be on
+   ``SIERRA_PLUGIN_PATH``. Paths are added to ``PYTHONPATH`` as needed to load
+   plugins correctly. For example, if you have a different version of the
+   ``--storage-medium`` plugin you'd like to use, and you have but the directory
+   containing the plugin in ``$HOME/plugins``, then you need to add
+   ``$HOME/plugins`` to your ``SIERRA_PLUGIN_PATH`` to so that SIERRA will find
+   it. This variable is used in stages 1-5.
+
+   Used for locating :term:`projects <Project>`; all projects specifiable with
+   ``--project`` are directories found within the directories on this path. For
+   example, if you have a project ``$HOME/git/projects/myproject``, then
+   ``$HOME/git`` must be on ``SIERRA_PLUGIN_PATH`` in order for you to be able
+   to specify ``--project=myproject``. This variable is used in stages 1-5.
+
+   You *cannot* just put the parent directory of your project on
+   :envvar:`PYTHONPATH` because SIERRA uses this path for other things
+   internally (e.g., computing the paths to YAML config files).
+
+.. envvar:: PYTHONPATH
+
+   Used for locating projects per the usual python mechanisms.
+
+.. envvar:: ARGOS_PLUGIN_PATH
+
+   Must be set to contain the library directory where you installed/built ARGoS,
+   as well as the library directory for your project ``.so``. Checked to be
+   non-empty before running stage 2 for all ``--exec-env`` plugins. SIERRA does
+   `not` modify this variable, so it needs to be setup properly prior to
+   invoking SIERRA (i.e., the directory containing the :term:`Project` ``.so``
+   file needs to be on it). SIERRA can't know, in general, where the location of
+   the C++ code corresponding to the loaded :term:`Project` is.
+
+.. envvar:: SIERRA_ARCH
+
+   Used to determine the names of ARGoS executables via ``argos3-$SIERRA_ARCH``,
+   so that in HPC environments with multiple queues/sub-clusters with different
+   architectures ARGoS can be compiled natively for each for maximum
+   performance.  Can be any string. Used when generating ARGoS cmds in stage 1,
+   and only if SIERRA is run on a cluster.
+
+.. envvar:: SIERRA_NODEFILE
+
+   Points to a file suitable for passing to :program:`parallel` via
+   ``--sshloginfile``. See :program:`parallel` docs for content/formatting
+   requirements.
+
+   Used by SIERRA to configure experiments during stage 1,2; if it is not
+   defined and ``--nodefile`` is not passed SIERRA will throw an error.
+
+.. envvar:: PARALLEL
+
+   Any and all environment variables needed by your :term:`Project` must be
+   exported via the ``PARALLEL`` environment variable before invoking SIERRA,
+   because GNU parallel does not export the environment of the node it is
+   launched from to slave nodes (or even on the local machine). Something like::
+
+     export PARALLEL="--workdir . \
+     --env PATH \
+     --env LD_LIBRARY_PATH \
+     --env LOADEDMODULES \
+     --env _LMFILES_ \
+     --env MODULE_VERSION \
+     --env MODULEPATH \
+     --env MODULEVERSION_STACK
+     --env MODULESHOME \
+     --env OMP_DYNAMICS \
+     --env OMP_MAX_ACTIVE_LEVELS \
+     --env OMP_NESTED \
+     --env OMP_NUM_THREADS \
+     --env OMP_SCHEDULE \
+     --env OMP_STACKSIZE \
+     --env OMP_THREAD_LIMIT \
+     --env OMP_WAIT_POLICY \
+     --env SIERRA_ARCH \
+     --env SIERRA_PLUGIN_PATH"
+
+   Should be a good starting point. Only used if SIERRA is run on a cluster with
+   ``exec_env=hpc.slurm|hpc.pbs``. Don't forget to include
+   :envvar:`ARGOS_PLUGIN_PATH`, :envvar:`ROS_PACKAGE_PATH`, etc., depending on
+   your chosen :term:`Platform`.
+
+.. envvar:: PARALLEL_SHELL
+
+   SIERRA sets up the :term:`Experiment` execution environments by running one
+   or more shell commands in a subprocess (treated as a ``shell``, which means
+   that :program:`parallel` can't determine ``SHELL``, and therefore defaults to
+   ``/bin/sh``, which is not what users expect. SIERRA explicitly sets
+   ``PARALLEL_SHELL`` to the result of ``shutil.which('bash')`` in keeping with
+   the Principle Of Least Surprise.
+
+.. envvar:: ROS_PACKAGE_PATH
+
+   The list of directories which defines where ROS will search for
+   packages. SIERRA does `not` modify this variable, so it needs to be setup
+   properly prior to invoking SIERRA (i.e., sourcing the proper ``setup.bash``
+   script).

scripts/ros1gazebo-integration-tests.sh

@@ -69,4 +69,8 @@ set -x
 
 rospack find sierra_rosbridge
 
+cd $SAMPLE_ROOT
+ls -alh
+python3 -m projects.ros1gazebo_project.generators.scenario_generators
+
 batch_criteria_test