Skip to content

Latest commit

 

History

History
183 lines (158 loc) · 9.8 KB

todo.org

File metadata and controls

183 lines (158 loc) · 9.8 KB

SIERRA

[#B] Add additional integration tests [60%]

  • [X] Verifying ARGoS stage 1,2,3,4 outputs
  • [X] Testing ARGoS video generation
  • [X] Verifying ROS1+Gazebo stage 1 outputs
  • [ ] ROS1+Gazebo sample should generate SOME outputs so I can test SIERRA stage 3,4 with that platform. Not critical, as those stages are currently getting tested with ARGoS.
  • [X] Test bivariate batch criteria
  • [X] Test on OSX (reall just getting the python environment setup).
  • [ ] Testing specific command line options and other elements of the SIERRA core not tested by the plugin integration tests. This should get its own workflow file. What shows up as github badges should be a core+plugins yml file with an “Integration tests” label.
  • [ ] Simple model for testing in the sample project, associated tests
  • [ ] Switch from bash to something else (python/noxfile?); my scripts are getting too complicated.
  • [X] Imagizing integration tests

Add additional unit tests [0%]

  • [ ] Parser unit tests (for CLI variables that can be used as batch criteria)
  • [ ] Nodefile parsing

Add quick quick start: a linux VM with SIERRA all set up and ready to go

Added tutorial for creating a new model

  • And add an example in the sample project

Add –seed to gazebo commands so experiments are more idempotent

[#A] Add plugin interface for experimental inputs

  • This is needed as part of making SIERRA directly able to support platforms which can’t take XML as input.

[#B] Switch reading intra experiment model dataframe index sizes

From using the size of a different dataframe in the exp to reading from the time setup written to the pkl file.

[#B] Add some example pbs/slurm scripts in the HPC docs

[#B] For all bivar perf measures, add scatter plot generation

Might be helpful to spot trends among different controllers. Would also need to be implemented in stage 5 as another comparison option.

[#B] Remove **kwargs usage from graphs–explicit arguments leads to way

better type checking

[#B] Rename perf measure config from ‘perf’ to ‘stage1/stage3/stage4/etc’.

Much clearer this way. Also fits better with the ‘stage5.yaml’ config file idea.

[#C] Check out the fastcore library

[#C] Try Modin as a parallel pandas wrapper

[#C] Fix all links in document to not be prefixed with ‘ln-sierra’

[#B] Document why you need 2 GNU parallel processes for rosrobot

Because you can’t have robot controllers running on the master, which could happen if you only had a single GNU parallel process.

Link to capture context: file:/opt/jharwell/git/sierra/docs/src/requirements.rst::.. _ln-req-xml:

[#B] Switch reading intra experiment model dataframe index sizes

From using the size of a different dataframe in the exp to reading from the time setup written to the pkl file.

–exec-resume does not work with rosrobot platform

At least not reliably… I think this is bcause if you cancel a previous run, GNU parallel does not run the necessary commands to clean up the previous experiment and kill any remaining roslaunch bits

Move stuff from –camera-config that relies on my changes to ARGoS to

TITERRA.

[#B] Fix GNU parallel sync

The slave and master GNU parallel commands for ROS+robot are run in separate processes, and so when the master finishes and says “X seconds until next run”, the process with the slave GNU parallel command is still running. A fix would be to have the slave GNU parallel process be a second command in the master’s commands.txt.

This might also be solved with the master/slave/timekeeper barrier functions I have to add anyway…

Link to capture context: file:~/git/python3.9/titerra/README.md::# TITERRA

[#B] Fix SIERRA cmdline tutorial–not accurate anymore :sierra

Link to capture context: /opt/jharwell/git/sierra/docs/src/tutorials/project/cmdline.rst

[#A] Create SIERRA initial pypi package [100%]

  • [X] Push –scenario to the FORDYCA plugin–not part of core SIERRA
  • [X] Push templates/scripts to the FORDYCA plugin–not part of core SIERRA
  • [X] Make PopulationDensity not coupled to SS,DS,RN,PL block distribution, which it is currently.
  • [X] Move convergence stuff out of exp_generator–not part of core SIERRA
  • [X] Move LOG4CXX_CONFIGURATION check out of HPC–not part of core SIERRA
  • [X] Make SIERRA runnable from directories other than the repo directory
  • [X] Verify that all the batch criteria I moved to FORDYCA still work as expected
  • [X] Move all yaml config out of SIERRA core. Should retain the ability to merge configs though, as I will need that for TITARRA common stuff merging with FORDYCA/PRISM specific config.
  • [X] Create derived TemporalVariance batch criteria for FORDYCA. (BC|BM|M types are specific to my project). This must be done AFTER creating titarra–the parser is used in vcs.py
  • [X] Rename xml_luigi to something that makes more sense
  • [X] Create trivial ARGoS library to use with SIERRA as a trivial example
  • [X] Document all the SIERRA hooks that exist/I use in TITERRA
  • [X] Update contributing guide: LIBRA development workflow so it makes sense for sierra (originally written for C++), and add CONTRIBUTING.md which points to this page in the docs.
  • [X] Refine docs for all overridable SIERRA hooks to make them more usable to others.
  • [X] Docs for what statistics SIERRA can generate
  • [X] More SAANoise out of SIERRA–to uses the perf.robustness dictionary, which I am moving out to TITERRA. ALso, SAA noise requires ARGoS functionality that is not in the main repo and will not probably ever make it there.
  • [X] Remove –help; it should be a generic message that directs people to the online docs
  • [X] Document the plugin search path somewhere
  • [X] Remove the pm_query() stuff from ARGoS batch criteria–that is specific to my project. If I need it, I should derive from the necessary ARGoS batch criteria and then add that single function.
  • [X] Update all parsers to use sections to make error messages cleaner
  • [X] Add tutorials for adding new platform to execution exec_env, and vice versa. Talk about how they are coupled.
  • [X] Add docs for which cmdline arguments all platforms must implement
  • [X] Setup travis for some basic integration tests with the sample project for ARGoS and ROS+Gazebo.
  • [X] Create interface classes for all plugins–will make documenting easier, as well as help users to find bugs when adding plugins
  • [X] Update config tutorials with changes to rosrobot (template input file, etc.)
  • [X] Go through API documentation and cleanup class docstrings so what is rendered so it looks nice. First pass done for class docstrings in core. Plugins still needs work.
  • [X] Go through API documentation and cleanup function docstrings so what is rendered so it looks nice.

[#B] Document that you probably want to set PARALLEL_SHELL with rosrobot

sh does not understand setup.bash files.

Link to capture context: file:/opt/jharwell/git/sierra/docs/src/requirements.rst::.. _ln-req-xml:

[#B] Fix the pylint import error false positives

[#B] Document that you need to INSTALL your plugin variables

Or otherwise but the directory you put on SIERRA_PLUGIN_PATH on PYTHONPATH Link to capture context: file:~/git/python3.9/titerra/README.md::# TITERRA

[#B] Fix the pylint import error false positives

[#B] Switch all collated .csvs to have .mean extension to make things consistent

Stat csv extension mapping should probably be put in the config module

Switch to pathlib

  • Much better than using raw strings everywhere for filesystem paths

[#B] Add –version flag

This really should be there.... And not require a valid –project to be passed.

Switch everything in examples.rst to use the sample project to increase

cohesion

Make bivariate inter-experiment heatmaps renderable into videos

Switch from my logging hack to haggis module

Does what I do, but much better

Add sys.excepthook hook asking the user to file a bug.

Switch from cmdopts dictionary to a data class with fields

I will get WAY better help from the static analyzer

Add short options with some category prefixes

  • To emulate GCC syntax, which is really nice. Some ideas:

-E -> –exec- -e -> –exp- -s -> –skip- -L -> –plot- -c -> –controller-

After implementing this, I’m not sure at this point that there is that much value in doing this, because it adds more complexity than it removes for most users. But its there, and can be removed easily enough…

Change n_robots -> n_agents everywhere

Better tasking framework/organization

  • Switch to calling nox instead of calling bash directly from the yml workflow

files. This will allow me to easily parameterize multiple versions of python, AND reduce the # of jobs I need/total workflow runtime on github by a good bit.

[#B] Make all fs paths shown be relative to batch root

  • Screen will be much less cluttered; could make this toggleable if desired.