2.4

2.4 Release Notes

• Minimum supported python version is now 3.6

• All IO or Processor intensive parts of Pavilion have been made multi-threaded or multi-process,
  vastly speeding up many Pavilion operations.

• Tests now have id's of the form '.'. The default label is 'main',
  so tests from your 'main' (PAV_CONFIG_DIR defined) pavilion config directory
  can leave off the label. Labels aren't saved, but determined when configs
  are loaded.

• Every config directory can (and should) now have a 'config.yaml' that sets the label for tests
  built from that config directory, as well as the working_directory for tests created
  from those configs. If this does not exist, a default label is applied to the tests,
  and the general pavilion.yaml defined working directory is used.

• Pavilion has removed all permissions management, except to operate with 'shared_group' as
  the default group. Given multiple working directories, it is now up to the user
  to set sticky bits on those directories if specific permissions are desired for those tests.

• Test series files now have a 'repeat' config item, to specify the number of times the tests
  in the series should be repeated.

• The code for test Series has been refactored and greatly cleaned up.

• Scheduler plugins have been completely revamped (see below).

  • Machines can be 'chunked', and those chunks can be permuted over for smaller run sizes that
    still cover the entire system.
  • Chunk node distributions can be contiguous, random, distributed, and more.
  • You can specify a percentage of the system for chunk size or node count.

• All builtin plugins are now directly integrated into the Pavilion code, to speed plugin search
  time.

• Test nodes are (generally) chosen before a test is created, so you can permute over
  sched.test_node_list and similar.

• Added pav series ls command

• Added pav config commands.

• Backwards compatibility breaks:

  • Scheduler configuration has been unified under the 'schedule' key in test configs (see below).
  • Imports for the base plugin libraries have changed (but those libraries have not).
  • Tests can no longer have skip conditions that are deferred.
  • The '--wait' argument of the run command has been removed.
  • Tests can no longer set test specific group and permissions
  • The 'restart' test series config item is no longer supported.
  • Many 'sched' variables in Pavilion test configs have changed names. See
    pav show sched --vars slurm (or 'raw').
  • The Slurm MPI scheduler plugin has been merged with Slurm. Use the slurm specific 'mpi_cmd'
    parameter to use mpirun instead of srun. Additional mpirun options are also available.
  • In test series files, the 'series' key is now 'test_sets'.
  • The pav series command is now 'pav series run`

See the RELEASE.txt for a short guide on upgrading tests to use the new scheduling configuration system.

2.3 - Test Series and Better Results

2.3 Release Notes

• The new 'pav series' command and associated files (stored in <config>/series/<name>.yaml)
  allow for the organization of groups of tests in a hierarchical manner with multiple testing steps.
  See https://pavilion2.readthedocs.io/en/v2.3/tutorials/writing_a_series.html
• Revamped the result parsing system for greater flexibility. Backwards compatibility is mostly maintained (see below).
  The updated system allows for line selection regexes for every parser, so the parser can concentrate on parsing
  the line/lines themselves. See https://pavilion2.readthedocs.io/en/v2.3/results/basics.html
• Added 'flatten_results' option to the base Pavilion config. Allows for
  producing a separate result log line for each 'per_file' result, making
  charting in Splunk significantly easier. The results can be significantly
  larger, however, especially on large machines.
• Added unified test run filtering in 'status' and 'result'.
• Added the 'list' command for searching test runs and series
• Added 'maint' command for maintanence operations.
• Results now include a 'uuid' key - A completely unique id for each test.
• Added spack options to test config. Allows for spack packages to be built
  and/or loaded inside of a pavilion test. Requires a valid spack path to
  be provided in the Pavilion config under the new key spack_path.
• Modules may now have nested paths (True lmod module organization)
• Updated permissions management.
• Added test series files for organizing multiple test runs.
• Added 'pav graph' command for creating graphs from results on the fly. Requires matplotlib.
• Added 'pav show hosts/modes/suites' commands.
• Switched to using epoch time for all timestamps.
• Test run attributes are now indexed for faster 'pav status' and 'pav result' searches.

See https://pavilion2.readthedocs.io/en/v2.3/ for documentation on many of these changes.

Backwards compatibility breaks

• 'fn' and 'n' result fields have been merged into 'per_file'.
• The 'status', and 'result' commands now search all available test runs
  and select recent (1 day) tests by the current user on the current
  machine. For the old behavior, use pav status last, etc.
• Old tests will lack some expected properties for searching,
  as that information has been consolidated into the 'attributes' file
  which they won't have.
• 'test_node_list' in the results has been replaced with the much shorter
  'test_node_list_short' key.
• The results command '-l' (--show-logs) option is now '-L'.
• The results command '--json' argument and '--oneline' argments were removed.
  Use --summary instead.
• The table result parser has entirely changed. See pav show results --doc table.
• match_type for the regex result parser is now general to all parsers, and called match_select.

Roadmap

2.4 will primarily focus on revamping and adding scheduler plugins. Expect additional options for scheduling tests, performing node sampling, consistent scaling studies, and more.

v2.2 - Expressions and more

2.2 Release notes

• All new test config parser.

• Variable references are now 'expressions'.

  • In addition to variables, they can contain math and functions.
    • 'hello {{world * 2}}'
    • Assuming 'world' has a value of 5, would be: 'hello 10'

• Iterations (formerly sub-strings) can now contain more than one variable.
  This results in all combinations of the variables being used.

  • 'Greetings: [~hello {{name}}{{num}}~,]'
  • With name: ['bob', 'janet'] and num: [1,2] would result in:
  • 'Greetings: hello bob1, hello bob2, hello janet1, hello janet2'

• per test permissions settings

  mytest:
    group: secret_group
    umask: '007'

• Backwards compatibility breaks

  • All duplicate keys in configs are now detected and cause errors.
  • Default values may no longer be specified using '{{foo?}}'
  • Dashes are no longer allowed in Pavilion variable names.
    • '{{name-len}}'
  • You can no longer have nested 'sub-strings' (now called iterations).
  • Not everything can be escaped in Pavilion strings anymore. This is now
    limited to all normal python escapes
    (https://docs.python.org/2.0/ref/strings.html),
    plus '[', '{', and '~'.
  • The results section is now 'result_parse'
  • Result parsers plugins now have explicit argument default and validation settings,
    and the _check_args method must return the modified argument dictionary.
    (rather than None before)

  mytest:
    variables:
      unit_base: 100
    result_parse:
      regex:
         speed: 
           regex: 'speed (\d+)'
    result_evaluate:
      speed_normalized: "speed / {{unit_base}}"
      another_thing: '"some don''t string"'

  • build.source_location has changed.

  mytest:
    build:
      source_path: mytest.zip
      source_url: http://github.com/hpc/mytest/latest.zip
      source_download: latest
  

• Build are now built in their final location, rather than in a tmp
  location. This fixes the issues with symlinks (as they no longer
  move, so they can be absolute), and issues with non-relocatable
  builds.

• New config keys:

  mytest:
    maintainer:
      name: Paul Ferrell
      email: [email protected]

2.1.2

2.1.2

• Fixed some regressions from the 2.1.1 release.

2.1.1

• Fixed speed of draw_table with new algorithm
• Fixed some name conflicts with the 'per_fullname' and 'per_name' result parser options.
• Fixed issue with pipe exceptions in pav results
• Fixed issue with slurm scheduler node selection.
• Fixed clean command exceptions
• Updated reserved result parser key list. Renamed 'errors' to 'pav_result_errors'.
• and many smaller bug fixes.
• Updated yaml_config version.
• Updated the ls and cat commands, merged the tree command into ls.
• Moved build code into its own module
• Added include/exclude_nodes option to slurm.
• Added a 'build' command. Like the 'run' command, but only builds.
• Added '--rebuild' option to both build and run (and the capability to do so)
• Added better build tracking output and verbosity under the 'run' command.

Release 2.1

2.1

• Quite a few bugfixes.
• Simplified time output (and dropped pytz dependency)
• Updated regex parser.
• Migrated documentation to Readthedocs and restructured text format.
  • Added API documentation
  • Added some tutorials
• Add ls and cat commands
• Moved exception tracking to a single log file.
• Added the concept of hidden tests.
• Re-organized how deferred variables work.
  • run.sh files are now regenerated when a test _run's
  • test variables are now saved in the test run directory.
• Fixed RUN_COMPLETE files. They are now made when a test is finished,
  fails, is cancelled, and in most error cases.

2.0

Added check for ending with an asterisk. (#51)