From 299e89214d0ac95d134918f886a41e9002021508 Mon Sep 17 00:00:00 2001 From: Axel Huebl Date: Wed, 7 Apr 2021 14:34:44 -0700 Subject: [PATCH] Snapshot naming, Remove Specs Mapping --- FORMAT_ADIOS.md | 11 ----------- STANDARD.md | 17 +++++++++++++++-- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/FORMAT_ADIOS.md b/FORMAT_ADIOS.md index ec09fd3..dfe6330 100644 --- a/FORMAT_ADIOS.md +++ b/FORMAT_ADIOS.md @@ -33,17 +33,6 @@ Output from `bpls -A` for a boolean attribute `pybool` stored in the location of There is no convention yet for a unique representation of ADIOS2 variables with boolean type. Thus, implementations should cast the data to and from `unsigned char` instead. -## `stepBased` Encoding of Iterations - -In order to correlate openPMD iterations with ADIOS steps, the *root* group (path `/`) in ADIOS must contain a variable: - - - `__step__` - - type: 1-dimensional array containing N *(int)* elements, where N is the number of ADIOS steps - - description: for each ADIOS step, this variable needs to be updated with the corresponding openPMD iteration. - - note: ADIOS steps are absolute and not every ADIOS step or openPMD iteration contains an update for each declared openPMD record. - - advice to implementers: [decide on this] an openPMD iteration for different openPMD records might be spread over multiple ADIOS steps. - An iteration of an openPMD record must correspond to exactly one ADIOS step. - ## Datasets An openPMD **data set** is represented by an group prefix that contains an ADIOS variable `__data__`. diff --git a/STANDARD.md b/STANDARD.md index 8e6add8..0e8d023 100644 --- a/STANDARD.md +++ b/STANDARD.md @@ -216,7 +216,7 @@ Each file's *root* group (path `/`) must further define the attributes: - allowed values: - `fileBased` (multiple files) - `groupBased` (one file) - - `stepBased` (one file with internally encoding, if supported by the data format) + - `stepBased` (one file with internal encoding for iterations, if supported by the data format) - `iterationFormat` - type: *(string)* @@ -227,13 +227,26 @@ Each file's *root* group (path `/`) must further define the attributes: for `fileBased` formats the iteration must be included in the file name; the format depends on the selected `iterationEncoding` method + - note: it is not required that every openPMD iteration contains an update for each declared openPMD record (see below) - examples: - for `fileBased`: - `filename_%T.h5` (without file system directories) - for `groupBased`: (fixed value) - `/data/%T/` (must be equal to and encoded in the `basePath`) - for `stepBased`: (fixed value) - - `slowest varying index` + - data-format internal convention + - *slowest varying index* of data + +### `stepBased` Encoding of Iterations + +In order to correlate openPMD iterations with an index of data-format internal updates/steps or an index in the slowest varying dimension of an array, the *root* group (path `/`) must contain an additional variable once `stepBased` is chosen for `iterationEncoding`: + + - `snapshot` + - type: 1-dimensional array containing N *(int)* elements, where N is the number of updates/steps in the file format + - description: for each update/step in a file format, this variable needs to be updated with the corresponding openPMD iteration. + - note: in some data formats, updates/steps are absolute and not every update/step contains an update for each declared openPMD record + - advice to implementers: an openPMD iteration might be spread over multiple updates/steps, but not vice versa. + In such a scenario, an individual openPMD record's update/step must appear exactly once per iteration. Required Attributes for the `basePath`