The OCP Test and Validation Specification establishes a standardized format for the structuring and transmission of test information emitted from a diagnostic, enabling the analysis and final determination of verdict on the overall health of the hardware or software under test.
This specification aims to be extensible, while still providing sufficient consistency across multiple diagnostic tests to integrate with test executive platforms and result storage systems. The level of data is intended to enable statistical process control and predictive modeling for machine learning applications. This allows for systems that support the root cause analysis of failures, and repair prediction for manufacturing hardware defects and fleet repair systems.
The high level goals of the specification are to document the schema and the format for the transfer of information from a diagnostic to a test executive, a persistent data store, or an analysis/statistical process control system.
The schema covers the organization and contents of the data being emitted from the diagnostic, whereas the format defines the way that the data is represented.
The two separate elements compose the overall representation of the data and seek to address the following high level objectives:
This specification has been developed by evaluating multiple proprietary test and diagnostic formats that have been leveraged across the industry at various points in time and at different organizations. By expressing a standard format for information generated during a test execution, it allows a single diagnostic test to be integrated into different data collection systems for later analysis. This specification aims to enable the critical diagnostic and test information to be exported to different data and analysis tools and systems for offline analysis, alerting, and debugging.
All consumers of test data are inherently aware of the importance of having well structured data, but in many cases, to get to this result, a "wrapping" function is required that is unique to each different diagnostic or test, as well as each environment where that test will be leveraged. This results in a combinatorial explosion of fragile, low-value integration effort. For example, consider storage diagnostics that are essentially wrappers around common testing utilities such as fio or networking diagnostics that are wrappers around netperf or iperf utilities. Utilities that grep for logging artifacts or other unstructured output are fragile and lead to test escapes.
The authors of this specification are equally aware that each different organization interested in test and validation data will most likely have their own representation of test data that may differ from the specification presented here for many potential reasons:
- The organization has a significant investment in an existing test data schema.
- The organization's data systems are tightly coupled with other downstream systems such as manufacturing execution systems, workflow scheduling tools, conformance databases, etc.
The objective for the specification here is that it serves as a feature complete, structured representation of test diagnostic information that can be easily transformed into a proprietary format with the expectation that this work is non-recurring engineering. An organization should be able to develop a common library of data transformation logic to go from this schema to their own schema which can be leveraged for each test or diagnostic that is developed in the OCP specification. In addition, although the destination format may not have all the features and properties of the data schema presented here, an objective is that the specification presented for this format is at parity or a higher level of fidelity than the destination system so that no data is lost in execution.
As the hardware under test continues to increase in complexity, the development time and non-recurring engineering investment in hardware health and validation diagnostics and tests continues to scale roughly linearly with the increases in hardware and software complexity. As a result, it is critical that the engineering effort spent on diagnostic and test development is able to be leveraged in many different environments, for many different functions including, but not limited to:
- Design Validation
- HALT/HASS Testing
- Margin Testing
- Compliance and Standards Testing
- EMI Compliance
- Environmental Testing
- Manufacturing Testing
- L5/L6 level manufacturing testing "Board Level Testing"
- L10 level "Machine Level Testing"
- L11 level "Rack Level Testing"
- Data Center Testing
- Incoming Quality Assurance Testing
- Repair and Maintenance Testing
- Repair and reverse logistic operations
- Hardware RMA
- Failure Analysis
All of these functions are typically performed in different environments with different capabilities and IT systems. As a result, the OCP Test and Validation specification has embraced JSON as a common interoperability format for the invocation and export of information for diagnostics executed on hardware or software under test. This design decision does come with trade-offs, which will be discussed in a later section of this document.
In addition, diagnostic tests are developed using a multitude of different languages and tools, and the necessary rich libraries to support the JSON dialect exist for common implementation languages for diagnostics such as C, C++, Python, Go, Rust, Java, and Scala. The authors of this specification feel that JSON is still currently the most portable format for data transmission across languages that may be used for diagnostic development and the development of systems that process and store such information.
The authors acknowledge that specialized needs regarding output will always exist, and that the specification should be extensible in a way that presents new types of data and information to be represented in a well structured manner, without impacting existing parsers and consumers of the information.
As a result, extensibility has been included as a goal and feature of the test data schema presented in this document. As with all design decisions, this feature comes with a trade-off where proprietary or unsupported extensions across different diagnostic tests can fragment the specification or lead to interoperability challenges. Support for versioning and change management are critical for the health of an ecosystem of diagnostics built around supporting this specification.
The duration of diagnostic execution can vary across different applications such as manufacturing testing vs. design validation. It's not unusual for some diagnostics to complete their execution in milliseconds whereas others may complete in multiple days, or even weeks, as the case of ongoing reliability testing (ORT).
When executing long running tests, it is essential that the diagnostic output format supports the concept of streaming results, so that additional information can be emitted and parsed by the consumers of the diagnostic information as it is generated and transmitted. This is required for several reasons:
- If the diagnostic execution is terminated or fails to complete, the loss of large amounts of testing information may incur a financial or schedule burden on a project or program.
- A test execution environment may wish to stop a running diagnostic in the event of a particular failure condition or threshold that has been exceeded.
A single large JSON-encoded result from diagnostics does not support this requirement, and in order for a JSON document to be parseable, it must be complete. To support the execution of long running tests, the specification outlined here supports emitting diagnostic information as individual self-contained artifacts.
The technology which lends itself best to this requirement is Line Delimited JSON (definition) which makes use of the fact that the JSON format does not allow return or newline characters within primitive values (in strings, these shall be escaped as \r and \n respectively) and most JSON formatters default to not including any whitespace, including returns and newlines. These features allow the newline character or return and newline character sequence to be used as a delimiter. The use of the newline as a delimiter also enables this format to work very well with traditional line-oriented Unix tools such as grep, jq, less, etc.
Some of the artifacts are stateful, having a beginning and an end message. For example, to build a MeasurementSeries (a time-series list of measurements), the diagnostic output model includes the measurementSeriesStart and measurementSeriesEnd messages which are accompanied by additional MeasurementSeriesElements to populate the information in the measurementSeries container object. The ending message is specifically designed not to allow for information loss if not present. Its omission, however, can be used to determine that there was a transmission error during the output.
For the purpose of this document's readability: while the LDJSON output is optimal for automated parsing, the lack of white-space does make it more difficult for human reviewers. The examples included have been converted to a 'pretty printed' format for easier consumption.
For example, a specification compliant JSON message to transmit the version of the specification format would be emitted as shown, with no newline or carriage return characters.
{"schemaVersion":{"major":2,"minor":0},"timestamp":"2022-07-25T01:33:46.953314Z","sequenceNumber":0}This will be rendered in the specification using the common 'pretty print' format:
{
"schemaVersion": {
"major": 2,
"minor": 0
},
"timestamp": "2022-07-25T01:33:46.953314Z",
"sequenceNumber": 0
}| Term | Definition |
| OCP Test and Validation Output Specification | The output [_schema_](#outputartifacts) as described in this document. This term may be used in various formulations throughout this document (eg. OCP spec, OCP schema, specification). |
| LDJSON | Line delimited JSON, a restricted JSON format where every newline is escaped, allowing for the newline character to delimit individual JSON documents (explanation) |
| Artifact | A LDJSON document which represents a self-contained message relaying information from a diagnostic test. |
| Attribute | An element or field of a message. |
| Diagnosis | An evaluation of the overall conformance of a unit under test accompanied by the supporting evidence to support the diagnosis (i.e. pass/fail). |
| Diagnostic | A software package used to determine conformance to a set of criteria for a hardware or software artifact under test. |
| DUT | Device Under Test. The device under test may be referred to as "system under test", "component under test" or other equivalent formulations throughout this document. |
| Nonconformant | A failure to conform or meet an expected criteria for a test or measurement. Opposite of conformant. |
| Symptom | An observed property or parametric measurement which is regarded as indicating an undesirable condition of non-conformance or impaired functionality. |
| Test executive | Sequencing software used to run diagnostic packages. It may have additional responsibilities such as allocating resources for testing, aggregating/storing results, etc. |
| Test Run | A test run is a high level container for all the execution artifacts that are emitted by a diagnostic test. |
| Test Step | A test step is a high level container for all the execution artifacts that are emitted by a particular section of a test run. A test run consists of 0-n test steps. |
| Unique Identifier | A string (as defined by JSON data types) that uniquely identifies the message it is in. An entire diagnostic output is considered to be the scope of the uniqueness property. |
| Verdict | A short pareto-able string indicating the determination of hardware health for a particular test step. A single diagnosis can contain one verdict. |
The following data types are used in this specification and are constrained by the JSON specification itself.
| Data Type | Description |
| Array | A list of elements consisting of the same data type. |
| Boolean | A single bit data type consisting of true or false. |
| Null | A nullable value |
| Number | An integer or floating point value expressed in base 10. |
| Object | A set of key/value pairs. Each key in the key/value pair of an object shall be a unique string. |
| String | Sequence of characters contained in double quotes. Strings support escape sequences which are outlined in the following section. |
This specification attempts to allow the use of common JSON parsing libraries that are available across multiple languages. This design decision comes with certain concessions that will be discussed below, such as loss of precision for numeric types. We will also address how these limitations of JSON data types can be mitigated. Alternative extensions to JSON such as OData were considered, but these come at the cost of non-standard parsing libraries.
The following escape sequences are supported in the diagnostic output, which are the standard JSON escape sequences.
| Character | Escaped Representation |
| Backspace | \b |
| Form-Feed | \f |
| New-Line | \n |
| Carriage Return | \r |
| Tab | \t |
| Double Quote | \" |
| Backslash | \\ |
JSON does not define a native date/time type.
All date-time strings are expressed in an ISO-8601 datetime format with optional timezone offset information. The format of a date-time string is expressed as YYYY-MM-DDTHH:MM:SS with the precision defined by the operating environment.
The time reference from diagnostics can come from multiple sources of varying accuracy. In a manufacturing scenario, timestamps may originate from a device under test whose clock is not properly set, and may not be set to a timezone correctly. In a data center production environment, timing sources may be very accurate NIST traceable timings.
As a result, no guidance is provided from this specification regarding the source or accuracy of timestamp information. The only guidance provided is that the implementation shall conform to this format, and should be referenced to UTC time when possible.
Example without time zone information
2022-01-05T06:37:41.211845017
Time zone support is considered optional, but when used the specification provides guidance that UTC time should be used but does not specify it as a requirement. This will be done with the format YYYY-MM-DDTHH:MM:SSZ.
Example with universal time zone
2022-01-05T06:37:41.211845017Z
When appending timezone information to the specification, it should follow the standard YYYY-MM-DDThh:mm:ss[.mmm] TZD (eg 2012-03-29T10:05:45-06:00) with the TZD offset specifying the difference between the local timezone from UTC (Coordinated Universal Time) with format ±HH:MM.
Example with time zone information specified
2022-01-05T06:37:41.211845017-06:00
The JSON specification does not include precision requirements for numeric values. This should not be interpreted as JSON having infinite precision, but that precision is coordinated between the producing and consuming endpoints, which are free to choose the values appropriate for their requirements.
The level of numeric precision required for diagnostic testing may exceed the limitations of common JSON parsers, which typically convert the number value type of JSON message to an appropriate native value type of the implementation language. When serializing to JSON, consider any such limitations in applications that may consume your JSON. In particular, it is common for JSON numbers to be deserialized into IEEE 754 double precision numbers and thus subject to that representation's range and precision limitations.
It is however recommended that the numeric values in the produced LDJSON are convertible to double precision floats.
The following enumerations are used in this specification. In each case, their value is represented in JSON as an upper case string matching the enumeration value.
Specifies the type or outcome value of a Diagnosis. A diagnosis type has the following allowable values.
| Value | Description |
| PASS | The diagnosis relates to hardware or software that was found to be conformant. |
| FAIL | The diagnosis relates to hardware or software that was found to be non-conformant. |
| UNKNOWN | The diagnostic evaluated the component under test and could not determine whether it is functioning within specification. |
Specifies the seriousness of a particular diagnostic log message. These values generally correspond with common log levels found in frequently used logging frameworks.
| Value | Description |
| INFO | Indicates normal or expected program execution. |
| DEBUG | Verbose information that is useful when a problem with a diagnostic is being debugged by a developer or expert user. |
| WARNING | Indicates that an unexpected event has occurred or a more serious problem may occur in the future. |
| ERROR | A condition or event has occurred that has prevented the diagnostic from executing normally. |
| FATAL | A serious error has occurred, and the diagnostic shall terminate or otherwise be unable to complete execution. |
Specifies the type of software that is represented by a SoftwareInfo message.
| Value | Description |
| UNSPECIFIED | Software element is not categorized. |
| FIRMWARE | Non-volatile software used to control component behavior on hardware e.g. SSD, NIC, BIC, BMC. This is typically programmed to a non-volatile storage device such as a SPI flash, EEPROM, etc. |
| SYSTEM | Operating system, drivers, kernel, or any other software that interfaces with hardware to enable utilities and applications to execute. |
| APPLICATION | Any other software type targeted to end-users. e.g. web browser, IDE, document creation or system software used to configure or optimize the interface to hardware through the operating system (e.g. vulnerability detection tool, storage analyzer tool) |
The SubcomponentType enumeration provides additional context on the fields in a Subcomponent message and how they should be interpreted. The subcomponent is an optional message which can be added to a Diagnosis, Measurement, or MeasurementSeriesStart message to help provide additional isolation to a particular element of the referenced hardware info object.
| Value | Description |
| UNSPECIFIED | Diagnostic has not provided information on the subcomponent type. |
| ASIC | The subcomponent refers to a particular device or ASIC on the referenced hardware device. |
| ASIC-SUBSYSTEM | The subcomponent refers to a particular subsystem or functional block of an ASIC on the referenced hardware device. |
| BUS | The subcomponent refers to a collection of pins/signals that define a particular bus on the referenced hardware device. |
| FUNCTION | The subcomponent refers to a logical (non-physical) function on the referenced hardware device. |
| CONNECTOR | The subcomponent refers to a particular port or off-device connector on the referenced hardware device. |
The final determination of the execution outcome of a diagnostic test. This is a final determination of the overall status of the device under test being assessed.
| Value | Description |
| NOT_APPLICABLE | The test status of a diagnostic was not complete so no final test result verdict can be successfully rendered. |
| PASS | Test has detected no non-conformances during execution. |
| FAIL | Test has detected non-conformances related to the hardware or software under test during execution. |
The final execution status of a diagnostic test. This is a final determination of the overall execution status of the diagnostic. The TestResult specifies the outcome of the test, and the TestStatus provides information concerning its execution completion state.
| Value | Description |
| COMPLETE | The diagnostic completed execution normally. |
| ERROR | The diagnostic did not complete execution normally due to an exception. |
| SKIP | The diagnostic was skipped or did not run to normal completion as part of its execution. |
The following combinations of TestResult and TestStatus are considered valid for a diagnostic compliant with this specification.
| Test Status | Test Result | Example Use-Case |
| SKIP | NOT_APPLICABLE |
The diagnostic did not find applicable hardware to test or exercise, or an external request to abort the diagnostic was received. The SKIP status exists so that a final passing TestResult where no testing took place is not ambiguous with a TestResult where hardware was tested and found to be conformant. |
| ERROR | NOT_APPLICABLE | The diagnostic encountered an error that prevented normal execution and determination of a pass/fail status. |
| COMPLETE | PASS | The diagnostic was completed successfully, and no non-conformances were detected on the tested hardware resulting in a passing diagnosis. |
| COMPLETE | FAIL | The diagnostic was completed successfully, and one or more non-conformances were detected on the tested hardware resulting in a failing diagnosis. |
Any other combination of TestResult and TestStatus shall be considered invalid with this specification.
The following validator comparison types provide arithmetic and string comparison operations that can be used to assess a measurement or measurement series element against a set of expected values or valid thresholds. The validators are defined in such a way that the measurement value is considered to be on the left side of the comparison operator, and the comparison value is on the right of the operator.
For example a LESS_THAN validator will validate that a measurement is "less than" the comparison value.
| Value | Description |
| EQUAL | Validates that a measurement and a comparison value are equal. |
| NOT_EQUAL | Validates that a measurement and a comparison value are not equal. |
| LESS_THAN | Validates that a measurement is less than a comparison value. |
| LESS_THAN_OR_EQUAL | Validates that a measurement is less than or equal to a comparison value. |
| GREATER_THAN | Validates that a measurement is greater than a comparison value. |
| GREATER_THAN_OR_EQUAL | Validates that measurement is greater than or equal to a comparison value. |
| REGEX_MATCH | Validates that a measurement is found to match one or more regex strings contained by the validator. |
| REGEX_NO_MATCH | Validates that a measurement does not match any regex strings contained by the validator. |
| IN_SET | Validates that a measurement exists within a set of allowable values contained by the validator. |
| NOT_IN_SET | Validates that a measurement does not exist within a set of allowables values contained by the validator. |
Due to the streaming nature of the specification, a common implementation pattern is the adoption of an identifier to indicate a relationship between an emitted artifact and a feature of the device under test. Examples include identifiers for hardware components and software elements of the DUT.
In both cases, these hardware/software related identifiers are emitted at the beginning of the test output so that they can be referenced from subsequent artifacts.
In addition, each test step, measurement and time series measurement element also have an id attribute to allow other artifacts to reference them by association, such as an error being associated to a particular software element, or a diagnosis being associated to one or more hardware components.
The specification provides guidance that implementations should use unique increasing zero based integers for identifiers, but only specifies that they shall be unique within the context of the overall Test Run execution artifact. Although GUID's are an acceptable implementation, there is no expectation of any global uniqueness across multiple test runs.
Due to the verbosity of streaming LDJSON, any optional fields in message types specified below, where the field is specifically called out as optional, may omit the field from a generated artifact message. An implementation where an empty field is included and defined as null shall also be considered a compliant implementation. The choice between these two approaches for handling optional fields is considered an implementation detail, and any parsers that consume the diagnostic output format shall be compatible with both implementations to be considered compliant with the specification.
The following examples below are both considered valid representations of an optional field for a message defined in this specification.
{
"requiredField": true,
"optionalField": null
}{
"requiredField": true
}All output artifacts are contained by an output artifact object, which is a top level container for all output messages.
The OutputArtifact is the single parseable output object from a diagnostic and represents the atomic unit of transmission from a diagnostic output. The OutputArtifact can contain one of the following descendant artifacts:
| Object | Description |
| SchemaVersionArtifact | An informational element conveying details about the data to be exported, primarily the version of the emitted data to follow. |
| TestRunArtifact | An informational element conveying details about a top level test run. A test run consists of zero or more test steps. |
| TestStepArtifact | An informational element conveying details about a test step which was executed as part of a test run. |
All top level OutputArtifacts contain the following attributes in addition to one artifact message which may be a SchemaVersionArtifact, TestRunArtifact, or TestStepArtifact message.
| Attribute | Type | Required | Description |
| sequenceNumber | number (integer) | Yes | A monotonically increasing field indicating the order of output returned from a diagnostic. A sequence number is valuable for determining if information regarding a diagnostic's execution has been lost in transmission or transformation and for re-sequencing possibly out of order transmissions. |
| timestamp | string | Yes | A timestamp indicating when the information was generated by the diagnostic returned in an ISO date-time format (see Date/Time strings for details). |
| <artifact> | JSON: object Message: various |
Yes | A SchemaVersionArtifact, TestRunArtifact, or TestStepArtifact message. |
The schema version is used to indicate the version of the OCP Test and Validation schema to the consuming client. This message shall be the first message emitted by the diagnostic and will be used to control the parsing of all following output artifact messages.
The format and content of the schemaVersion message is intended to be static and will be consistent across all future versions of the OCP Test and Validation specification.
The schema version is defined by two fields, a major and minor version.
A major change to the schema represents a breaking change that affects the parsing of the ensuing information. Examples would include renaming/deletion of fields, removal of enumeration values, or a change to cardinality of parent/child relationships of messages. A change to the major version automatically resets the minor version back to a value of zero.
A minor change to the schema represents a non-breaking change such as the addition of a field or a clarification of documentation/intent which would not compromise the accuracy of any existing diagnostics or tests that adhere to the schema.
| Attribute | JSON Type | Required | Description |
| major | number (integer) | Yes | Major version of the OCP Test and Validation output specification. |
| minor | number (integer) | Yes | Minor version of the OCP test and Validation output specification. |
{
"schemaVersion": {
"major": 2,
"minor": 0
},
"timestamp": "2022-07-25T01:33:46.953314Z",
"sequenceNumber": 0
}A testRunArtifact contains a single field which is a message of the valid types for a test run.
| Attribute | Type | Required | Description |
| <artifact> | JSON: object Message: various |
Yes | The artifact output message. |
The following message types are valid messages that may be populated in the artifact field of a testRunArtifact message.
| Object | Description |
| Log | An entry of free-form text documenting the output of a diagnostic execution typically meant for the purpose of debugging or tracing program execution. |
| Error | An exception or unexpected event has occurred in the diagnostic's execution |
| TestRunStart | Contains information about the invocation or start of a particular test run. |
| TestRunEnd | Contains information about the completion and final status of a particular test run. |
A testStepArtifact refers to data and information associated with a particular sub-sequence of a diagnostic's overall execution.
| Attribute | Type | Required | Description |
| testStepId | string | Yes | A unique identifier for the test step. This identifier is used to associate any additional messages with the test step. |
| <artifact> | JSON: object Message: various |
Yes | The artifact test step message. |
The following artifacts are supported attributes of a test step artifact:
| Object | Description |
| TestStepStart | Information pertaining to the starting condition of a test step. |
| TestStepEnd | Information pertaining to the ending condition of a test step. |
| Measurement | Information concerning a discrete measurement or telemetry from a device under test. |
| MeasurementSeriesStart | Information concerning the beginning of a measurement series. |
| MeasurementSeriesEnd | Information concerning the ending of a measurement series. |
| MeasurementSeriesElement | Information concerning a discrete observation contained in a measurement or measurement series. |
| Diagnosis | The identification of the status of a device or system under test based on the examination of measurements and symptoms. |
| Error | Information concerning an invalid state or condition that was encountered during the execution of a diagnostic. |
| File | An arbitrary collection of related information that was gathered in a file produced by the execution of a diagnostic. The content type is specified by the meta data included in this artifact type. Storage backend of the file may vary. |
| Log | An entry of free-form text documenting the output of a diagnostic execution typically meant for the purpose of debugging or tracing program execution. |
| Extension | An extensible data type that allows for new objects to be added to the diagnostic specification that will be compatible with existing parsers. |
The following section of the specification presents an alphabetical comprehensive list of all output artifacts and examples of each element.
A diagnosis gives the verdict of the health status for hardware components under test.
| Attribute | Type | Required | Description |
| verdict | string | Yes | A short string describing the overall determination of the diagnostic (e.g. cpu-good, hdd-overtemp, etc). These short human readable strings are typically used to aggregate counts of observed symptoms. |
| type | JSON: string Message: enum DiagnosisType |
Yes | The overall assessment of the result of the diagnostic execution, expressed as a DiagnosisType enum value. |
| message | string | No | Detailed explanation why the Diagnosis gives the above verdict of the hardware health. |
| hardwareInfoId | string | No | Hardware identifier to which the diagnosis applies. The referenced info ID information can be found in the DutInfo of TestRunStart. |
| subcomponent | JSON: object Message: Subcomponent |
No | Provides additional information about a specific functional subcomponent of the hardware referenced by ID. If present, the diagnosis applies to this subcomponent instead of the parent hardware component. |
| sourceLocation | JSON: object Message: SourceLocation |
No | A SourceLocation information for debugging or tracing program execution. |
{
"testStepArtifact": {
"diagnosis": {
"verdict": "mlc-intranode-bandwidth-pass",
"type": "PASS",
"message": "intranode bandwidth within threshold.",
"hardwareInfoId": "1",
"subcomponent": {
"type": "BUS",
"name": "QPI1",
"location": "CPU-3-2-3",
"version": "1",
"revision": "0"
},
"sourceLocation": {
"file": "examples/simple/simple_main.cc",
"line": 90
}
},
"testStepId": "1"
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T02:15:58.296032Z"
}The DUT info contains information about the device under test as determined by the diagnostic. It encompasses information about the device itself, its hardware and software configurations.
| Attribute | Type | Required | Description |
| dutInfoId | string | Yes | A unique identifier for the device under test. |
| name | string | No | The name of the device under test against which the diagnostic is currently executing. |
| metadata | JSON: object Message: Metadata |
No | An arbitrary non-nested JSON object containing metadata about the DUT. |
| platformInfos | JSON: array Message: list of PlatformInfo |
No | A list of platformInfo objects containing information about the platform under test. |
| hardwareInfos | JSON: array Message: list of HardwareInfo |
No | A list of hardware components discovered by the diagnostic which will be enumerated with unique IDs that can be used for later reference in the diagnostic output. This is an array of HardwareInfo messages, of which the elements are detailed below. |
| softwareInfos | JSON: array Message: list of SoftwareInfo |
No | A list of software components discovered by the diagnostic which will be enumerated with unique IDs that can be used for later reference in diagnostic output. This is an array of SoftwareInfo messages, of which the elements are detailed below. |
{
"testRunArtifact": {
"testRunStart": {
"name": "mlc_test",
"version": "1.0",
"commandLine": "mlc/mlc --use_default_thresholds=true --data_collection_mode=true",
"parameters": {
"max_bandwidth": 7200.0,
"mode": "fast_mode",
"data_collection_mode": true,
"min_bandwidth": 700.0,
"use_default_thresholds": true
},
"dutInfo": {
"dutInfoId": "1",
"name": "ocp_lab_0222",
"platformInfos": [
{
"info": "memory_optimized"
}
],
"softwareInfos": [
{
"softwareInfoId": "1",
"computerSystem": "primary_node",
"softwareType": "FIRMWARE",
"name": "bmc_firmware",
"version": "10",
"revision": "11"
}
],
"hardwareInfos": [
{
"hardwareInfoId": "1",
"computerSystem": "primary_node",
"manager": "bmc0",
"name": "primary node",
"location": "MB/DIMM_A1",
"odataId": "/redfish/v1/Systems/System.Embedded.1/Memory/DIMMSLOTA1",
"partNumber": "P03052-091",
"serialNumber": "HMA2022029281901",
"manufacturer": "hynix",
"manufacturerPartNumber": "HMA84GR7AFR4N-VK",
"partType": "DIMM",
"version": "1",
"revision": "2"
}
]
}
}
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T04:28:12.903943Z"
}An error is an output artifact that reports a software, firmware, test or any other hardware-unrelated issues. Errors can be child artifacts of either TestRunArtifact or TestStepArtifact messages.
| Attribute | Type | Required | Description |
| symptom | string | Yes | A short string of the software issued verdict. |
| message | string | No | Detailed explanation why the Error gives the above symptom. |
| softwareInfoIds | JSON: array Message: list of string |
No | A list of unique IDs of software components or firmware components to which the Error is associated. The software ID information can be found in the DutInfo field of TestRunStart. |
| sourceLocation | JSON: object Message: SourceLocation |
No | A SourceLocation information for debugging or tracing program execution. |
{
"testStepArtifact": {
"error": {
"symptom": "bad-return-code",
"message": "software exited abnormally.",
"softwareInfoIds": [
"3",
"4"
],
"sourceLocation": {
"file": "examples/simple/simple_main.cc",
"line": 90
}
},
"testStepId": "1"
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T03:06:11.254865Z"
}This specification includes support for logging with a severity level of "error" which closely follows the standard conventions of most native logging facilities for various development languages and log collection systems. In addition, it also supports the concept of an error that prevents successful completion of a diagnostic.
One common scenario where this is significant is in the area of a "cleanup" or "tear-down" section of a test where an error may occur, but the logic to make a hardware health determination has been completed successfully. In this event, an Error message should not be emitted by the diagnostic, but a Log message with the severity of Error is appropriate.
In some edge-cases, an error artifact requires being associated with a software entity that was produced as an artifact of the diagnostic's execution itself. An example might be a particular executable which was compiled during the diagnostic's runtime, but did not exist at the invocation of the diagnostic, so a SoftwareInfoId may not be known at the diagnostic's execution time. In this event, the specification guidance is that the error should be directly attributed to the diagnostic software artifact itself, which can be included as an artifact in the softwareInfo objects in the DutInfo artifact.
An extension message that allows returning structured data in a manner that should be compatible with all conformant parsers.
The extension message type allows a test step to record any data record as long as it can be expressed as a valid JSON object.
| Attribute | Type | Required | Description |
| name | string | Yes | Identifier for the extension message. |
| content | JSON: object Message: not specified |
Yes | Object message for the extension, this can be any valid nested JSON object. |
{
"testStepArtifact": {
"extension": {
"name": "extension example",
"content": {
"@type": "ExampleExtension",
"stringField": "example string",
"numberField": "42"
}
},
"testStepId": "2321"
},
"sequenceNumber": 600,
"timestamp": "2022-07-26T02:34:06.249683Z"
}Represents a file containing information that was generated or collected during the execution of a diagnostic. This specification does not dictate the manner in which the file is persisted or stored. The specification only includes a mechanism for indicating the location of a file, and the type of information that is expected to be contained by the file.
As a file can contain any type of structured or unstructured output, the 'contentType' field is provided to allow the user to specify a valid 'MIME-TYPE' which can be used to indicate the content for any decoders such as images, sound-files, or other well known media types.
The content type is an optional field that is provided to allow a consuming framework to know the format of the file that may be useful in rendering the final output to the user.
For a compliant list of defined MIME types by the W3 please refer to w3docs mime types.
The required URI parameter is used to identify how the file can be accessed remotely or locally. The URI specification is covered in the RFC 3986 Uniform Resource Identifier (URI) Generic Syntax document.
For local files, the URI should take the format of a File Uri as defined in RFC 8089, however, the following external documentation provided by the IETF (Internet Engineering Task Force) is considered more complete and concise.
When using the "file://" URI scheme, all file locations shall be interpreted as being relative to the host executing the diagnostic test. For instance, if a diagnostic was executed by a control computer off of the DUT, the File URIs would be relative to the control computer. For a diagnostic that was executed on the device under test, the File URI specified would refer to a local disk on that device.
The "file://" URI scheme is unusual in that it does not specify an Internet protocol or access method for shared files; as such, its utility in network protocols between hosts is limited. Examples of file server protocols that do define such access methods include SMB/CIFS [MS-SMB2], NFS [RFC3530] and NCP [NOVELL].
It is not excluded that the file is retrievable from any other internet enabled protocol such as HTTP [RFC2616], FTP [RFC959] or others.
A few examples of common URI's are outlined below. Please refer to the external documentation around URI syntax for more comprehensive coverage of URI formatting and syntax:
| Description | Example |
| Local file with a full path identified | file:///usr/bin/example |
| Local file with a Microsoft Windows drive letter | file:///c:/tmp/test.txt |
| File provided via SMB network protocol | smb://server/share smb://workgroup/server/share |
| File provided via SMB network protocol with username and passwords | smb://username:password@server/share smb://username:password@workgroup/server/share |
| File accessible via SCP | scp://root@host:3131/root/ids/rules.tar.gz |
| File accessible via NFS | nfs://server<:port>/path |
| Attribute | Type | Required | Description |
| displayName | string | Yes | User-friendly name of the file to be uploaded. This is intended to be displayed to the user or as output in a test executive. |
| uri | string | Yes | Specifies the specific URI that can be used to access the file remotely or locally. |
| description | string | No | A description of the file and its contents. |
| contentType | string | No | A MIME-type specifying the content contained within the file. |
| isSnapshot | boolean | Yes | Indicates if the file is a temporary snapshot of the final output, or if it represents the complete file output. |
| metadata | JSON: object Message: Metadata |
No | A JSON object containing values which contain additional information about the file object. |
{
"testStepArtifact": {
"file": {
"displayName": "mem_cfg_log",
"uri": "file:///root/mem_cfg_log",
"description": "DIMM configuration settings.",
"contentType": "text/plain",
"isSnapshot": false
},
"testStepId": "1"
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T03:13:18.755979Z"
}Provides information on a hardware component in the system that is exercised or enumerated by the diagnostic test.
| Attribute | Type | Required | Description |
| hardwareInfoId | string | Yes | Unique identifier used to reference this hardware device in the diagnostic output. |
| computerSystem | string | No |
Name of the system to which this hardware info is visible. This could be a hostname, IP address, or unique identifier for a computer system in the DUT which is a device for running arbitrary software. This attribute value is intended to align with the computerSystem as defined in the Redfish specification from the DMTF. |
| name | string | Yes | Name of the hardware device. |
| location | string | No | Location for a component. The source of the location is left up to the implementation of the diagnostic, but this is generally a location that a user would use to locate the device uniquely in a hardware system. This may be a silkscreen indicator (i.e. CPU0) or a unique location code. |
| odataId | string | No | Indicates an odata.Id identifier for a redfish managed physical component. |
| partNumber | string | No | Part number for the hardware device. This may be an internal or external part number. |
| serialNumber | string | No | Unique serial number for the hardware device. |
| manager | string | No | The name of the manager associated with the particular hardware component. This is typically an identifier for the baseband management controller or other OOB management device. |
| manufacturer | string | No | Name of the device manufacturer. |
| manufacturerPartNumber | string | No | The manufacturer part number of the device. |
| partType | string | No | Provides a category to define a particular part type (i.e. GPU, CPU, DIMM, etc). |
| version | string | No | The version of the hardware component. |
| revision | string | No | The revision of the hardware component. |
{
"testRunArtifact": {
"testRunStart": {
"name": "mlc_test",
"version": "1.0",
"dutInfo": {
"dutInfoId": "1",
"name": "ocp_lab_0222",
"hardwareInfos": [
{
"hardwareInfoId": "1",
"computerSystem": "primary_node",
"manager": "bmc0",
"name": "primary node",
"location": "MB/DIMM_A1",
"odataId": "/redfish/v1/Systems/System.Embedded.1/Memory/DIMMSLOTA1",
"partNumber": "P03052-091",
"serialNumber": "HMA2022029281901",
"manufacturer": "hynix",
"manufacturerPartNumber": "HMA84GR7AFR4N-VK",
"partType": "DIMM",
"version": "1",
"revision": "2"
}
]
}
}
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T04:28:12.903943Z"
}A log is an entry of free-form text documenting the output of a diagnostic execution typically meant for the purpose of debugging or tracing program execution. This is intended for humans and not intended to be machine interpreted.
| Attribute | Type | Required | Description |
| severity | JSON: string Message: enum Severity |
Yes | Indicates the significance of the log message. |
| message | String | Yes | Text string intended to aid in debugging or tracing of diagnostic execution. |
| sourceLocation | JSON: object Message: SourceLocation |
No | A SourceLocation information for debugging or tracing program execution. |
{
"testStepArtifact": {
"log": {
"message": "file operation completed successfully.",
"severity": "INFO",
"sourceLocation": {
"file": "examples/simple/simple_main.cc",
"line": 90
}
},
"testStepId": "1"
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T03:19:01.274584Z"
}A record of data collected during a test step execution. Measurements (and MeasurementSeries) exist to provide evidence for a particular diagnosis. This information is also commonly used for time-series analysis and statistical process control (e.g. Cpk studies).
| Attribute | Type | Required | Description |
| name | string | Yes | Label of the measurement. |
| unit | string | No | Unit of measure for the measurement (e.g. volts, seconds, transfers/sec) |
| hardwareInfoId | string | No | Unique ID of the hardware device to which the measurement information is associated. |
| subcomponent | JSON: object Message: Subcomponent |
No | Additional information about a particular functional or physical element of the referenced hardware to which this measurement applies. See the subcomponent message for additional details. |
| validators | JSON: array Message: list of Validator |
No | Array of validator objects to be applied to the message. See the Validator section for additional information. |
| value | String, number, boolean | Yes | Value of the measurement |
| metadata | JSON: object Message: Metadata |
No | A collection of metadata associated with the measurement. |
{
"testStepArtifact": {
"measurement": {
"name": "measured-fan-speed-100",
"unit": "RPM",
"hardwareInfoId": "5",
"subcomponent": {
"name": "FAN1",
"location": "F0_1",
"version": "1",
"revision": "1",
"type": "UNSPECIFIED"
},
"validators": [
{
"name": "80mm_fan_upper_limit",
"type": "LESS_THAN_OR_EQUAL",
"value": 11000.0
},
{
"name": "80mm_fan_lower_limit",
"type": "GREATER_THAN_OR_EQUAL",
"value": 8000.0
}
],
"value": 100221.0
},
"testStepId": "300"
},
"sequenceNumber": 400,
"timestamp": "2022-07-26T02:11:57.857853Z"
}A single measurement contained in a time-based MeasurementSeries message. A measurement series element can optionally include validators to specify the values which are considered to be conformant for the measurement element. See the Validator section for additional information on how they apply to measurements which are members of a measurement series.
| Attribute | Type | Required | Description |
| index | number (integer) | Yes | Specifies a zero based offset indicating the order in which the measurement is located in the series. |
| measurementSeriesId | string | Yes | Identifies the MeasurementSeries to which the MeasurementSeriesElement belongs via its unique measurement series id. |
| value | string, number, boolean | Yes | Value of the measurement in the measurement series. |
| timestamp | string | Yes | An ISO formatted date time string indicating when the measurement was recorded from the perspective of the RTC of the device under test. |
| metadata | JSON: object Message: Metadata |
No | Optional metadata concerning the measurement series element. |
{
"testStepArtifact": {
"measurementSeriesElement": {
"index": 144,
"measurementSeriesId": "12",
"value": 100219.0,
"timestamp": "2022-07-26T02:45:00.102414Z"
},
"testStepId": "45398"
},
"sequenceNumber": 700,
"timestamp": "2022-07-26T02:45:00.097896Z"
}Indicates the completion of a measurement series in a test step.
| Attribute | Type | Required | Description |
| measurementSeriesId | string | Yes | A test run scoped unique id which is used to identify data associated to the measurement series. |
| totalCount | number | Yes | The number of measurement elements in the completed measurement series. |
{
"testStepArtifact": {
"measurementSeriesEnd": {
"measurementSeriesId": "0",
"totalCount": 51
},
"testStepId": "231"
},
"sequenceNumber": 300,
"timestamp": "2022-07-26T02:16:10.393209Z"
}Indicates the start of a new measurement series in a test step. A measurement series consists of time based data that is emitted from a diagnostic with no expectation of the transmission sequence being ordered. Each element of the measurement series is numbered so that the order can be established when the test execution is complete. No additional measurement elements for the series should be transmitted after the measurementSeriesEnd message.
| Attribute | Type | Required | Description |
| measurementSeriesId | string | Yes | A test run scoped unique id which is used to identify data associated to the measurement series. |
| name | string | Yes | Name of the measurement. |
| unit | string | No | Unit of measure of the series element. |
| hardwareInfoId | string | No | Unique ID of the hardware device to which the measurement information is associated. |
| subcomponent | JSON: object Message: Subcomponent |
No | Additional information about a particular functional or physical element of the referenced hardware to which this measurement series applies. |
| validators | JSON: array Message: list of Validator |
No | Array of validators that are applied against ALL elements of a measurement series. |
| metadata | JSON: object Message: Metadata |
No | Optional metadata concerning the measurement series. |
{
"testStepArtifact": {
"measurementSeriesStart": {
"measurementSeriesId": "0",
"name": "measured-fan-speed-100",
"unit": "RPM",
"hardwareInfoId": "5",
"subcomponent": {
"name": "FAN1",
"location": "F0_1",
"version": "1",
"revision": "1",
"type": "UNSPECIFIED"
},
"validators": [
{
"name": "80mm_fan_upper_limit",
"type": "LESS_THAN_OR_EQUAL",
"value": 11000.0
},
{
"name": "80mm_fan_lower_limit",
"type": "GREATER_THAN_OR_EQUAL",
"value": 8000.0
}
]
},
"testStepId": "33"
},
"sequenceNumber": 300,
"timestamp": "2022-07-26T02:04:02.083679Z"
}Additional data concerning a measurement, measurement series, validator, file, or DUTInfo. Metadata essentially provides a well-known location for attaching additional data via a JSON object to any supported message.
| Attribute | Type | Required | Description |
| <various> | JSON: object Message: various |
No | Any valid JSON object |
{
"testStepArtifact": {
"measurement": {
"name": "measured-fan-speed-100",
"unit": "RPM",
"hardwareInfoId": "5",
"subcomponent": {
"name": "FAN1",
"location": "F0_1",
"version": "1",
"revision": "1",
"type": "UNSPECIFIED"
},
"validators": [
{
"name": "80mm_fan_upper_limit",
"type": "LESS_THAN_OR_EQUAL",
"value": 11000.0
},
{
"name": "80mm_fan_lower_limit",
"type": "GREATER_THAN_OR_EQUAL",
"value": 8000.0
}
],
"value": 100221.0,
"metadata": {
"@type": "ExampleMetadata",
"exampleField1": "example string",
"exampleField2": "42"
}
},
"testStepId": "4123"
},
"sequenceNumber": 800,
"timestamp": "2022-07-26T03:06:56.549372Z"
}Parameters represent the configuration values that were passed to the diagnostic run at the time of invocation. They are command line or file based settings that control the execution and behavior of a diagnostic test.
The optional parameter definition is left undefined in this specification with the caveat being that the message object if present shall be a valid JSON object. The usage of a non-hierarchical parameter space is encouraged as it will map well with command line parameters, but is not an explicit requirement.
{
"testRunArtifact": {
"testRunStart": {
"name": "mlc_test",
"version": "1.0",
"commandLine": "mlc/mlc --use_default_thresholds=true --data_collection_mode=true",
"parameters": {
"max_bandwidth": 7200.0,
"mode": "fast_mode",
"data_collection_mode": true,
"min_bandwidth": 700.0,
"use_default_thresholds": true
},
"dutInfo": {
"dutInfoId": "1",
"name": "ocp_lab_0222",
"platformInfos": [
{
"info": "memory_optimized"
}
],
"softwareInfos": [
{
"softwareInfoId": "1",
"computerSystem": "primary_node",
"softwareType": "FIRMWARE",
"name": "bmc_firmware",
"version": "10",
"revision": "11"
}
],
"hardwareInfos": [
{
"hardwareInfoId": "1",
"computerSystem": "primary_node",
"manager": "bmc0",
"name": "primary node",
"location": "MB/DIMM_A1",
"odataId": "/redfish/v1/Systems/System.Embedded.1/Memory/DIMMSLOTA1",
"partNumber": "P03052-091",
"serialNumber": "HMA2022029281901",
"manufacturer": "hynix",
"manufacturerPartNumber": "HMA84GR7AFR4N-VK",
"partType": "DIMM",
"version": "1",
"revision": "2"
}
]
}
}
},
"sequenceNumber": 1,
"timestamp": "2022-07-26T02:04:02.083679Z"
}Describes properties or attributes of the device under test. This can be used as an indicator of product genealogy such as product family, product type, etc.
This object is intended primarily for future expansion for a genealogy structure for the device under test.
| Attribute | Type | Required | Description |
| info | string | Yes | A string containing information about the platform of the device under test. |
{
"testRunArtifact": {
"testRunStart": {
"name": "example",
"version": "1.0",
"commandLine": "example",
"dutInfo": {
"dutInfoId": "1",
"name": "ocp_lab_0222",
"platformInfos": [
{
"info": "memory_optimized"
}
],
}
}
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T04:28:12.903943Z"
}Provides information about software that was discovered or exercised during the execution of a diagnostic test.
| Attribute | Type | Required | Description |
| softwareInfoId | string | Yes | Unique identifier used to reference this software component in the diagnostic output. |
| computerSystem | string | No | Name of the computer system on which this software is executing. |
| name | string | Yes | Name of the software component. |
| version | string | No | Version of the software component. |
| revision | string | No | Revision of the software component. |
| softwareType | JSON: string Message: enum SoftwareType |
No | Optional field provided to categorize software into specific types that may be useful for post-analysis. This is expressed as a SoftwareType enumeration. |
{
"testRunArtifact": {
"testRunStart": {
"name": "example",
"version": "1.0",
"commandLine": "example",
"dutInfo": {
"dutInfoId": "1",
"name": "ocp_lab_0222",
"softwareInfos": [
{
"softwareInfoId": "1",
"computerSystem": "primary_node",
"softwareType": "FIRMWARE",
"name": "bmc_firmware",
"version": "10",
"revision": "11"
}
],
}
}
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T04:28:12.903943Z"
}Provides information about which file/line of the source code in the diagnostic package that generates the output for the purpose of debugging or tracing program execution.
| Attribute | Type | Required | Description |
| file | string | Yes | A string indicates a part or the full path of the source code in the diagnostic package that generates the output. |
| line | number(Integer) | Yes | An integer indicates the line number in the file that generates the output. |
{
"testStepArtifact": {
"log": {
"message": "file operation completed successfully.",
"severity": "INFO",
"sourceLocation": {
"file": "examples/simple/simple_main.cc",
"line": 90
}
},
"testStepId": "1"
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T03:19:01.274584Z"
}
Provides information about a sub-system or functional block of a hardware device. This can be used to complement and provide additional coverage/isolation information for Measurement and Diagnosis messages to provide additional optional detail about the device to which the data applies.
Typically a subcomponent may refer to a device or subsection of a device under the FRU-able component for a device under test.
The subcomponent is very loosely defined and may refer to a physical device, logical function, or collection of related devices that are related to the corresponding hardware device which is referenced by the hardwareInfoId on the parent message.
| Attribute | Type | Required | Description |
| type | JSON: string Message: enum SubcomponentType |
No | An optional enumeration indicating the subcomponent type. |
| name | string | Yes | Name of the functional block, signal, bus or element of the hardwareId to which the supporting data is applicable. |
| location | string | No | A location identifier for the subcomponent. |
| version | string | No | Version of the subcomponent. |
| revision | string | No | Revision of the subcomponent. |
{
"testStepArtifact": {
"measurement": {
"name": "measured-fan-speed-100",
"unit": "RPM",
"hardwareInfoId": "5",
"subcomponent": {
"name": "FAN1",
"location": "F0_1",
"version": "1",
"revision": "1",
"type": "UNSPECIFIED"
},
"value": 100221.0
},
"testStepId": "300"
},
"sequenceNumber": 400,
"timestamp": "2022-07-26T02:16:10.393209Z"
}An output artifact indicating the overall completion of a test run.
In the event that an error or condition that occurs that would prevent this message from being emitted in its entirety, it's expected that the client consuming this diagnostic output will have a reasonable client side timeout which would cause the diagnostic execution to be determined an 'Error' if the TestRunStart and TestRunEnd events are not both received in a parseable form prior to the elapsed maximum execution time for the diagnostic.
| Attribute | Type | Required | Description |
| status | JSON: string Message: enum TestStatus |
Yes | TestStatus enumeration which specifies the execution status of the test run. |
| result | JSON: string Message: enum TestResult |
Yes | TestResult enumeration which indicates the overall result of the test run. |
{
"testRunArtifact": {
"testRunEnd": {
"status": "COMPLETE",
"result": "PASS"
}
},
"sequenceNumber": 100,
"timestamp": "2022-07-25T04:49:25.262947Z"
}The TestRunStart artifact establishes the information pertaining to the scope and execution of the overall test run which may contain zero or more test steps.
| Attribute | Type | Required | Description |
| name | string | Yes | Name of the diagnostic being executed. |
| version | string | Yes | Version of the diagnostic being executed. |
| commandLine | string | Yes |
Full command line with arguments that was used to invoke the diagnostic. Note that this may be different from the parameters since they may be determined from a second level of interpolation based on implementation. For example, a command line argument may include a file name, and the parameters message may include information captured from that particular file. |
| parameters | JSON: object Message: various |
Yes |
Object containing information about the state or configuration of the diagnostic as it executed. Typically parameters can be input from the command line, configuration files, or may be default values of the Mdiagnostic. These parameters are specific to the diagnostic test, and are included to allow capture of the configuration of the diagnostic in a data store. |
| metadata | JSON: object Message: Metadata |
No | Optional metadata concerning the test run. |
| dutInfo | JSON: object Message: DutInfo |
Yes | Collection of structured information obtained by the diagnostic concerning the device under test. |
{
"testRunArtifact": {
"testRunStart": {
"name": "mlc_test",
"version": "1.0",
"commandLine": "mlc/mlc --use_default_thresholds=true --data_collection_mode=true",
"parameters": {
"max_bandwidth": 7200.0,
"mode": "fast_mode",
"data_collection_mode": true,
"min_bandwidth": 700.0,
"use_default_thresholds": true
},
"dutInfo": {
"dutInfoId": "1",
"name": "ocp_lab_0222",
"platformInfos": [
{
"info": "memory_optimized"
}
],
"softwareInfos": [
{
"softwareInfoId": "1",
"computerSystem": "primary_node",
"softwareType": "FIRMWARE",
"name": "bmc_firmware",
"version": "10",
"revision": "11"
}
],
"hardwareInfos": [
{
"hardwareInfoId": "1",
"computerSystem": "primary_node",
"manager": "bmc0",
"name": "primary node",
"location": "MB/DIMM_A1",
"odataId": "/redfish/v1/Systems/System.Embedded.1/Memory/DIMMSLOTA1",
"partNumber": "P03052-091",
"serialNumber": "HMA2022029281901",
"manufacturer": "hynix",
"manufacturerPartNumber": "HMA84GR7AFR4N-VK",
"partType": "DIMM",
"version": "1",
"revision": "2"
}
]
}
}
},
"sequenceNumber": 1,
"timestamp": "2022-07-25T04:49:25.262947Z"
}Artifact indicating the completion of a test step.
| Attribute | Type | Required | Description |
| status | JSON: string Message: enum TestStatus |
Yes | TestStatus enumeration expressing the overall status of the test step. |
{
"testStepArtifact": {
"testStepEnd": {
"status": "COMPLETE"
},
"testStepId": "22"
},
"sequenceNumber": 200,
"timestamp": "2022-07-25T04:54:41.292679Z"
}The TestStepStart artifact contains all the information relating to the beginning of a new phase, task, step, or area of focus for a diagnostic.
| Attribute | Type | Required | Description |
| name | string | Yes | Name of the test step |
{
"testStepArtifact": {
"testStepStart": {
"name": "intranode-bandwidth-check"
},
"testStepId": "22"
},
"sequenceNumber": 200,
"timestamp": "2022-07-25T04:56:42.249367Z"
}A validator is the description of a comparison or evaluation of a measurement or measurement series element against a criteria to determine if the measurement is conformant or within expected tolerances. Validators are polymorphic and can support multiple value types, although the data types on both sides of the evaluation shall have the same type, or the behavior is not supported.
| Attribute | Type | Required | Description |
| name | string | No | The name of the comparison being performed (i.e. checksum verification) |
| type | JSON: string Message: enum ValidatorType |
Yes | A supported validator/comparison type. |
| value | JSON: string, number, boolean | Yes | The value to use on the right side of the arithmetic comparison. |
| metadata | JSON: object Message: Metadata |
No | Optional metadata about the validator. |
The following table indicates the value types that are supported for each validator.
| Value | Supported Measurement Value Types | Supported Validator Value Types | Description |
| EQUAL | string, number, boolean | string, number, boolean | Validates that a measurement and a comparison value are equal. |
| NOT_EQUAL | string, number, boolean | string, number, boolean | Validates that a measurement and a comparison value are not equal. |
| LESS_THAN | number | number | Validates that a measurement is less than a comparison value. |
| LESS_THAN_OR_EQUAL | number | number | Validates that a measurement is less than or equal to a comparison value. |
| GREATER_THAN | number | number | Validates that a measurement is greater than a comparison value. |
| GREATER_THAN_OR_EQUAL | number | number | Validates that measurement is greater than or equal to a comparison value. |
| REGEX_MATCH | string | string, array[string] | Validates that a measurement is found to match one or more regex strings contained by the validator. |
| REGEX_NO_MATCH | string | string, array[string] | Validates that a measurement does not match any regex strings contained by the validator. |
| IN_SET | string,number | array[string], array[number] | Validates that a measurement exists within a set of allowable values contained by the validator. |
| NOT_IN_SET | string, number | array[string], array[number]. | Validates that a measurement does not exist within a set of allowables values contained by the validator. |
{
"testStepArtifact": {
"measurement": {
"name": "measured-fan-speed-100",
"unit": "RPM",
"hardwareInfoId": "5",
"subcomponent": {
"name": "FAN1",
"location": "F0_1",
"version": "1",
"revision": "1",
"type": "UNSPECIFIED"
},
"validators": [
{
"name": "80mm_fan_upper_limit",
"type": "LESS_THAN_OR_EQUAL",
"value": 11000.0
},
{
"name": "80mm_fan_lower_limit",
"type": "GREATER_THAN_OR_EQUAL",
"value": 8000.0
}
],
"value": 100221.0
},
"testStepId": "300"
},
"sequenceNumber": 400,
"timestamp": "2022-07-26T02:16:10.393209Z"
}