Consistent format usage in json schemas

[ghost]

User Story:

As an OSCAL application engineer, I would like consistency in the format parameter in the JSON schemas.

Goals:

In some cases, for instance ("date-authorized" in Leveraged Authentications)
Schema definitions use a regex instead of the format parameter for "date" or "date-time"
While in other definitions format: "date-time" is used.
For consistency adding a format parameter in addition to the regex gives additional flexibility for application & API development.

Dependencies:

Other instances of definitions that have an unused matching format parameter: "byte" for base64
OpenApi does suggest using a "uuid" format for uuid's although it's not part of the draft, swagger will have an easier time understanding that uuid's are uuids.

Acceptance Criteria

• new schemas with more format annotations where it fits.

[david-waltermire]

It looks like "uuid" was added in JSON Schema Draft 2019-09, which is one newer than DRAFT 07. Since only DRAFT 07 is widely adopted, using the "uuid" format is problematic if it will cause errors in tools.

I poked around and couldn't find the specification reference for "byte". It doesn't look like it is supported in DRAFT 07 either.

Given this information, we may want to delay addressing this issue until we are ready to move beyond DRAFT 07.

[aj-stein-nist]

Given this is an issue pertaining to how Metaschema serializes its types into JSON Schemas during schema generation and not specific to OSCAL: @david-waltermire-nist, are we ok to transfer this issue to the usnistgov/metaschema repo?

[david-waltermire]

I looked into this quite a bit more. Metaschema provides 4 date-related data types:

1. date
2. date-with-timezone
3. date-time
4. date-time-with-timezone

The *-with-timezone data types required a timezone, while the others allow an optional timezone. The required timezone types were created to support non-ambiguous dates and date/times. The optional timezone types allow this ambiguity.

In JSON schema the date and date-time formats are based on RFC 3339, section 5.6.

The date format does not permit a timezone.
The date-time format requires a timezone.

As a result, the following is true about the Metaschema date data types.

1. date (implemented by the DateDatatype) is derived from the RFC 3339 format, but allows an optional timezone, which doesn't align with the JSON schema date format. This is why the "format": "date" is omitted.
2. date-with-timezone (implemented by the DateWithTimezoneDatatype) is derived from the RFC 3339 format, but requires a timezone, which doesn't align with the JSON schema date format. This is why the "format": "date" is omitted.
3. date-time (implemented by the DateTimeDatatype) is derived from the RFC 3339 format, but allows an optional timezone, which doesn't align with the JSON schema date-time format. This is why the "format": "date-time" is omitted.
4. date-time-timezone (implemented by the DateTimeDatatype) is derived from the RFC 3339 format, but requires a timezone, which does align with the JSON schema date-time format. This is why the "format": "date-time" is included.

In OSCAL date-authorized uses the date data type. This is to handle cases were a timezone might not be known.

For other types the following was considered.

• day-time-duration (implemented by the DayTimeDurationDatatype) and year-month-duration (implemented by the YearMonthDurationDatatype) are assigned "format": "duration", but this was added in JSON schema 2019-09. Since Metaschema currently supports draft-07, this is out-of-alignment.
• uuid (implemented by the UUIDDatatype) does not include the "format": "uuid", since this was added in JSON schema 2019-09. This is in alignment.

I don't see byte listed as a valid format as of JSON schema 2020-12. Not sure about that one.

Given all of this, it looks like OSCAL is fine. We may want to consider either:

1. Advancing Metaschema's use of JSON schema to 2019-09. This would make the duration format use correct and allow use of the uuid format.
2. Stay at JSON schema draft-07 and remove the duration format from use.

It looks like JSON schema to 2019-09 has greater adoption now. Option 1 might be a good move.

Thoughts?

[aj-stein-nist]

usnistgov/OSCAL#837 (comment) was really my summary of my mistaken impression of this issue based on the last update, @david-waltermire-nist. From what I am understanding here, the OSCAL JSON Schemas are generated by metaschema-xslt, based upon alignment with the metaschema specification. Should I not transfer this to metaschema or metaschema-xslt depending on whether the spec will dictate this or the XSLT and Java implementations need to align on this as an implementation detail?

[aj-stein-nist]

I am in support of "Option 3" (Option 1 + Option 2) and evaluate supporting Draft 07 and Draft 2019-09 JSON Schema variants in tandem and aligning with duration support being in the latter. Dave will evaluate with WIP test code and we assess at a later date.

[wendellpiez]

Solid work on the analysis @david-waltermire-nist