2020 JSON Schema Support

[BrandonGardner2]

Hi,

Is it possible to revisit the support of the 2020 JSON Schema spec? This issue from June 2022 is related but we are well into the stable release and adoption of the 2020 schema now. This would be very beneficial for utilizing any of the newer spec features without having to do Unsafe work arounds.

@alexvdw Hi. Just an update on this. I have done a fair review of the 2020 specification, and TypeBox won't be implementing this specification for the foreseeable future. There are a few awkward updates in the 2020 specification for tuples and arrays that make supporting 2020 difficult, and implementing this now may result in issues on earlier draft versions. I think the preference here will be to wait until there is enough install base for the 2020 spec, then consider porting TypeBox over to it at that time.

In the interim, I've added a new type called Type.Unsafe<T>() which might prove useful for constructing schemas that match the 2020 spec for OpenAPI. You can read about this new type https://github.com/sinclairzx81/typebox#Unsafe-Types. But for the time being, will be waiting on community feedback to gauge how best to proceed with 2020

Will close off this issue for now
Many Thanks
S
Originally posted by @sinclairzx81 in #169 (comment)

[sinclairzx81]

@BrandonGardner2 Hi,

Is it possible to revisit the support of the 2020 JSON Schema spec? This issue from June 2022 is related but we are well into the stable release and adoption of the 2020 schema now. This would be very beneficial for utilizing any of the newer spec features without having to do Unsafe work arounds.

TypeBox advertises itself as being Draft 7 compliant as the "minimum" baseline specification. But as it stands, TypeBox is already "mostly" 2020 compliant.

The only type that isn't is Tuple due to issues with Ajv (as the 07 and 2020 Tuple representations are incompatible and there is no way to reconcile them). Unfortunately, this aspect is outside of my control, and why the #490 issue has been hanging around for so long (but it hasn't been forgotten about). But adopting the 2020 Tuple representation means TypeBox loses compatibility with older versions of the specification (which isn't ideal)

Note that with the exception of Tuple, every other type should be compatible across 06, 07, 2019* and 2020 draft versions.

Future of Tuple

I do eventually want Tuples on the 2020 specification (because 2020 is capable of representing [T, ...U] rest type structures)

const T = Type.Tuple([
  Type.Number(),
  Type.Rest(Type.String())
])

type T = Static<typeof T> // type T = [number, ...string[]]

But to achieve this, TypeBox is going to need internal infrastructure to make the Tuple representation configurable (where if the user is on 2019 or below, they will need to configure for the older representation, and where errors will be thrown if attempting to construct Type.Rest signatures if on the older representation).

This is being planned for, but won't be actioned in the short to medium term.

---

Hope this brings some insight into where things are at. Will close up this issue for now as it's a partial duplicate of #490.

Just be mindful that while updating TypeBox's Tuple representation to support 2020 would be relatively straight forward, the reluctance to do it mostly stems from the ecosystems reliance on Draft 7 (as per Ajv default implementation) and that for TypeBox to adopt 2020, it needs to do it in a way that isn't going to break users (which is a forever difficult problem to solve)

Happy to field any follow up questions on this thread if you have any.
Cheers
S

[BrandonGardner2]

Thanks for that huge write up @sinclairzx81 ! that is super helpful.

On a side note, is there any formal support for $defs or the $ref pattern? I realize maybe that isn't a TypeBox issue outright but it seems to have compatibility problems with AJV at least.

For example,

I would expect to do something like:

const simpleSchema = T.Object({ foo: T.String() })

const parentSchema = T.Object({
  ...other,
 $defs: {
   simpleSchema
 }
})

const otherSchema = T.Intersect([
  T.Ref(simpleSchema),
  T.Object(otherSchema)
] )

however, you have to pass a $id field to T.Ref for it to work I believe and there are strict rules around an ID field not starting with # so you couldn't do #/$defs/simpleSchema for example.

This lead me to doing a little bit of a hack with Unsafe such as:

const UseDef = <T extends TSchema>(path: string) => T.Unsafe<Static<T>>({ $ref: path })

This is a quick write up so probably not the cleanest example but I wasn't find a clear way to achieve this without a work around similar to the above.

[sinclairzx81]

@BrandonGardner2 Hi,

On a side note, is there any formal support for $defs or the $ref pattern? I realize maybe that isn't a TypeBox issue outright but it seems to have compatibility problems with AJV at least.

TypeBox added functionality to express $defs constructs on 0.34.0. You can use the Type.Module to create a set of cross referenced types, and use Type.Ref to reference them. TypeBox produces a $defs schema for each contained type. It also handles auto $id generation derived from the schemas property name.

TypeScript Link Here

const Module = Type.Module({
  A: Type.Object({ x: Type.Number() }),
  B: Type.Object({ y: Type.Number() }),
  C: Type.Intersect([
    Type.Ref('A'), 
    Type.Ref('B')
  ])
})

const C = Module.Import('C')   // const C = {
                               //   $defs: {
                               //     A: {
                               //       type: 'object',
                               //       required: [ 'x' ],
                               //       properties: { x: { type: 'number' } },
                               //       $id: 'A',
                               //     },
                               //     B: {
                               //       type: 'object',
                               //       required: [ 'y' ],
                               //       properties: { y: { type: 'number' } },
                               //       $id: 'B',
                               //     },
                               //     C: {
                               //       allOf: [
                               //         { '$ref': 'A' },
                               //         { '$ref': 'B' }
                               //       ],
                               //       $id: 'C',
                               //     }
                               //   },
                               //   $ref: 'C',
                               // }

type C = Static<typeof C>      // type C = {
                               //   x: number;
                               // } & {
                               //   y: number;
                               // }

Just keep in mind, this functionality is quite new. If you run into any problems, feel free to ping an issue.
Cheers
S

[BrandonGardner2]

@sinclairzx81 That is great. I actually missed that entirely. I think it solves my issue for the most part. Perhaps I am missing something, though.

In a large schema would there would be an issue where the module is continually reused and it is pasting in the defs every single time as well?

[sinclairzx81]

@BrandonGardner2 Heya,

In a large schema would there would be an issue where the module is continually reused and it is pasting in the defs every single time as well?

This is a good question. At this stage, Import will return a $defs schema containing all definitions (irrespective of if they are being referenced or not). Because all definitions are returned, this equates to a ton of redundancy if publishing each imported type separately (but perhaps less of an issue if just validating with the schematics). Needless to say, TypeBox could be doing better here as generating publishable schematics is "in scope".

The functionality for Module went in on 0.34.0 (just over a week ago) and there is work underway to improve some of the finer details of the feature (including plans to cull Import to only include referenced types within the Module). However I am somewhat tempted to keep things as they are (at least until the next minor semver), and try to gather some information as to how users are trying to interact with Module (it's been a difficult thing to design for)

I am keen to keep a active discussion with users on the feature, and to learn of the use cases in which it's being applied. Did you want to start up a Discussion thread to go into some of the details?

Let me know.
S