Support for later JSON Schema versions

[gregsdennis]

Your readme says that this is built for draft 6. That is now six years old and three versions behind.

Any plans on supporting later versions, especially draft 2020-12?

[sinclairzx81]

@gregsdennis Hi!

Any plans on supporting later versions, especially draft 2020-12?

I'd like to have TypeBox producing schematics inline with the 2020-12 specification, however the library is currently keeping parity with Ajv's default spec implementation which is back on Draft 6. This mostly to cause the least friction for users who integrate TypeBox with JavaScript frameworks that use Ajv for validation.

import Ajv from 'ajv'                  // Requires: import Ajv from 'ajv/dist/2020'

const result = (new Ajv()).validate({
  prefixItems: [                       // Error: strict mode: unknown keyword: "prefixItems"
    { const: 'hello' },
    { const: 'world' }
  ],
  items: false
}, ['hello', 'world'])

As of today, many JavaScript frameworks take on Draft 6 by consequence of them pulling ajv instead of the more recent ajv/dist/2020 (which is largely due to historical reasons and perhaps some apprehension to adopt recent specs for fear of breaking users). Mostly this means if TypeBox were to generate the 2019-09, 2020-12 by default, the schematics would likely cause downstream problems for users, requiring them to either use TB unsafe types to express certain constructs (like tuple) or request their respective framework take on the more recent spec (which may not be feasible in some cases).

The Draft 6 specification seems compatible with 2020-12 (as far as representable TB types go when running on Ajv). So generating Draft 6 casts the widest net.

Support Modern Specifications

I have been exploring a few options to get TypeBox producing a variety of different schema representations, and have a WIP implementation of JSON Type Definition found here. From this, there may be potential to provide @sinclair/typebox/typedef, @sinclair/typebox/2019 and @sinclair/typebox/2020 or similar in future, but probably not until TB implements a more flexible compiler infrastructure to support multiple representations.

For now, users can express 2020-12 representations using the following mechanism

Type Representation

// Tuple: Draft 2020-12
export interface TTuple2020<T extends TSchema[]> extends TSchema {
  [Kind]: 'Tuple2020',
  static: Static<TTuple<T>>
  type: 'array'
  minItems: number  
  items: false
  prefixItems: T
}
export const Tuple2020 = <T extends TSchema[]>(prefixItems: [...T]): TTuple2020<T> => {
  const [type, minItems] = ['array', prefixItems.length]
  return { [Kind]: 'Tuple2020', type, minItems, items: false, prefixItems } as any
}

Usage

import Ajv from 'ajv/dist/2020'

const result = (new Ajv()).validate(Tuple2020([
  Type.Literal('hello'),
  Type.Literal('world')
]), ['hello', 'world'])

Open to thoughts and suggestions!
S

[gregsdennis]

Have you considered using https://github.com/hyperjump-io/json-schema instead of ajv? It's by @jdesrosiers, another maintainer of the JSON Schema spec, and it's fully compliant. (I also think you using this would help break the hold that ajv has on the Javascript/Typescript world.)

[jdesrosiers]

the library is currently keeping parity with Ajv's default spec implementation which is back on Draft 6

You must mean draft-07. Ajv's default is draft-07.

Ajv hasn't been actively supported for a while now. If you're waiting for them to move forward, you're going to end up stuck in the past with them.

[sinclairzx81]

@gregsdennis @jdesrosiers Hi, thanks for the responses and reference to https://github.com/hyperjump-io/json-schema, I may be interested in including this library in the TB test suite. It looks good.

Ajv hasn't been actively supported for a while now. If you're waiting for them to move forward, you're going to end up stuck in the past with them.

Bear in mind that TypeBox doesn't actually use Ajv as a dependency, it only uses it as a tool for testing a subset of the Draft 6 (and some of 2019-09) specifications (as Ajv is presumed to have fairly rigorous conformance to these specifications). In this respect, TypeBox isn't actually limited to (or waiting on) the capabilities of Ajv (as it implements it's own validation compiler), but does still produce schematics targeting the lowest specifications as this tends to work across the broadest range of community frameworks that use Ajv for their validation (such as Fastify) and who are also on Draft 7 (you're right this is the default, my mistake) and who don't want to use the validation compiler provided by TypeBox.

Also, this library is generally more orientated towards being a simple schema builder that produces TypeScript compatible schematics only. It's generally more focused on creating schematics able to be used across a broad range of validators (so not just Ajv) while having those schematics best represent the characteristics of the TS type system. If I felt I had a choice, I would certainly opt for the modern versions of the specification as a default, but am very mindful of breaking users (or forcing them to adopt a newer specification if their existing frameworks and tools can't support it)

With all this said, TypeBox does abstract type representation (allowing it to express a multitude of different schema representations), so offering a @sinclair/typebox/2020-12 would be feasible. However this library would need a more flexible compiler system able to operate across different specifications (including JSON Type Definition), but would still likely need to keep to Draft 6 as the default (at least in the short to mid term)

Happy to discuss more!

[jdesrosiers]

Understood. I admittedly don't know much about this project.

targeting the lowest specifications

I don't think you actually meant the lowest, just not the latest. But, just in case, I'll warn that targeting the lowest specifications can be problematic. JSON Schema is not backwards compatible, so if you have a draft-04 schema, you need to a draft-04 specific implementation to support it. Many implementations (including ajv by default) have dropped support for these older versions. My suggestion is to target the latest version that has wide support in the ecosystem you cater to. That very well could be draft-07, but it's certainly not lower than that.

[sinclairzx81]

@jdesrosiers Hey, thanks for the follow up!

But, just in case, I'll warn that targeting the lowest specifications can be problematic. JSON Schema is not backwards compatible, so if you have a draft-04 schema, you need to a draft-04 specific implementation to support it.

So running the libraries test suite against the 2012-12 specification (Ajv), I note it's only Type.Tuple that is failing (all the other types/schematics should technically work ok across legacy and newer versions of the spec), although the failing tests may have to do with Ajv's handling of tuples/arrays on 2012 (where I think there was some issues here around items and additionalItems, but I can't recall what they were), I'll investigate.

I think what might be a good idea is to take a deeper look at your https://github.com/hyperjump-io/json-schema library and run the current test suite through it (and test each Type schematic tested against each version of the specification (as I see you support a them all)). Could probably generate a table of supported types per spec. It would be good to get some very strict tests in place here, where I cannot say for certain Ajv is being perfectly accurate in it's interpretation.

I'll have a think about things. I guess I'm more open to supporting draft 2020-12 (default) if I've first "done the work" to verify ALL the existing types against ALL the specifications (including Draft 4), and communicated spec support in documentation (as a table). Getting visibility here would be a good first step to getting on the newer specifications, and possibly promote the idea of users getting on newer versions of the spec to get support for certain types (like tuple)

[jdesrosiers]

Here's a summary of the changes to items/additionalItems, https://json-schema.org/draft/2020-12/release-notes.html#changes-to-items-and-additionalitems. I don't think there are any other changes that would effect your library across the versions.

[gregsdennis]

There are some notable changes listed in https://json-schema.org/draft/2019-09/release-notes.html as well.

• definitions is now $defs
• dependencies was split to dependentRequired and dependentSchemas

Other things are additions.