Adds Ion Schema 2.0 specification

[popematt]

Issue #, if available:

Fixes #76

Description of changes:

• Adds a BNF-style grammar
• Adds a prose description of Ion Schema Language 2.0

Examples are still TODO because I would like to put the examples into ion-schema-tests and then embed them into the spec from there.

You can see the rendered view of these changes at https://popematt.github.io/ion-schema/docs/isl-2-0/spec

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[rmarrowstone]

... that is non-null. Maybe a little refactoring of this section would avoid needing to respecify that.

[rmarrowstone]

More broadly... I read the writeup about the potential ambiguities with nullable in 1.0 and my recollection from working with it was that it was confusing. So I agree that we can and should do better with 2.0.

I also agree that the solution should be conceptually sound and it should be reasonably obvious how to do the common things. I can see how the model here reflects the ion type system, but I suspect that users will find it confusing, for two reasons: remembering presence or absence of '$' sigil (consider something like Kotlin's '?'), and my experience is that most users, when something is null, don't actually care if it's strongly typed or not.

Were there some other models that were considered and rejected?

[popematt]

Maybe a little refactoring of this section would avoid needing to respecify that

Sure, I can do that.

Were there some other models that were considered and rejected?

Does this question refer to other models for the built-in types in general, or does it refer specifically to the replacement for nullable?

[rmarrowstone]

Replacement for nullable.

[popematt]

Yes. Based on the goal of finding a replacement that was explicit about what would be null (ie. no figuring out "base types"), we came up with the following alternatives that were considered:

1. nullable field in type definitions that define which typed nulls are allowed with the nullable annotation. This was rejected for being "awkward" and also because it looks like a constraint but behaves unlike any other constraint. The nullable field could also be on a type that is imported from another file, making it more difficult for a human reader to discover what sort of null would be allowed.
2. nullable:: annotation changed to always allow any sort of null at all. This was rejected because it would be weird/unintuitive for nullable::struct to allow a null.int—it doesn't properly support any use case that people would want to use nullable for.
3. Get rid of nullable entirely. This was considered the second-best option, but everyone who was consulted agreed that nullability is a common enough use case that we should provide some sort of syntactical sugar for it.
4. Replace nullable with nullable_text, nullable_string, nullable_struct, etc. annotations to indicate which types of nulls should be allowed. This was rejected for being more complex than what seemed necessary. However, rejecting this is not a one-way door with respect to the chosen solution. It would be possible in Ion Schema 2.1 to add annotations like null_string_or::, etc. so that there is syntactical sugar for all of the strongly-typed nulls.

[rmarrowstone]

I'd be curious how often folks want to constrain to allow untyped but not typed null.

[popematt]

We don't know, nor do we have a good way of knowing.

my experience is that most users, when something is null, don't actually care if it's strongly typed or not

I agree, but I would add I think users benefit from having a single representation of null within their application. So they might not care if it's a strongly typed null or not, but they don't want a mix of the two. It's like indenting code—in many languages you can indent with 2 spaces or 4 spaces (we're ignoring tab characters). What matters most is not how much white space is used for indenting, but that the amount of whitespace is consistent.

[desaikd]

Overall looks good!
I like all the notes for constraint sections, those seem very helpful to me. I have added some minor suggestions.

[zslayton]

correspond precisely with the types defined by the Ion data model, including strongly-typed null values

I find this (pre-existing) sentence is a bit muddy. I think it means that (e.g.) $int represents the union of [int and null.int] and does NOT include null/null.null. If that's the case, I think we should spell it out.

[zslayton]

Nice work, Matt!

[tgregg]

For accuracy, consider "biggest extant species of penguin". We don't want to anger fans of the late Colossus penguin.

[popematt]

🐧 👍

[tgregg]

Nice.

[net-yehor-tretiakov]

Hey! Esteemed ion-schema contributors, and especially @popematt, I have just started working on a new project that will use ion as encoding/decoding. And i need to write some codegen for different langs depend on .isl files.
Would you mind sharing with me an ion-schema ebnf or smth like ANTLR4 full grammar? I saw https://amazon-ion.github.io/ion-schema/docs/isl-2-0/bnf-grammar, but it cannot be used due to obvious reasons.
Basically, i need to write some kind of schema-lang file parser at first.