v0.11.0

2021 August 12

This release is an odd numbered release, which means it may contain breaking changes.

Unfortunately, the changes here may be particularly, tricky, as well -- for the most part, they're not compile-time detectable. They're behavioral changes. Much more subtle. Run tests on your systems before accepting these changes. Specifically: several codecs now enforce sorting when emitting serial data.

There's also some details of what's changing that makes it milder than it first sounds: most of the changes are around codecs becoming more spec-compliant. So, for example, if you were using another IPLD library that always enforced sorting on e.g. DAG-CBOR, you won't be surprised or experience it much like a "change" when using this version of go-ipld-prime, which now also enforces such sorting in that codec.

Also! At least one huge and awesome new feature: bindnode. This is a new implementation of ipld.Node which can bind to native golang structures using reflection, which provides a new and easy-to-use way to move data in and out of golang structures (or traverse them, etc!) with IPLD interfaces and codecs.

See the full change list for details:

• New: some new helpful constructors for making Selectors out of serial forms can now be found in the traversal/selector/parse package.[#199]
  • Some constants are also included which show some examples of creating common selectors from JSON.
• Fixed: cbor, dag-cbor, json, and dag-json codecs now all accept parsing a block that contains just a null token alone. (Previously, this returned an "unexpected EOF" error, which was silly.) [#217]
• Fixed (upstream): json floats are actually supported. (You might've had this already, if anything dragged in a newer version of the refmt library. We just make sure to require this ourselves in our go.mod file now.) [#215]
• New: Selectors now support some kinds of conditions. Specifically, ExploreRecursive clauses can contain a stopAt condition, and the condition system now supports Condition_IsLink, which can be used to do an equality check for CIDs. [#214]
• Fixed: in codegen'd types, the LinkTargetNodePrototype on links was returning the wrong prototype; now it returns the right one. [#211]
• New: schema.TypedPrototype interface, which is like ipld.NodePrototype but also has methods for asking Type() schema.Type and Representation() ipld.NodePrototype, both of which should probably instantly make sense to you. [#195]
• Changed: the dag-json and dag-cbor codecs now apply sorting. [#203, #204]
  • This means all serial data created with these codecs is sorted as advised by their respective specifications. Previously, the implementations of these codecs was order-preserving, and emitted data in whatever order the ipld.Node yielded it.
  • There may be new performance costs originating from this sorting.
  • The codecs do not reject other orderings when parsing serial data. The ipld.Node trees resulting from deserialization will still preserve the serialized order. However, it has now become impossible to re-encode data in that same preserved order.
  • If doing your own encoding, there are customization options in dagcbor.EncodeOptions.MapSortMode and dagjson.EncodeOptions.MapSortMode. (However, note that these options are not available to you while using any systems that only operate in terms of multicodec codes.)
  • Be cautious of this change. It is now extremely easy to write code which puts data into an ipld.Node in memory in one order, then save and load that data using these codecs, and end up with different data as a result because the sorting changes the order of data. For some applications, this may not be a problem; for others, it may be surprising. In particular, mind this carefully in the presense of other order-sensitive logic -- for example, such as when using Selectors, whose behaviors also depend on ordering of data returned when iterating over an ipld.Node.
• Fixed/Changed: the dag-json codec no longer emits whitespace (!). It is now spec-compliant. [#202]
  • This means hashes of content produced by dag-json codec will change. This is unfortunate, but the previous implementation was woefully and wildly out of sync with the spec, and addressing that is a predominating concern.
• Removed: fluent/quip has been dropped. fluent/qp is superior. fluent/quip was too easy to use incorrectly, so we no longer offer it. [#197]
  • This was an experimental package introduced a few releases ago, together with caveats that we may choose to drop it. The warning was purposeful! We don't believe that this will be too painful of a change; not many things depended on the fluent/quip variant, and those that did should not be difficult to rewrite to fluent/qp.
• New: node/basic.Chooser is a function that implements traversal.LinkTargetNodePrototypeChooser. It's a small handy quality-of-life increase if you need to supply such a function, which is common. [#198]
• New: bindnode! This is a huge feature. The beginnings of it may have been visible in v0.10.0, but it's grown into a usable thing we're ready to talk about.
  • Bindnode lets you write golang types and structures, and "bind" them into being IPLD Nodes and supporting Data Model operations by using golang reflection.
  • The result of working with bindnode is somewhere between using basicnode and using codegen: it's going to provide some structural constraints (like codegen) and provide moderate performance (it lets you use structs rather than memory-expensive maps; but reflection is still going to be slower than codegen).
  • However, most importantly, bindnode is nice to use. It doesn't have a huge barrier to entry like codegen does.
  • bindnode can be used with or without IPLD Schemas. For basic golang types, a schema can be inferred automatically. For more advanced features (e.g. any representation customization), you can provide a Schema.
  • Please note that though it is now usable, bindnode remains in development. There is not yet any promise that it will be frozen against changes.
    • In fact, several changes are expected; in particular, be advised there is some sizable change expected around the shape of golang types expected for unions.
• Improved: tests for behavior of schema typed nodes are now extracted to a package, where they are reusable.
  • The same tests now cover the bindnode implementation, as well as being used in tests of our codegen outputs.
  • Previously, these tests were already mostly agnostic of implementation, but had been thrown into packages in a way that made them hard to reuse.
• Improved (or Fixed, depending on your point of view): dag-json codec now supports bytes as per the spec. [#166,#216]
  • Bytes are encoded in roughly this form: {"/":{"bytes":"base64data"}}.
  • Note: the json codec does not include this behavior; this is behavior specific to dag-json.