Gradual(?) rollout of Signal K v2 features

[tkurki]

@panaaj and I (well, mostly Adrian) have been working on two new Signal K APIs:

• Resources API
• Course API

While Signal K v1 was a lot about defining the paths, their hierarchy and the full data model schema, the current ideas for Signal K v2 are centered around multiple APIs (HTTP endpoints) that complete a specific operation (e.g. set destination) defined as OpenApi specifications. Calling a SK v2 API to perform an operation will result in values being set in all the related SK paths.

Here’s a rambling list of considerations related to v1 and v2:

There will be changes in the path definitions

There are some mistakes that we’d like to fix, for example, like having duplicate Great Circle and Rhumbline course paths.
So in addressing these duplicate paths, the resultant changes are likley to break compatibility with v1.

Full model paths are not the starting point

The v2 APIs expose HTTP endpoints which enact operations, e.g activate a route, advance to next point, etc - they don’t just expose a generic data model with some well known paths. There is a connection with SK full data model, but it is not 1:1 and full model paths are not the starting point for the design.

Coexistence with v1

We can mount the new APIs under /signalk/v2/ so they can coexist with v1 APIs

Connection with v1 full data

The current implementation of the v2 API emits deltas that use v2 paths and structure which end up in the full model.
Is this ok?
Should deltas originating from v2 API be flagged in some way?
Should there be a separate means for v2 data access using the v1 full mechanism?

How do we stream updates?

As the paths are not v1 compatible how should a client treat the data?
Should there be a single stream, with v1 and v2 values intermixed and do we need a way to discern the version of a path/value?
or should there be two ws endpoints and a client can subscribe to both or either?

The current v2 implementation emits values in the deltas that are object valued.
For example navigation.course.previousPoint.position is not sent separately as a course is activated. Instead you get a composite object for navigation.course which includes activeRoute, nextPoint and previousPoint.

How do we advertise the new services / how does discovery work?

One way to proceed would be to add a new v2 http endpoint in the /signalk response. If we assume streaming works within the context of the v1 world, this would allow clients to discover v2 support and adjust.

GET HTTP http://demo.signalk.org/signalk would return

{
  “endpoints”: {
    “v1": {
      “version”: “...“,
      “signalk-http”: “http://demo.signalk.org/signalk/v1/api/”,
      “signalk-ws”: “ws://demo.signalk.org/signalk/v1/stream”,
      “signalk-tcp”: “tcp://demo.signalk.org:8375”
    },
    “v2”: {
      “signalk-http”: “http://demo.signalk.org/signalk/v2/api/”
    }
  },
  “server”: {
    “id”: “signalk-server-node”,
    “version”: “...”
  }
}

Units

How do we document the units used in the different payloads when using OpenAPI?

Closing

I think v2 is going to be work in progress and subject to change for quite some time. There’s a lot of things here to work out.

[bobhy]

I think the discussion about discovery is key to figuring out the coexistence of V1 and V2. V2 should stand on its own as a functionally complete and self-consistent interface, even though for a long time there won't be a functionally complete implementation.

But you can and should document a functionally complete discovery protocol day one, and the first releases of V2-aware components should implement all of it. The discovery protocol should anticipate V2 services that haven't been invented yet, and even the possibility of a whole V3 protocol that hasn't been imagined yet.
So the discovery example you give above might be the response you get from a soon-to-be released version of a server that is not yet fully functional in V2, but the V2 spec should document the subkey for the streaming API endpoint and that the value is some kind of URI (I presume), even if you don't know yet what the details of the endpoint are.

I guess it's an accepted goal that V2 is a functional superset of V1> that there is (eventually ) a way to do everything in V2 that you do today in V1 (minus the misguided bugs)? So the early versions of the spec (and initial V2-aware releases) should say a lot about sensor data and streaming, and how the client collects all this.

So I have 2 suggestions for data model improvements V2:

• Anticipating a world where many sensors are native (V2) servers, allow sensor to report data in non-SI units and make discovery of what those units are a lot more "natural" and standardized than current metadata. Means V2 should specify how a server (sensor) can dynamically register a new unit of measure and document how it relates to SI units (e.g for a core marine or a de novo physical quantity).
• Document an inherently multicast way for a client to listen to a collection of servers at once, again, where many of the servers on the network are simple sensors that just happen to speak SK V2. CANbus is a given, and it's an inherently multicast medium, but SK so far has more of the gateway flavor, where there's only one interesting SK data source to listen to on the net.