Wrapper message for google.protobuf.Any + google.protobuf.Type

[chriso]

Dispatch functions accept a google.protobuf.Any as input and output, which carries a type URL along with a serialized value (opaque bytes).

Tools and systems observing the stream of Dispatch requests and responses, for example the CLI and Console, would ideally be able to interpret the bytes in order to provide observability features (e.g. you called function X with input Y).

From https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/any.proto:

The URL/resource name uniquely identifies the type of the serialized
protocol buffer message. This string must contain at least
one "/" character. The last segment of the URL's path must represent
the fully qualified name of the type (as in
path/google.protobuf.Duration). The name should be in a canonical form
(e.g., leading "." is not accepted).

In practice, teams usually precompile into the binary all types that they
expect it to use in the context of Any. However, for URLs which use the
scheme http, https, or no scheme, one can optionally set up a type
server that maps type URLs to message definitions as follows:

• If no scheme is provided, https is assumed.
• An HTTP GET on the URL must yield a google.protobuf.Type
  value in binary format, or produce an error.
• Applications are allowed to cache lookup results based on the
  URL, or have them precompiled into a binary to avoid any
  lookup. Therefore, binary compatibility needs to be preserved
  on changes to types. (Use versioned type names to manage
  breaking changes.)

Note: this functionality is not currently available in the official
protobuf release, and it is not used for type URLs beginning with
type.googleapis.com. As of May 2023, there are no widely used type server
implementations and no plans to implement one.

The type URL may serve a google.protobuf.Type, but in practice this doesn't seem to be common. The built-in protobuf messages have type URLs that 404 (try http://type.googleapis.com/google.protobuf.BytesValue), and same for Buf (e.g. try http://buf.build/stealthrocket/dispatch-proto/dispatch.sdk.python.v1.Pickled).

To enable introspection of these messages, an alternative would be to wrap the google.protobuf.Any in a message that also carries a google.protobuf.Type, e.g.

message TypedAny {
   google.protobuf.Type type = 1;
   google.protobuf.Any any = 2;
}

The SDKs would be responsible for wrapping inputs and outputs and attaching a type, just like they're responsible for boxing proto messages in a google.protobuf.Any message today.

The downside of this approach is that it's extra space on the wire. Each message carries not only the serialized input/output, but also a google.protobuf.Type and two type URLs (one for the inner google.protobuf.Any, and one for the outer TypedAny).