server-reflection.md

GRPC Server Reflection Protocol

This document describes server reflection as an optional extension for servers to assist clients in runtime construction of requests without having stub information precompiled into the client.

The primary usecase for server reflection is to write (typically) command line debugging tools for talking to a grpc server. In particular, such a tool will take in a method and a payload (in human readable text format) send it to the server (typically in binary proto wire format), and then take the response and decode it to text to present to the user.

This broadly involves two problems: determining what formats (which protobuf messages) a server’s method uses, and determining how to convert messages between human readable format and the (likely binary) wire format.

Method reflection

We want to be able to answer the following queries:

1. What methods does a server export?
2. For a particular method, how do we call it? Specifically, what are the names of the methods, are those methods unary or streaming, and what are the types of the argument and result?

#TODO(dklempner): link to an actual .proto later.
package grpc.reflection.v1alpha;

message ListApisRequest {
}

message ListApisResponse {
  repeated google.protobuf.Api apis = 1;
}

message GetMethodRequest {
  string method = 1;
}
message GetMethodResponse {
  google.protobuf.Method method = 1;
}

service ServerReflection {
  rpc ListApis (ListApisRequest) returns (ListApisResponse);
  rpc GetMethod (GetMethodRequest) returns (GetMethodResponse);
}

Note that a server is under no obligation to return a complete list of all methods it supports. For example, a reverse proxy may support server reflection for methods implemented directly on the proxy but not enumerate all methods supported by its backends.

Open questions on method reflection

• Consider how to extend this protocol to support non-protobuf methods.

Argument reflection

The second half of the problem is converting between the human readable input/output of a debugging tool and the binary format understood by the method.

This is obviously dependent on protocol type. At one extreme, if both the server and the debugging tool accept JSON, there may be no need for such a conversion in the first place. At the opposite extreme, a server using a custom binary format has no hope of being supported by a generic system. The intermediate interesting common case is a server which speaks binary-proto and a debugging client which speaks either ascii-proto or json-proto.

One approach would be to require servers directly support human readable input. In the future method reflection may be extended to document such support, should it become widespread or standardized.

Protobuf descriptors

A second would be for the server to export its google::protobuf::DescriptorDatabase over the wire. This is very easy to implement in C++, and Google implementations of a similar protocol already exist in C++, Go, and Java.

This protocol mostly returns FileDescriptorProtos, which are a proto encoding of a parsed .proto file. It supports four queries:

1. The FileDescriptorProto for a given file name
2. The FileDescriptorProto for the file with a given symbol
3. The FileDescriptorProto for the file with a given extension
4. The list of known extension tag numbers of a given type

These directly correspond to the methods of google::protobuf::DescriptorDatabase. Note that this protocol includes support for extensions, which have been removed from proto3 but are still in widespread use in Google’s codebase.

Because most usecases will require also requesting the transitive dependencies of requested files, the queries will also return all transitive dependencies of the returned file. Should interesting usecases for non-transitive queries turn up later, we can easily extend the protocol to support them.

Reverse proxy traversal

One potential issue with naive reverse proxies is that, while any individual server will have a consistent and valid picture of the proto DB which is sufficient to handle incoming requests, incompatibilities will arise if the backend servers have a mix of builds. For example, if a given message is moved from foo.proto to bar.proto, and the client requests foo.proto from an old server and bar.proto from a new server, the resulting database will have a double definition.

To solve this problem, the protocol is structured as a bidirectional stream, ensuring all related requests go to a single server. This has the additional benefit that overlapping recursive requests don’t require sending a lot of redundant information, because there is a single stream to maintain context between queries.

package grpc.reflection.v1alpha;
message DescriptorDatabaseRequest {
  string host = 1;
  oneof message_request {
    string files_for_file_name = 3;
    string files_for_symbol_name = 4;
    FileContainingExtensionRequest file_containing_extension = 5;
    string list_all_extensions_of_type = 6;
  }
}

message FileContainingExtensionRequest {
  string base_message = 1;
  int64 extension_id = 2;
}

message DescriptorDatabaseResponse {
  string valid_host = 1;
  DescriptorDatabaseRequest original_request = 2;
  oneof message_response {
    // These are proto2 type google.protobuf.FileDescriptorProto, but
    // we avoid taking a dependency on descriptor.proto, which uses
    // proto2 only features, by making them opaque
    // bytes instead
    repeated bytes fd_proto = 4;
    ListAllExtensionsResponse extensions_response = 5;
    // Notably includes error code 5, NOT FOUND
    int32 error_code = 6;
  }
}

message ListAllExtensionsResponse {
  string base_type_name;
  repeated int64 extension_number;
}

service ProtoDescriptorDatabase {
  rpc DescriptorDatabaseInfo(stream DescriptorDatabaseRequest) returns (stream DescriptorDatabaseResponse);
}

Any given request must either result in an error code or an answer, usually in the form of a series of FileDescriptorProtos with the requested file itself and all previously unsent transitive imports of that file. Servers may track which FileDescriptorProtos have been sent on a given stream, for a given value of valid_host, and avoid sending them repeatedly for overlapping requests.

| message_request message | Result | | files_for_file_name | transitive closure of file name | | files_for_symbol_name | transitive closure file containing symbol | | file_containing_extension | transitive closure of file containing a given extension number of a given symbol | | list_all_extensions_of_type | ListAllExtensionsResponse containing all known extension numbers of a given type |

At some point it would make sense to additionally also support any.proto’s format. Note that known any.proto messages can be queried by symbol using this protocol even without any such support, by parsing the url and extracting the symbol name from it.

Language specific implementation thoughts

All of the information needed to implement Proto reflection is available to the code generator, but I’m not certain we actually generate this in every language. If the proto implementation in the language doesn’t have something like google::protobuf::DescriptorPool the grpc implementation for that language will need to index those FileDescriptorProtos by file and symbol and imports.

One issue is that some grpc implementations are very loosely coupled with protobufs; in such implementations it probably makes sense to split apart these reflection APIs so as not to take an additional proto dependency.