Hi everyone,
Since protobufs are used quite a bit in network protocols, I'm looking for
some advice requiring extensible protocol definitions using protos. Me nor
any of my colleagues have used them for this purpose, so trying to avoid
blunders. The other end of the connection is a mobile device using
protobuf-lite, so I'm not using "Any" fields that don't have library
support on the lite version. The use case is pushing various types of
messages to a mobile device and back over WebSocket or UDP without really
having to concern over transport level issues.
I'm leaving out the authentication part of the protocol, since it's not
relevant here. There are some key requirements:
1. It should be possible to ack messages (to support e.g. delivery
callbacks and return errors).
2. It should be easy to add new payload types as well as remove old ones
that are obsolete.
3. The protocol needs to be versioned and the server might need to handle
old clients.
It would also help if types are chosen in a way that makes parsing quick
without sacrificing ease of use and extensibility, since the server fleet
will handle millions of concurrent users.
With these requirements in place, a somewhat simplified protocol definition
with a payload type invented for this question would be:
syntax = "proto3";
package protocol;
import "google/protobuf/timestamp.proto";
// Basic protocol
enum Platform {
PLATFORM_UNSPECIFIED = 0;
PLATFORM_ANDROID = 1;
PLATFORM_IOS = 2;
}
enum MessageKind {
MESSAGE_KIND_UNSPECIFIED = 0;
MESSAGE_KIND_STATUS = 1;
MESSAGE_KIND_NOTIFICATION = 2;
// ... and others
}
enum StatusCode {
STATUS_CODE_UNSPECIFIED = 0;
STATUS_CODE_OK = 1;
STATUS_CODE_PERMISSION_DENIED = 2;
STATUS_CODE_INTERNAL_ERROR = 3;
}
// Every message written onto the wire is of this type.
message ProtocolMessage {
int32 version = 1;
int32 sequence_number = 2;
MessageKind kind = 3; // Indicates protobuf type encoded in 'data'
field.
bytes data = 4; // Serialized protobuf.
}
message Status {
int32 sequence_number = 1;
StatusCode code = 2;
string error_string = 3;
}
// Notification payload
message NotificationMessage {
google.protobuf.Timestamp issued_at = 1;
google.protobuf.Timestamp expiration = 2;
google.protobuf.Timestamp not_before = 3;
oneof kind {
Promotion promotion = 4;
PurchaseCompleted purchase = 5;
// ... and others
}
}
Any comments or improvement ideas? Does oneof make good sense here for fast
parsing and extensibility/deprecation? Does it make sense to e.g. group
various parts of the messages and choose field numbers from a specific
range for each group? For example, in "NotificationMessage" would it make
sense to start the field numbers in the oneof at e.g. 100 to make room for
adding more fields before that? Of course, nothing forces to write the
fields in the protobuf definition in increasing numerical order. I remember
having seen this done in some protos back in the day when I worked at
Google.
I'm planning on phasing out fields by prepending "deprecated_" to fields
that will be removed and then prepend the field name with "OBSOLETE_" once
it's not in use anywhere.
Interested if someone has any lessons learned from using protobufs in
network protocols and improvement ideas to make the design more future
proof.
Best,
Edvard
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/66772256-2f20-4bb2-941f-0b88363a102fn%40googlegroups.com.
