On Thu, Mar 3, 2016 at 6:37 PM, Tim Kientzle <[email protected]> wrote:
>
> I think your current Any JSON design requires that protobuf serialization
> be able to fail. This is definitely a major change from proto2 and
> something the conformance test does not explore.
>
> 1. Decode a valid JSON-encoded message with an Any field in an unknown
> schema.
>
> 2. Attempt to encode it in protobuf format.
>
> If the descriptor cannot be fetched, then step #2 here must fail.
>
> (The alternative would be for #1 to fail in this case, but that would
> imply that JSON-to-JSON transcoding must fail in some circumstances.)
>
With the current Any JSON design, #1 must fail. It's the same for the other
direction. Encoding an Any message with an unknown schema to JSON must fail
as well.
>
> If Any had been defined slightly differently, then transcoding would be
> trivial. If you are unable to change the current definition of Any, please
> consider introducing an alternative:
>
> package google.protobuf;
>
> message Other {
> string type_url = 1;
> oneof encoded {
> bytes protobuf = 2;
> google.protobuf.Value json = 3;
> }
> }
>
This and similar proposals (like storing JSON in the value field and append
a ":json" suffix to type URL to tell consumers the payload is JSON) had
been considered in the design process, and it's decided to stay with the
current Any representation. +liujisi may be able to provide more context
around this decision.
I think the advantage of the current Any representation is that the JSON to
protobuf is done once in the proxy without the end users to worry about
whether the Any payload is protobuf or JSON (i.e., it simplifies the end
user APIs). With your proposal, the conversion from JSON to protobuf will
spread out to all places. Every reader of the data must be able to do the
conversion in order to read the data. That will complicate things quite a
bit in a large system.
>
> This is trivial to transcode: it only requires that the final consumer be
> able to decode whatever format the originator attached. Intermediate
> systems can freely transcode without needing to understand the schema at
> all, since the ‘protobuf’ field here can be represented in JSON and the
> ‘json’ field can be encoded to protobuf. The actual JSON serialization is
> very similar to the current Any serialization (with the addition of the
> extra “json” tag).
>
> Cheers,
>
> Tim
>
>
>
> > On Feb 5, 2016, at 1:53 PM, Josh Haberman <[email protected]> wrote:
> >
> > Hi Tim,
> >
> > I think your analysis is correct. The answer to your question is that
> it's not possible to transcode Any from JSON <-> protobuf unless you have
> the schema. That is one of the reasons why the type URL exists as a way to
> fetch schemas for types you otherwise wouldn't know about.
> >
> > I'm glad the conformance suite was helpful!
> >
> > Best,
> > Josh
> >
> > On Wednesday, February 3, 2016 at 1:37:40 PM UTC-8, Tim Kientzle wrote:
> > I’m trying to implement Any, including proper JSON and protobuf
> encoding, and the following scenario has me stumped:
> >
> >
> >
> > I’m looking at the any.proto file, which includes this example:
> >
> >
> >
> > Sample proto file:
> >
> > // package google.profile;
> >
> > // message Person {
> >
> > // string first_name = 1;
> >
> > // string last_name = 2;
> >
> > // }
> >
> >
> >
> > Resulting JSON encoding:
> >
> > // {
> >
> > // "@type": "type.googleapis.com/google.profile.Person",
> >
> > // "firstName": <string>,
> >
> > // "lastName": <string>
> >
> > // }
> >
> >
> >
> > The scenario I’m trying to understand involves three systems:
> >
> >
> >
> > Originating system:
> >
> > * Knows google.profile and writes out the JSON above
> >
> >
> >
> > Middle system:
> >
> > * Reads the JSON example above
> >
> > * Does *not* know google.profile (doesn’t have the descriptor and has no
> way to get it)
> >
> > * Writes out protobuf
> >
> >
> >
> > Receiving system:
> >
> > * Knows google.profile and reads the protobuf
> >
> >
> >
> > In particular: What is the protobuf serialization between the middle
> and receiving systems?
> >
> >
> >
> > Without the descriptor, the middle system cannot poduce the “real"
> protobuf binary format (the field numbers aren’t in that format), so the
> only option would seem to be to serialize the JSON data as a protobuf
> Struct representation (which in turn has interesting implications for the
> receiving system). Am I missing some other option?
> >
> >
> >
> > Tim
> >
> >
> >
> > P.S. Nice work on the conformance test suite, by the way! That
> clarified a *lot* of details about the expected handling for protobuf JSON
> format.
> >
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Protocol Buffers" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected].
> > To post to this group, send email to [email protected].
> > Visit this group at https://groups.google.com/group/protobuf.
> > For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
