Thanks to both of you. That all makes a ton of sense.
I'm thinking about the use of capnp in an environment where the
systems producing and consuming messages are at least sometimes under
enough control to make this kind of thing possible/desirable.
There are some cases where the native capnp versioning behavior seems
highly congenial, and others where I'm less certain. If it's not too
much of a bore, here's another case I've been thinking of that I'm not
sure how to handle.
Imagine I have an RPC protocol where the request has this form:
struct ListMatchingPeople {
age @0: Text;
emailDomain @1: Text;
}
Here, the implied semantics of the RPC is that it should return a list
of all people who match the listed criteria. Now, let's say I decide
that I want to extend the RPC to allow people to also filter by
occupation, so I add a new field.
struct ListMatchingPeople {
age @0: Text;
emailDomain @1: Text;
occupation @2: Text = "any";
}
Note that this has the nice property that the default value of the
field has the same semantics as just omitting the field, so if an old
client sends a message to a new RPC server, it will get the behavior
that would be expected.
The reverse versioning story doesn't work out that well, though. If I
send a message from a new client to an old server, then any occupation
specified by the old client will be unceremoniously ignored. You
might prefer the behavior of having the new message be rejected when a
non-default value for occupation was sent, but I think there's no way
to implement that within capnp.
Again, I'm curious in practice how people deal with this kind of
issue. Maybe the approach is simply as before to be aware of this
kind of problem, and roll the server before you roll clients.
You could also imagine some kind of dynamic exchange and validation of
schema that could detect this problem in advance, but since there's no
schema compatibility validator at present, I imagine no one is doing
that...
y
On Tue, May 14, 2019 at 5:12 PM Kenton Varda <[email protected]> wrote:
>
> Hi Yaron,
>
> Ian already answered the question, but I thought I'd add:
>
> For protocols that are published publicly and used by arbitrary parties that
> you don't control, retroactive unionization may indeed be too unsafe to
> really use.
>
> Many protocols, though, are used privately between components of a system. In
> this case, forwards- and backwards-compatibility may be important in order to
> allow components to be updated independently, but compatibility only needs to
> extend between all components that are currently in production. In that case,
> it's quite common to do something like:
>
> 1) Retroactively unionize a field, but don't actually use them new variant
> yet.
> 2) Update each component that receives messages of the modified type, so that
> they are aware of the union.
> 3) Now, start setting the new variant where desired.
>
> -Kenton
>
> On Tue, May 14, 2019 at 1:06 PM Yaron Minsky <[email protected]> wrote:
>>
>> Retroactive unionization is only backwards compatible, not forward
>> compatible, right? So, if I start with this struct:
>>
>> struct Person {
>> name @0 :Text;
>> email @1 :Text;
>> }
>>
>> And decide that I want to evolve it to this one:
>>
>> struct Person {
>> name @0 :Text;
>> union {
>> email @1 :Text;
>> age @2 :Float64;
>> }
>> }
>>
>> (I know it's not a very meaningful example).
>>
>> If I write something in the new spec that uses the age branch of the
>> union, the old struct can try to read it, and get very confused. In
>> particular, if someone tries to read the email for a struct that
>> actually populates age, they'll end up reading a Float64 as if it were
>> a pointer to a text block.
>>
>> Am I understanding the issue correctly? If so, how do people handle
>> these kinds of protocol changes? Do people use retroactive
>> unionization in practice? Do people use schema validation of some
>> kind to detect when someone makes a potentially unsafe change like
>> this one?
>>
>> y
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> Visit this group at https://groups.google.com/group/capnproto.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/capnproto/CACLX4jRucxh5%2BmrvkkbcvTgmEbxeAcY%2BEJa4XKw5y_-DZGHorQ%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
Visit this group at https://groups.google.com/group/capnproto.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/CACLX4jQ2_PJtPFxH0NxDw64P5K4UFNqkrVr7ohyjNCfCiG95pQ%40mail.gmail.com.
