On Mon, Aug 26, 2019, at 05:23, Magnus Edenhill wrote:
> Great KIP as always, Colin!
>
> Some comments:
>
> > If the flexible versions are not specified, it is assumed that all
> versions are flexible.
>
> This is ambiguous, if a protocol-generator is pointed to an older Kafka
> protocol specif
Great KIP as always, Colin!
Some comments:
> If the flexible versions are not specified, it is assumed that all
versions are flexible.
This is ambiguous, if a protocol-generator is pointed to an older Kafka
protocol specification
it can't know if the lack of flexibleVersions field means they're
Hi Satish,
I wasn't originally going to propose supporting the struct type, although
perhaps we could consider it.
In general, supporting a struct containing an array has the same serialization
issues as just supporting the array.
Probably, we should just have a two-pass serialization mechanis
Hi David,
Good point. I think it makes the most sense to put the tagged fields sections
at the end of the header. For consistency's sake, we can also put them at the
end of the structures / requests / responses as well. I've updated the KIP.
best,
Colin
On Mon, Aug 19, 2019, at 02:23, David
Hi Colin,
Thank you for the updated KIP.
wrt. Request and Response Headers v1, where do you plan to put the optional
fields? You mention that all the Requests and Responses will start with a
tagged field buffer. Is it for that purpose?
Best,
David
On Mon, Aug 19, 2019 at 7:27 AM Satish Duggana
Please read struct type as a complex record type in my earlier mail.
The complex type seems to be defined as Schema[1] in the protocol
types.
1.
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/protocol/types/Schema.java#L27
On Mon, Aug 19, 2019 at 9:46 A
Sorry! Colin, I may not have been clear in my earlier query about
optional field type restriction. It is mentioned in one of your
replies "optional fields are serialized starting with their total
length". This brings the question of whether optional fields support
struct types (with or without arra
On Tue, Aug 13, 2019, at 10:01, Jason Gustafson wrote:
> > Right, I was planning on doing exactly that for all the auto-generated
> > RPCs. For the manual RPCs, it would be a lot of work. It’s probably a
> > better use of time to convert the manual ones to auto gen first (with the
> > possible exce
On Tue, Aug 13, 2019, at 11:19, David Jacot wrote:
> Hi Colin,
>
> Thank you for the KIP! Things are well explained!. It is huge improvement
> for the Kafka protocol. I have few comments on the proposal:
>
> 1. The interleaved tag/length header sounds like a great optimisation as it
> would be sh
Hi Colin,
Thanks for the KIP. Optional fields and var length encoding support is a great
improvement for the protocol.
>>Optional fields can have any type, except that they cannot be arrays.
Note that the restriction against having tagged arrays is just to simplify
serialization. We can relax thi
Hi Colin,
Thank you for the KIP! Things are well explained!. It is huge improvement
for the Kafka protocol. I have few comments on the proposal:
1. The interleaved tag/length header sounds like a great optimisation as it
would be shorter on average. The downside, as
you already pointed out, is th
> Right, I was planning on doing exactly that for all the auto-generated
RPCs. For the manual RPCs, it would be a lot of work. It’s probably a
better use of time to convert the manual ones to auto gen first (with the
possible exception of Fetch/Produce, where the ROI may be higher for the
manual wo
On Mon, Aug 12, 2019, at 11:22, Jason Gustafson wrote:
> Hi Colin,
>
> Thanks for the KIP! This is a significant improvement. One of my personal
> interests in this proposal is solving the compatibility problems we have
> with the internal schemas used to define consumer offsets and transaction
>
Hi Colin,
Thanks for the KIP! This is a significant improvement. One of my personal
interests in this proposal is solving the compatibility problems we have
with the internal schemas used to define consumer offsets and transaction
metadata. Currently we have to guard schema bumps with the inter-br
Hi all,
I've made some updates to this KIP. Specifically, I wanted to avoid including
escape bytes in the serialization format, since it was too complex. Also, I
think this is a good opportunity to slim down our variable length fields.
best,
Colin
On Thu, Jul 11, 2019, at 20:52, Colin McCab
On Tue, Jul 9, 2019, at 15:29, Jose Armando Garcia Sancio wrote:
> Thanks Colin for the KIP. For my own edification why are we doing this
> "Optional fields can have any type, except for an array of structures."?
> Why can't we have an array of structures?
Optional fields are serialized starting w
Thanks Colin for the KIP. For my own edification why are we doing this
"Optional fields can have any type, except for an array of structures."?
Why can't we have an array of structures?
--
-Jose
On Tue, Jul 2, 2019, at 02:55, Tom Bentley wrote:
> Hi Colin,
>
> In the example given, of a FooResponse, both the optional fields have the
> tag 0x0001. The text says "This number must be unique within the context it
> appears in.". My first thought was the example tags, must be wrong. But I
> th
Hi Colin,
In the example given, of a FooResponse, both the optional fields have the
tag 0x0001. The text says "This number must be unique within the context it
appears in.". My first thought was the example tags, must be wrong. But I
think I misunderstood what you meant by "the context" – I assume
Hi all,
I would like to start a discussion for KIP-482:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-482%3A+The+Kafka+Protocol+should+Support+Optional+Fields
cheers,
Colin
20 matches
Mail list logo
