Agree, it's a bit confusing based on wiki alone right now. I think of MemberAssignment as a double-serialized field: first serialize to bytes using the MemberAssignment definition (version, topics, partitions, etc); then serialize the bytes into a request. It's the second serialization step that adds the 32bit length. Repeat backwards for deserializing.
-Dana On Mar 27, 2016 6:01 AM, "Cees de Groot" <c...@pagerduty.com> wrote: > Ah. That makes sense. That also means that on the outgoing request, this > part should be coded as <length><MemberAssignment bytes>? These parts of > the documentation aren't entirely clear on what is part of the > client/server protocol and what is part of the (client-side only) consumer > group implementation. > > I'll put my request and response next to each other, I do send some > assignments in the SyncGroup request but it may be that because of me > misinterpreting the docs I accidently drop four zeroes in a bit that the > server expects as a length marker. > > On Sat, Mar 26, 2016 at 6:05 PM, Dana Powers <dana.pow...@gmail.com> > wrote: > > > The MemberAssignment bytes returned in SyncResponse should be the bytes > > that your leader sent in its SyncRequest. <<0, 0, 0, 0>> is simply an > empty > > Bytes array (32 bit length of 0). The broker does not alter those bytes > as > > far as I know, so despite the protocol doc describing what a > > MemberAssignment struct *should* look like, it really is up to your > client > > to encode and decode that struct. Can you check what bytes your leader > code > > is sending? > > > > I assume you have this covered, but for completeness: each consumer > should > > check its JoinResponse to determine whether it has been selected as the > > group leader. If the consumer is the leader, it needs to do the > assignments > > and send those back in its SyncRequest. All other consumers just send an > > empty SyncRequest. SyncResponse for all consumers then includes the > > assignments sent by the leader. > > > > -Dana > > > > On Sat, Mar 26, 2016 at 2:39 PM, Cees de Groot <c...@pagerduty.com> > wrote: > > > > > I'm helping out on adding 0.9 stuff to the Elixir Kafka driver ( > > > https://github.com/kafkaex/kafka_ex), and I'm getting an unexpected > > > response on an integration test that makes a simple SyncGroup call. > > > > > > In Elixir terms, I'm getting <<0, 0, 0, 3, 0, 0, 0, 0, 0, 0>> back. My > > > interpretation: > > > > > > 32 bits correlation id = 0,0,0,3 > > > 16 bits error code = 0,0 > > > > > > There's now 32 bits left. However, I'm expecting (following > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-SyncGroupResponse > > > ) > > > a MemberAssignment structure here, which is > > > > > > 16 bits of version (0) > > > 32 bits of length (it's a simple integration test, so for now i'm happy > > to > > > accept the answer "0 assignments for you, dude!" ;-)). > > > > > > Which clearly is more data than left in the response. > > > > > > Am I misinterpreting the documentation here? > > > > > > > > > -- > > *Cees de Groot* > PRINCIPAL SOFTWARE ENGINEER > [image: PagerDuty logo] <http://pagerduty.com/> > pagerduty.com > c...@pagerduty.com <m...@pagerduty.com> > +1(416)435-4085 > > [image: Twitter] <http://twitter.com/pagerduty>[image: FaceBook] > <https://www.facebook.com/PagerDuty>[image: Google+] > <https://plus.google.com/114339089137644062989>[image: LinkedIn] > <https://www.linkedin.com/company/pagerduty>[image: Blog] > <https://blog.pagerduty.com/> >
