I'm getting "Unknown api code 11" even when both client and server are
0.8.3/trunk, when "KafkaConsumer.subscribe(String... topics)" is used.
Bug?
Kind regards,
Stevo Slavic.
On Fri, Apr 24, 2015 at 7:13 PM, Neha Narkhede <n...@confluent.io> wrote:
> Yes, I was clearly confused :-)
>
> On Fri, Apr 24, 2015 at 9:37 AM, Sean Lydon <lydon.s...@gmail.com> wrote:
>
> > Thanks for the responses. Ewen is correct that I am referring to the
> > *new* consumer (org.apache.kafka.clients.consumer.KafkaConsumer).
> >
> > I am extending the consumer to allow my applications more control over
> > committed offsets. I really want to get away from zookeeper (so using
> > the offset storage), and re-balancing is something I haven't really
> > needed to tackle in an automated/seamless way. Either way, I'll hold
> > off going further down this road until there is more interest.
> >
> > @Gwen
> > I set up a single consumer without partition.assignment.strategy or
> > rebalance.callback.class. I was unable to subscribe to just a topic
> > ("Unknown api code 11" on broker), but I could subscribe to a
> > topicpartition. This makes sense as I would need to handle re-balance
> > outside the consumer. Things functioned as expected (well I have an
> > additional minor fix to code from KAFKA-2121), and the only exceptions
> > on broker were due to closing consumers (which I have become
> > accustomed to). My tests are specific to my extended version of the
> > consumer, but they basically do a little writing and reading with
> > different serde classes with application controlled commits (similar
> > to onSuccess and onFailure after each record, but with tolerance for
> > out of order acknowledgements).
> >
> > If you are interested, here is the patch of the hack against trunk.
> >
> > On Thu, Apr 23, 2015 at 10:27 PM, Ewen Cheslack-Postava
> > <e...@confluent.io> wrote:
> > > @Neha I think you're mixing up the 0.8.1/0.8.2 updates and the
> > 0.8.2/0.8.3
> > > that's being discussed here?
> > >
> > > I think the original question was about using the *new* consumer
> > ("clients
> > > consumer") with 0.8.2. Gwen's right, it will use features not even
> > > implemented in the broker in trunk yet, let alone the 0.8.2.
> > >
> > > I don't think the "enable.commit.downgrade" type option, or supporting
> > the
> > > old protocol with the new consumer at all, makes much sense. You'd end
> up
> > > with some weird hybrid of simple and high-level consumers -- you could
> > use
> > > offset storage, but you'd have to manage rebalancing yourself since
> none
> > of
> > > the coordinator support would be there.
> > >
> > >
> > > On Thu, Apr 23, 2015 at 9:22 PM, Neha Narkhede <n...@confluent.io>
> > wrote:
> > >
> > >> My understanding is that ideally the 0.8.3 consumer should work with
> an
> > >> 0.8.2 broker if the offset commit config was set to "zookeeper".
> > >>
> > >> The only thing that might not work is offset commit to Kafka, which
> > makes
> > >> sense since the 0.8.2 broker does not support Kafka based offset
> > >> management.
> > >>
> > >> If we broke all kinds of offset commits, then it seems like a
> > regression,
> > >> no?
> > >>
> > >> On Thu, Apr 23, 2015 at 7:26 PM, Gwen Shapira <gshap...@cloudera.com>
> > >> wrote:
> > >>
> > >> > I didn't think 0.8.3 consumer will ever be able to talk to 0.8.2
> > >> > broker... there are some essential pieces that are missing in 0.8.2
> > >> > (Coordinator, Heartbeat, etc).
> > >> > Maybe I'm missing something. It will be nice if this will work :)
> > >> >
> > >> > Mind sharing what / how you tested? Were there no errors in broker
> > >> > logs after your fix?
> > >> >
> > >> > On Thu, Apr 23, 2015 at 5:37 PM, Sean Lydon <lydon.s...@gmail.com>
> > >> wrote:
> > >> > > Currently the clients consumer (trunk) sends offset commit
> requests
> > of
> > >> > > version 2. The 0.8.2 brokers fail to handle this particular
> request
> > >> > > with a:
> > >> > >
> > >> > > java.lang.AssertionError: assertion failed: Version 2 is invalid
> for
> > >> > > OffsetCommitRequest. Valid versions are 0 or 1.
> > >> > >
> > >> > > I was able to make this work via a forceful downgrade of this
> > >> > > particular request, but I would like some feedback on whether a
> > >> > > "enable.commit.downgrade" configuration would be a tolerable
> method
> > to
> > >> > > allow 0.8.3 consumers to interact with 0.8.2 brokers. I'm also
> > >> > > interested in this even being a goal worth pursuing.
> > >> > >
> > >> > > Thanks,
> > >> > > Sean
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks,
> > >> Neha
> > >>
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Ewen
> >
>
>
>
> --
> Thanks,
> Neha
>
