I don't think we have consumer coordinator in 0.8.2 brokers. So
KafkaConsumer in 0.8.3 will only be able to subscribe to partitions
explicitly. Subscribing to a topic won't work with 0.8.2 brokers.
Jiangjie (Becket) Qin
On Wed, Jul 22, 2015 at 4:26 AM, Stevo Slavić <ssla...@gmail.com> wrote:
> I'm getting "Unknown api code 11" even when both client and server are
> 0.8.3/trunk, when "KafkaConsumer.subscribe(String... topics)" is used.
>
> Bug?
>
> Kind regards,
> Stevo Slavic.
>
> On Fri, Apr 24, 2015 at 7:13 PM, Neha Narkhede <n...@confluent.io> wrote:
>
> > Yes, I was clearly confused :-)
> >
> > On Fri, Apr 24, 2015 at 9:37 AM, Sean Lydon <lydon.s...@gmail.com>
> wrote:
> >
> > > Thanks for the responses. Ewen is correct that I am referring to the
> > > *new* consumer (org.apache.kafka.clients.consumer.KafkaConsumer).
> > >
> > > I am extending the consumer to allow my applications more control over
> > > committed offsets. I really want to get away from zookeeper (so using
> > > the offset storage), and re-balancing is something I haven't really
> > > needed to tackle in an automated/seamless way. Either way, I'll hold
> > > off going further down this road until there is more interest.
> > >
> > > @Gwen
> > > I set up a single consumer without partition.assignment.strategy or
> > > rebalance.callback.class. I was unable to subscribe to just a topic
> > > ("Unknown api code 11" on broker), but I could subscribe to a
> > > topicpartition. This makes sense as I would need to handle re-balance
> > > outside the consumer. Things functioned as expected (well I have an
> > > additional minor fix to code from KAFKA-2121), and the only exceptions
> > > on broker were due to closing consumers (which I have become
> > > accustomed to). My tests are specific to my extended version of the
> > > consumer, but they basically do a little writing and reading with
> > > different serde classes with application controlled commits (similar
> > > to onSuccess and onFailure after each record, but with tolerance for
> > > out of order acknowledgements).
> > >
> > > If you are interested, here is the patch of the hack against trunk.
> > >
> > > On Thu, Apr 23, 2015 at 10:27 PM, Ewen Cheslack-Postava
> > > <e...@confluent.io> wrote:
> > > > @Neha I think you're mixing up the 0.8.1/0.8.2 updates and the
> > > 0.8.2/0.8.3
> > > > that's being discussed here?
> > > >
> > > > I think the original question was about using the *new* consumer
> > > ("clients
> > > > consumer") with 0.8.2. Gwen's right, it will use features not even
> > > > implemented in the broker in trunk yet, let alone the 0.8.2.
> > > >
> > > > I don't think the "enable.commit.downgrade" type option, or
> supporting
> > > the
> > > > old protocol with the new consumer at all, makes much sense. You'd
> end
> > up
> > > > with some weird hybrid of simple and high-level consumers -- you
> could
> > > use
> > > > offset storage, but you'd have to manage rebalancing yourself since
> > none
> > > of
> > > > the coordinator support would be there.
> > > >
> > > >
> > > > On Thu, Apr 23, 2015 at 9:22 PM, Neha Narkhede <n...@confluent.io>
> > > wrote:
> > > >
> > > >> My understanding is that ideally the 0.8.3 consumer should work with
> > an
> > > >> 0.8.2 broker if the offset commit config was set to "zookeeper".
> > > >>
> > > >> The only thing that might not work is offset commit to Kafka, which
> > > makes
> > > >> sense since the 0.8.2 broker does not support Kafka based offset
> > > >> management.
> > > >>
> > > >> If we broke all kinds of offset commits, then it seems like a
> > > regression,
> > > >> no?
> > > >>
> > > >> On Thu, Apr 23, 2015 at 7:26 PM, Gwen Shapira <
> gshap...@cloudera.com>
> > > >> wrote:
> > > >>
> > > >> > I didn't think 0.8.3 consumer will ever be able to talk to 0.8.2
> > > >> > broker... there are some essential pieces that are missing in
> 0.8.2
> > > >> > (Coordinator, Heartbeat, etc).
> > > >> > Maybe I'm missing something. It will be nice if this will work :)
> > > >> >
> > > >> > Mind sharing what / how you tested? Were there no errors in broker
> > > >> > logs after your fix?
> > > >> >
> > > >> > On Thu, Apr 23, 2015 at 5:37 PM, Sean Lydon <lydon.s...@gmail.com
> >
> > > >> wrote:
> > > >> > > Currently the clients consumer (trunk) sends offset commit
> > requests
> > > of
> > > >> > > version 2. The 0.8.2 brokers fail to handle this particular
> > request
> > > >> > > with a:
> > > >> > >
> > > >> > > java.lang.AssertionError: assertion failed: Version 2 is invalid
> > for
> > > >> > > OffsetCommitRequest. Valid versions are 0 or 1.
> > > >> > >
> > > >> > > I was able to make this work via a forceful downgrade of this
> > > >> > > particular request, but I would like some feedback on whether a
> > > >> > > "enable.commit.downgrade" configuration would be a tolerable
> > method
> > > to
> > > >> > > allow 0.8.3 consumers to interact with 0.8.2 brokers. I'm also
> > > >> > > interested in this even being a goal worth pursuing.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Sean
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Thanks,
> > > >> Neha
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Ewen
> > >
> >
> >
> >
> > --
> > Thanks,
> > Neha
> >
>
