We have been using the (new) Java consumer API in 0.9.0.1 for a while
now. We have some well known issues with it - like heart beats being
part of the same thread causing the consumer to sometimes be considered
dead. I understand that this has been fixed in 0.10.0.1 but we haven't
yet had a chance to migrate to it. We plan to do that in the next month
or so.
Personally, I would be OK if the beta label is removed from it if the
dev team is sure the API isn't going to change. I don't know if that's
true or not post 0.10.0.1. For me the major thing that I think needs to
be addressed is these JIRAs which actually expose some API
implementation level issues. Not sure if solving those issues will
involve changes to API itself:
https://issues.apache.org/jira/browse/KAFKA-1894
https://issues.apache.org/jira/browse/KAFKA-3540
https://issues.apache.org/jira/browse/KAFKA-3539
If solving issues like these will not involve changes to the API, I
think it's safe to move it out of beta label.
-Jaikiran
On Tuesday 30 August 2016 05:09 PM, Ismael Juma wrote:
Thanks for the feedback everyone. Since Harsha said that he is OK either
way and everyone else is in favour, I think we should go ahead with this.
Since we committed to API stability for the new Java consumer in 0.10.0.0
via KIP-45, this is simply a documentation change and I don't think we need
an official vote thread (we didn't have one for the equivalent producer
change).
Ismael
On Mon, Aug 29, 2016 at 7:37 PM, Jay Kreps <j...@confluent.io> wrote:
+1 I talk to a lot of kafka users, and I would say > 75% of people doing
new things are on the new consumer despite our warnings :-)
-Jay
On Thu, Aug 25, 2016 at 2:05 PM, Jason Gustafson <ja...@confluent.io>
wrote:
I'm +1 also. I feel a lot more confident about this with all of the
system
testing we now have in place (including the tests covering Streams and
Connect).
-Jason
On Thu, Aug 25, 2016 at 9:57 AM, Gwen Shapira <g...@confluent.io> wrote:
Makes sense :)
On Thu, Aug 25, 2016 at 9:40 AM, Neha Narkhede <n...@confluent.io>
wrote:
Yeah, I'm supportive of this.
On Thu, Aug 25, 2016 at 9:26 AM Ismael Juma <ism...@juma.me.uk>
wrote:
Hi Gwen,
We have a few recent stories of people using Connect and Streams in
production. That means the new Java Consumer too. :)
Ismael
On Thu, Aug 25, 2016 at 5:09 PM, Gwen Shapira <g...@confluent.io>
wrote:
Originally, we suggested keeping the beta label until we know
someone
successfully uses the new consumer in production.
We can consider the recent KIPs enough, but IMO it will be better
if
someone with production deployment hanging out on our mailing list
will confirm good experience with the new consumer.
Gwen
On Wed, Aug 24, 2016 at 8:45 PM, Ismael Juma <ism...@juma.me.uk>
wrote:
Hi all,
We currently say the following in our documentation:
"As of the 0.9.0 release we have added a new Java consumer to
replace
our
existing high-level ZooKeeper-based consumer and low-level
consumer
APIs.
This client is considered beta quality."[1]
Since then, Jason and the community have done a lot of work to
improve
it
(including KIP-41 and KIP-62), we declared it API stable in
0.10.0.0
and
it's the only option for those that need security support. Yes,
it
still
has bugs, but so does the old consumer and all development is
currently
focused on the new consumer.
As such, I propose we remove the beta label for the next release
and
switch
our tools to use the new consumer by default unless the
zookeeper
command-line option is present (for compatibility). This is
similar
to
what
we did it for the new producer in 0.9.0.0, but backwards
compatible.
Thoughts?
Ismael
[1] http://kafka.apache.org/documentation.html#consumerapi
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
--
Thanks,
Neha
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
