Re: [VOTE] KIP-371: Add a configuration to build custom SSL principal name

[Manikumar]

Hi All,

The vote has passed with 3 binding votes (Harsha, Rajini, Jun) and 2
non-binding votes (Priyank, Satish).

Thanks everyone for the votes.

Thanks,
Manikumar

On Wed, Oct 10, 2018 at 1:36 AM Jun Rao  wrote:

> Hi, Mani,
>
> Thanks for the KIP. +1 from me.
>
> Jun
>
> On Wed, Sep 19, 2018 at 5:19 AM, Manikumar 
> wrote:
>
> > Hi All,
> >
> > I would like to start voting on KIP-371, which adds a configuration
> option
> > for building custom SSL principal names.
> >
> > KIP:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 371%3A+Add+a+configuration+to+build+custom+SSL+principal+name
> >
> > Discussion Thread:
> > https://lists.apache.org/thread.html/e346f5e3e3dd1feb863594e40eac1e
> > d54138613a667f319b99344710@%3Cdev.kafka.apache.org%3E
> >
> > Thanks,
> > Manikumar
> >
>

Re: [ANNOUNCE] New Committer: Manikumar Reddy

[Manikumar]

Thanks everyone!

It has been a great learning experience working Apache Kafka community.
I am thankful to the committers/community members for all the support and
help.

Thanks
Manikumar

On Fri, Oct 12, 2018 at 1:55 PM Edoardo Comar  wrote:

> Well done Manikumar !
> --
>
> Edoardo Comar
>
> IBM Event Streams
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
>
> From:   "Matthias J. Sax" 
> To: dev 
> Cc: users 
> Date:   11/10/2018 23:41
> Subject:Re: [ANNOUNCE] New Committer: Manikumar Reddy
>
>
>
> Congrats!
>
>
> On 10/11/18 2:31 PM, Yishun Guan wrote:
> > Congrats Manikumar!
> > On Thu, Oct 11, 2018 at 1:20 PM Sönke Liebau
> >  wrote:
> >>
> >> Great news, congratulations Manikumar!!
> >>
> >> On Thu, Oct 11, 2018 at 9:08 PM Vahid Hashemian
> 
> >> wrote:
> >>
> >>> Congrats Manikumar!
> >>>
> >>> On Thu, Oct 11, 2018 at 11:49 AM Ryanne Dolan 
> >>> wrote:
> >>>
> >>>> Bravo!
> >>>>
> >>>> On Thu, Oct 11, 2018 at 1:48 PM Ismael Juma 
> wrote:
> >>>>
> >>>>> Congratulations Manikumar! Thanks for your continued contributions.
> >>>>>
> >>>>> Ismael
> >>>>>
> >>>>> On Thu, Oct 11, 2018 at 10:39 AM Jason Gustafson
> 
> >>>>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> The PMC for Apache Kafka has invited Manikumar Reddy as a committer
> >>> and
> >>>>> we
> >>>>>> are
> >>>>>> pleased to announce that he has accepted!
> >>>>>>
> >>>>>> Manikumar has contributed 134 commits including significant work to
> >>> add
> >>>>>> support for delegation tokens in Kafka:
> >>>>>>
> >>>>>> KIP-48:
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
>
> >>>>>> KIP-249
> >>>>>> <
> >>>>>
> >>>>
> >>>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+KafkaKIP-249
>
> >>>>>>
> >>>>>> :
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient
>
> >>>>>>
> >>>>>> He has broad experience working with many of the core components in
> >>>> Kafka
> >>>>>> and he has reviewed over 80 PRs. He has also made huge progress
> >>>>> addressing
> >>>>>> some of our technical debt.
> >>>>>>
> >>>>>> We appreciate the contributions and we are looking forward to more.
> >>>>>> Congrats Manikumar!
> >>>>>>
> >>>>>> Jason, on behalf of the Apache Kafka PMC
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >>
> >> --
> >> Sönke Liebau
> >> Partner
> >> Tel. +49 179 7940878
> >> OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
>
> [attachment "signature.asc" deleted by Edoardo Comar/UK/IBM]
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

[DISCUSS] 2.0.1 bug fix release

[Manikumar]

Hi all,

I would like to volunteer to be the release manager for 2.0.1 bug fix
release.
2.0 was released July 30, 2018 and 44 issues are fixed so far.

Please find all the resolved tickets here:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%202.0.1

Please find the Release plan:
https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1

If you have any JIRA in progress and would like to include it in this
release, please discuss with your reviewer.
There is currently only one blocking issue (
https://issues.apache.org/jira/browse/KAFKA-7464).

Next week, Once the blocking issue gets addressed,  I plan to create the
first RC for 2.0.1 release.

Thanks,
Manikumar

[VOTE] 2.0.1 RC0

[Manikumar]

Hello Kafka users, developers and client-developers,

This is the first candidate for release of Apache Kafka 2.0.1.

This is a bug fix release closing 49 tickets:
https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1

Release notes for the 2.0.1 release:
http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html

*** Please download, test and vote by  Tuesday, October 30, end of day

Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~manikumar/kafka-2.0.1-rc0/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/

* Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
https://github.com/apache/kafka/releases/tag/2.0.1-rc0

* Documentation:
http://kafka.apache.org/20/documentation.html

* Protocol:
http://kafka.apache.org/20/protocol.html

* Successful Jenkins builds for the 2.0 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-2.0-jdk8/177/

/**

Thanks,
Manikumar

Re: Frequent under-replicated partitions

[Manikumar]

We can monitor below replica related metrics. Try  tuning "
replica.lag.time.max.ms" , "replica.fetch.max.bytes" .
look for logs starting with "Shrinking ISR for partition ...".

kafka.server:type=ReplicaManager,name=IsrShrinksPerSec
kafka.server:type=ReplicaManager,name=IsrExpandsPerSec
kafka.server:type=ReplicaFetcherManager,name=MaxLag,clientId=Replica
kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=([-.\w]+),topic=([-.\w]+),partition=([0-9]+)

On Thu, Oct 25, 2018 at 7:18 PM Suman B N  wrote:

> Still looking for some response here. Pls assist.
>
> On Sat, Oct 20, 2018 at 12:43 AM Suman B N  wrote:
>
> > Rate of ingestion is not 150-200rps. Its 150k-200k rps.
> >
> > On Fri, Oct 19, 2018 at 11:12 PM Suman B N 
> wrote:
> >
> >> Team,
> >> We have been observing some partitions being under-replicated. Broker
> >> version 0.10.2.1. Below actions were carried out but in vain:
> >>
> >>- Tried restarting nodes.
> >>- Tried increasing replica fetcher threads. Recommend ideal replica
> >>fetcher threads for a 20 node cluster with 150-200rps spread across
> 1000
> >>topics and 3000 partitions.
> >>- Tried increasing network threads. (I think this doesn't have any
> >>effect but still wanted to try). Recommend ideal network threads for
> a 20
> >>node cluster with 150-200rps spread across 1000 topics and 3000
> partitions.
> >>
> >> Logs look very clean. No exceptions. I don't have much idea on how
> >> replica fetcher threads and logs can be debugged. So asking for help
> here.
> >> Any help or leads would be appreciated.
> >>
> >> --
> >> *Suman*
> >> *OlaCabs*
> >>
> >
> >
> > --
> > *Suman*
> > *OlaCabs*
> >
>
>
> --
> *Suman*
> *OlaCabs*
>

Re: [VOTE] 2.1.0 RC0

[Manikumar]

minor observation: config sections are empty in the documentation page.
http://kafka.apache.org/21/documentation.html#producerconfigs

On Wed, Oct 24, 2018 at 10:49 PM Ted Yu  wrote:

> +1
>
> InternalTopicIntegrationTest failed during test suite run but passed with
> rerun.
>
> On Wed, Oct 24, 2018 at 3:48 AM Andras Beni  .invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > Verified signatures and checksums of release artifacts
> > Performed quickstart steps on rc artifacts (both scala 2.11 and 2.12) and
> > one built from tag 2.1.0-rc0
> >
> > Andras
> >
> > On Wed, Oct 24, 2018 at 10:17 AM Dong Lin  wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for feature release of Apache Kafka 2.1.0.
> > >
> > > This is a major version release of Apache Kafka. It includes 28 new
> KIPs
> > > and
> > >
> > > critical bug fixes. Please see the Kafka 2.1.0 release plan for more
> > > details:
> > >
> > > *
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
> > > <
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044
> > > >
> > >
> > > Here are a few notable highlights:
> > >
> > > - Java 11 support
> > > - Support for Zstandard, which achieves compression comparable to gzip
> > with
> > > higher compression and especially decompression speeds(KIP-110)
> > > - Avoid expiring committed offsets for active consumer group (KIP-211)
> > > - Provide Intuitive User Timeouts in The Producer (KIP-91)
> > > - Kafka's replication protocol now supports improved fencing of
> zombies.
> > > Previously, under certain rare conditions, if a broker became
> partitioned
> > > from Zookeeper but not the rest of the cluster, then the logs of
> > replicated
> > > partitions could diverge and cause data loss in the worst case
> (KIP-320)
> > > - Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353,
> KIP-356)
> > > - Admin script and admin client API improvements to simplify admin
> > > operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
> > > - DNS handling improvements (KIP-235, KIP-302)
> > >
> > > Release notes for the 2.1.0 release:
> > > http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote ***
> > >
> > > * Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~lindong/kafka-2.1.0-rc0/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~lindong/kafka-2.1.0-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 2.1 branch) is the 2.1.0-rc0 tag:
> > > https://github.com/apache/kafka/tree/2.1.0-rc0
> > >
> > > * Documentation:
> > > *http://kafka.apache.org/21/documentation.html*
> > > 
> > >
> > > * Protocol:
> > > http://kafka.apache.org/21/protocol.html
> > >
> > > * Successful Jenkins builds for the 2.1 branch:
> > > Unit/integration tests: *
> > https://builds.apache.org/job/kafka-2.1-jdk8/38/
> > > *
> > >
> > > Please test and verify the release artifacts and submit a vote for this
> > RC,
> > > or report any issues so we can fix them and get a new RC out ASAP.
> > Although
> > > this release vote requires PMC votes to pass, testing, votes, and bug
> > > reports are valuable and appreciated from everyone.
> > >
> > > Cheers,
> > > Dong
> > >
> >
>

Re: [VOTE] 2.0.1 RC0

[Manikumar]

Hi Eno,

This looks like an existing issue occuring only on source artifacts.  We
are able to generate aggregate docs on cloned repo.
I am getting similar error on previous release and 2.1.0 RC0 src artifacts.
maybe related to gradle task ordering.
I will look into it and try to fix it on trunk.

Similar issue reported here:
https://jira.apache.org/jira/browse/KAFKA-6500

Thanks,


On Mon, Oct 29, 2018 at 5:28 PM Eno Thereska  wrote:

> Thanks. Tested basic building and running of unit and integration tests.
> They work.
> Tested docs. The following fails. Is it a known issue?
>
> "
> ./gradlew aggregatedJavadoc
> with info:
> > Configure project :
> Building project 'core' with Scala version 2.11.12
> Building project 'streams-scala' with Scala version 2.11.12
>
> > Task :aggregatedJavadoc FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> A problem was found with the configuration of task ':aggregatedJavadoc'.
> > No value has been specified for property 'outputDirectory'.
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full
> insights.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 3s
> "
> Eno
>
> On Fri, Oct 26, 2018 at 3:29 AM Manikumar 
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 2.0.1.
> >
> > This is a bug fix release closing 49 tickets:
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> >
> > Release notes for the 2.0.1 release:
> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by  Tuesday, October 30, end of day
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> >
> > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> >
> > * Documentation:
> > http://kafka.apache.org/20/documentation.html
> >
> > * Protocol:
> > http://kafka.apache.org/20/protocol.html
> >
> > * Successful Jenkins builds for the 2.0 branch:
> > Unit/integration tests:
> https://builds.apache.org/job/kafka-2.0-jdk8/177/
> >
> > /**
> >
> > Thanks,
> > Manikumar
> >
>

Re: 請求加入 Jira & Confluence 的權限

[Manikumar]

Hi, Navya

Thanks for your interest. Gave you permission to both jira and wiki.

Thanks,

On Wed, Oct 31, 2018 at 1:57 PM 張育琦  wrote:

> 請求加入 Jira & Confluence 的權限  ：
>
> Jira id ：Navya Snow
> cwiki id   ：NavyaSnow
>
> 謝謝！
>
>
> Best regards,
> 亦思科技is-land
> Navya Chang 張育琦
> 產品應用部-管理師
> 104台北市中山區復興北路288號9樓
> Office:+886-2-25178682#13
> 300新竹科學園區展業二路4號3樓
> TEL:+886-3-5630345
> www.is-land.com.tw
> 
>

Re: [VOTE] 2.0.1 RC0

[Manikumar]

We were waiting for the system test results. There were few failures:
KAFKA-7579,  KAFKA-7559, KAFKA-7561
they are not blockers for 2.0.1 release. We need more votes from
PMC/committers :)

Thanks Stanislav! for the system test results.

Thanks,
Manikumar

On Thu, Nov 1, 2018 at 10:20 PM Eno Thereska  wrote:

> Anything else holding this up?
>
> Thanks
> Eno
>
> On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz  wrote:
>
> > +1 (non-binding) ... I used the staged binaries and run tests with
> > different clients.
> >
> > On Fri, Oct 26, 2018 at 4:29 AM Manikumar 
> > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for release of Apache Kafka 2.0.1.
> > >
> > > This is a bug fix release closing 49 tickets:
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> > >
> > > Release notes for the 2.0.1 release:
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by  Tuesday, October 30, end of day
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> > >
> > > * Documentation:
> > > http://kafka.apache.org/20/documentation.html
> > >
> > > * Protocol:
> > > http://kafka.apache.org/20/protocol.html
> > >
> > > * Successful Jenkins builds for the 2.0 branch:
> > > Unit/integration tests:
> > https://builds.apache.org/job/kafka-2.0-jdk8/177/
> > >
> > > /**
> > >
> > > Thanks,
> > > Manikumar
> > >
> >
>

Re: [VOTE] 2.0.1 RC0

[Manikumar]

KAFKA-7581, KAFKA-7579 are not blockers for 2.0.1 release. KAFKA-7579 got
fixed on 2.0 branch.
This can be part of 2.0.1, if we are going with another RC.

We need couple of more PMC votes to pass this vote thread.

On Wed, Nov 7, 2018 at 4:43 PM Eno Thereska  wrote:

> Two JIRAs are still marked as blockers, although it's not clear to me if
> they really are. Any update?
> Thanks
> Eno
>
> On Thu, Nov 1, 2018 at 5:10 PM Manikumar 
> wrote:
>
> > We were waiting for the system test results. There were few failures:
> > KAFKA-7579,  KAFKA-7559, KAFKA-7561
> > they are not blockers for 2.0.1 release. We need more votes from
> > PMC/committers :)
> >
> > Thanks Stanislav! for the system test results.
> >
> > Thanks,
> > Manikumar
> >
> > On Thu, Nov 1, 2018 at 10:20 PM Eno Thereska 
> > wrote:
> >
> > > Anything else holding this up?
> > >
> > > Thanks
> > > Eno
> > >
> > > On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz  wrote:
> > >
> > > > +1 (non-binding) ... I used the staged binaries and run tests with
> > > > different clients.
> > > >
> > > > On Fri, Oct 26, 2018 at 4:29 AM Manikumar  >
> > > > wrote:
> > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the first candidate for release of Apache Kafka 2.0.1.
> > > > >
> > > > > This is a bug fix release closing 49 tickets:
> > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> > > > >
> > > > > Release notes for the 2.0.1 release:
> > > > >
> http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download, test and vote by  Tuesday, October 30, end of
> > day
> > > > >
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > http://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > > https://repository.apache.org/content/groups/staging/
> > > > >
> > > > > * Javadoc:
> > > > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > > > > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> > > > >
> > > > > * Documentation:
> > > > > http://kafka.apache.org/20/documentation.html
> > > > >
> > > > > * Protocol:
> > > > > http://kafka.apache.org/20/protocol.html
> > > > >
> > > > > * Successful Jenkins builds for the 2.0 branch:
> > > > > Unit/integration tests:
> > > > https://builds.apache.org/job/kafka-2.0-jdk8/177/
> > > > >
> > > > > /**
> > > > >
> > > > > Thanks,
> > > > > Manikumar
> > > > >
> > > >
> > >
> >
>

Re: [kafka-clients] Re: [VOTE] 2.0.1 RC0

[Manikumar]

+1 (binding)
Built from src and ran tests, verified the artifacts, signatures, Ran core
quick start.

Thanks,
Manikumar

On Thu, Nov 8, 2018 at 12:15 AM Rajini Sivaram 
wrote:

> +1 (binding)
>
> Checked source build and unit tests. Ran quickstart with source and binary.
>
> Thank you for managing the release, Manikumar!
>
> Regards,
>
> Rajini
>
> On Wed, Nov 7, 2018 at 6:18 PM Gwen Shapira  wrote:
>
>> +1 (binding)
>>
>> Checked signatures, build and quickstart.
>>
>> Thank you for managing the release, Mani!
>>
>>
>> On Thu, Oct 25, 2018 at 7:29 PM Manikumar 
>> wrote:
>> >
>> > Hello Kafka users, developers and client-developers,
>> >
>> > This is the first candidate for release of Apache Kafka 2.0.1.
>> >
>> > This is a bug fix release closing 49 tickets:
>> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
>> >
>> > Release notes for the 2.0.1 release:
>> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
>> >
>> > *** Please download, test and vote by  Tuesday, October 30, end of day
>> >
>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > http://kafka.apache.org/KEYS
>> >
>> > * Release artifacts to be voted upon (source and binary):
>> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
>> >
>> > * Maven artifacts to be voted upon:
>> > https://repository.apache.org/content/groups/staging/
>> >
>> > * Javadoc:
>> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
>> >
>> > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
>> > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
>> >
>> > * Documentation:
>> > http://kafka.apache.org/20/documentation.html
>> >
>> > * Protocol:
>> > http://kafka.apache.org/20/protocol.html
>> >
>> > * Successful Jenkins builds for the 2.0 branch:
>> > Unit/integration tests:
>> https://builds.apache.org/job/kafka-2.0-jdk8/177/
>> >
>> > /**
>> >
>> > Thanks,
>> > Manikumar
>>
>>
>>
>> --
>> Gwen Shapira
>> Product Manager | Confluent
>> 650.450.2760 | @gwenshap
>> Follow us: Twitter | blog
>>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAOJcB38LDf1Fgkr8HMG2b2%2B1tA5usQ%3DeeRv0Vjcj63Tq2zoYZQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kafka-clients/CAOJcB38LDf1Fgkr8HMG2b2%2B1tA5usQ%3DeeRv0Vjcj63Tq2zoYZQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

Re: [kafka-clients] Re: [VOTE] 2.0.1 RC0

[Manikumar]

Yes, I was about to send closing mail.

On Thu, Nov 8, 2018 at 9:39 PM Ismael Juma  wrote:

> Manikumar, looks like you have the necessary votes. :)
>
> Ismael
>
> On Wed, Nov 7, 2018, 10:45 AM Rajini Sivaram  wrote:
>
>> +1 (binding)
>>
>> Checked source build and unit tests. Ran quickstart with source and
>> binary.
>>
>> Thank you for managing the release, Manikumar!
>>
>> Regards,
>>
>> Rajini
>>
>> On Wed, Nov 7, 2018 at 6:18 PM Gwen Shapira  wrote:
>>
>> > +1 (binding)
>> >
>> > Checked signatures, build and quickstart.
>> >
>> > Thank you for managing the release, Mani!
>> >
>> >
>> > On Thu, Oct 25, 2018 at 7:29 PM Manikumar 
>> > wrote:
>> > >
>> > > Hello Kafka users, developers and client-developers,
>> > >
>> > > This is the first candidate for release of Apache Kafka 2.0.1.
>> > >
>> > > This is a bug fix release closing 49 tickets:
>> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
>> > >
>> > > Release notes for the 2.0.1 release:
>> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
>> > >
>> > > *** Please download, test and vote by  Tuesday, October 30, end of day
>> > >
>> > > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > > http://kafka.apache.org/KEYS
>> > >
>> > > * Release artifacts to be voted upon (source and binary):
>> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
>> > >
>> > > * Maven artifacts to be voted upon:
>> > > https://repository.apache.org/content/groups/staging/
>> > >
>> > > * Javadoc:
>> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
>> > >
>> > > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
>> > > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
>> > >
>> > > * Documentation:
>> > > http://kafka.apache.org/20/documentation.html
>> > >
>> > > * Protocol:
>> > > http://kafka.apache.org/20/protocol.html
>> > >
>> > > * Successful Jenkins builds for the 2.0 branch:
>> > > Unit/integration tests:
>> > https://builds.apache.org/job/kafka-2.0-jdk8/177/
>> > >
>> > > /**
>> > >
>> > > Thanks,
>> > > Manikumar
>> >
>> >
>> >
>> > --
>> > Gwen Shapira
>> > Product Manager | Confluent
>> > 650.450.2760 | @gwenshap
>> > Follow us: Twitter | blog
>> >
>>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAD5tkZYww8XbFFwx5Os4pOjiANGz-Lq81xcSGiTXs-G%2BwpG%3Duw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kafka-clients/CAD5tkZYww8XbFFwx5Os4pOjiANGz-Lq81xcSGiTXs-G%2BwpG%3Duw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

[RESULTS] [VOTE] Release Kafka version 2.0.1

[Manikumar]

This vote passes with 12 +1 votes (3 bindings) and no 0 or -1 votes.

+1 votes
PMC Members:
* Jun Rao
* Gwen Shapira
* Rajini Sivaram

Committers:
* Harsha
* Ewen Cheslack-Postava
* Manikumar

Community:
* Magnus Edenhill
* Eno Thereska
* Vahid Hashemian
* Jakub Scholz
* Satish Duggana
* Mickael Maison

0 votes
* No votes

-1 votes
* No votes

Vote thread:
https://markmail.org/thread/3gy4a6fmjv2xuoou

Thank you all for the votes.
I'll continue with the release process and the release announcement will
follow in the next few days.

Manikumar

[ANNOUNCE] Apache Kafka 2.0.1

[Manikumar]

The Apache Kafka community is pleased to announce the release for Apache
Kafka 2.0.1

This is a bug fix release and it includes fixes and improvements from 51
JIRAs.

All of the changes in this release can be found in the release notes:
https://www.apache.org/dist/kafka/2.0.1/RELEASE_NOTES.html


You can download the source and binary release (Scala 2.11 and Scala 2.12)
from:
https://kafka.apache.org/downloads#2.0.1

---


Apache Kafka is a distributed streaming platform with four core APIs:


** The Producer API allows an application to publish a stream records to
one or more Kafka topics.

** The Consumer API allows an application to subscribe to one or more
topics and process the stream of records produced to them.

** The Streams API allows an application to act as a stream processor,
consuming an input stream from one or more topics and producing an
output stream to one or more output topics, effectively transforming the
input streams to output streams.

** The Connector API allows building and running reusable producers or
consumers that connect Kafka topics to existing applications or data
systems. For example, a connector to a relational database might
capture every change to a table.


With these APIs, Kafka can be used for two broad classes of application:

** Building real-time streaming data pipelines that reliably get data
between systems or applications.

** Building real-time streaming applications that transform or react
to the streams of data.


Apache Kafka is in use at large and small companies worldwide, including
Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
Target, The New York Times, Uber, Yelp, and Zalando, among others.

A big thank you for the following 40 contributors to this release!

Amit Sela, Anna Povzner, Arjun Satish, Bibin Sebastian, Bill Bejeck, Bob
Barrett, Bridger Howell,
Colin Hicks, Dhruvil Shah, Dong Lin, Flavien Raynaud, Guozhang Wang, huxi,
huxihx, Ismael Juma,
Jason Gustafson, Joan Goyeau, John Roesler, Jon Lee, Kamal Chandraprakash,
Kevin Lafferty,
Konstantine Karantasis, lambdaliu, Lincong Li, Lucas Wang, Maciej Bryński,
Manikumar Reddy,
Matthias J. Sax, Max Zheng, Michal Dziemianko, Michał Borowiecki,
radai-rosenblatt, Rajini Sivaram,
Randall Hauch, Robert Yokota, Simon Clark, Stanislav Kozlovski, Sébastien
Launay, tedyu,
Zhanxiang (Patrick) Huang

We welcome your help and feedback. For more information on how to
report problems, and to get involved, visit the project website at
https://kafka.apache.org/

Thank you!

Regards,
Manikumar

Re: [VOTE] 2.1.0 RC1

[Manikumar]

+1
I've verified signature, docs, built from sources, ran unit tests  (no test
failures observed)
ran quick start.

Thanks for running the release!.

Thanks,
Manikumar

On Sat, Nov 17, 2018 at 1:10 PM Dong Lin  wrote:

> Hey Eno, Vahid and everyone,
>
> Thanks for reporting the test error!
> https://builds.apache.org/job/kafka-2.1-jdk8/ shows the list of recent
> unit
> test runs. 7 out of 10 recent runs have passed all tests. Each of the three
> runs shows one unique flaky test failure.
>
> I have opened umbrella JIRA
> https://issues.apache.org/jira/browse/KAFKA-7645
> to track these flake test. There are currently 7 flaky tests reported in
> either https://builds.apache.org/job/kafka-2.1-jdk8/  or the voting
> threads. Among these 7 flaky tests, 3 tests failed due to issue in the test
> logic, 3 tests are related SSL with similar failure in 2.0 branch which has
> been running well. So these 6 tests should not be blocking issue for 2.1.0
> release.
>
> Regarding the other test failure for
>
> LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic
> (KAFKA-7647 <https://issues.apache.org/jira/browse/KAFKA-7647>). I did due
> diligence to understand the failure but could not find any bug. Since this
> test failure happens rarely and I could not find any issue by looking at
> the stacktrace, I am again inclined not to consider this as blocking issue.
> We can discuss more if there is different opinion in the mailing list.
>
> Thanks,
> Dong
>
>
> On Thu, Nov 15, 2018 at 10:28 PM Guozhang Wang  wrote:
>
> > +1 (binding).
> >
> > I've verified the signature, and ran quickstart / unit test with scala
> 2.12
> > binary.
> >
> > On my local laptop the unit test did not fail though on Jenkins it seems
> > indeed flaky.
> >
> > Guozhang
> >
> > On Fri, Nov 9, 2018 at 3:33 PM Dong Lin  wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the second candidate for feature release of Apache Kafka 2.1.0.
> > >
> > > This is a major version release of Apache Kafka. It includes 28 new
> KIPs
> > > and
> > >
> > > critical bug fixes. Please see the Kafka 2.1.0 release plan for more
> > > details:
> > >
> > > *
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
> > > <
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044
> > > >
> > >
> > > Here are a few notable highlights:
> > >
> > > - Java 11 support
> > > - Support for Zstandard, which achieves compression comparable to gzip
> > with
> > > higher compression and especially decompression speeds(KIP-110)
> > > - Avoid expiring committed offsets for active consumer group (KIP-211)
> > > - Provide Intuitive User Timeouts in The Producer (KIP-91)
> > > - Kafka's replication protocol now supports improved fencing of
> zombies.
> > > Previously, under certain rare conditions, if a broker became
> partitioned
> > > from Zookeeper but not the rest of the cluster, then the logs of
> > replicated
> > > partitions could diverge and cause data loss in the worst case
> (KIP-320)
> > > - Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353,
> KIP-356)
> > > - Admin script and admin client API improvements to simplify admin
> > > operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
> > > - DNS handling improvements (KIP-235, KIP-302)
> > >
> > > Release notes for the 2.1.0 release:
> > > http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by Thursday, Nov 15, 12 pm PT ***
> > >
> > > * Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~lindong/kafka-2.1.0-rc1/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~lindong/kafka-2.1.0-rc1/javadoc/
> > >
> > > * Tag to be voted upon (off 2.1 branch) is the 2.1.0-rc1 tag:
> > > https://github.com/apache/kafka/tree/2.1.0-rc1
> > >
> > > * Documentation:
> > > *http://kafka.apache.org/21/documentation.html*
> > > <http://kafka.apache.org/21/documentation.html>
> > >
> > > * Protocol:
> > > http://kafka.apache.org/21/protocol.html
> > >
> > > * Successful Jenkins builds for the 2.1 branch:
> > > Unit/integration tests: *
> > https://builds.apache.org/job/kafka-2.1-jdk8/50/
> > > <https://builds.apache.org/job/kafka-2.1-jdk8/50/>*
> > >
> > > Please test and verify the release artifacts and submit a vote for this
> > RC,
> > > or report any issues so we can fix them and get a new RC out ASAP.
> > Although
> > > this release vote requires PMC votes to pass, testing, votes, and bug
> > > reports are valuable and appreciated from everyone.
> > >
> > > Cheers,
> > > Dong
> > >
> >
> >
> > --
> > -- Guozhang
> >
>

Re: Need to subscribe to mail list and get access to contribute to jira tickets

[Manikumar]

Hi,

I have given JIRA permissions for "kaushik srinivas" JIRA username.

On Tue, Nov 20, 2018 at 3:29 PM KAUSHIK SRINIVAS <
kaushiksrinivas...@gmail.com> wrote:

> Hi,
>
> Need subscription to kafka mailing list.
>
> Also need to assign jira tickets to myself. Have worked on few pull
> requests and need to submit the code.
>
> Need support in getting the required permissions to assign the kafka jira
> ticket to myself.
>
> Thanks & Regards,
> kaushik
>

Re: [VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

[Manikumar]

+1 (binding)
Thanks for the KIP.

Can you update the KIP stating that this option is not supported with
"--zookeeper" option.

On Mon, Nov 26, 2018 at 11:15 PM Mickael Maison 
wrote:

> +1 (non-binding)
> Thanks for the KIP!
> On Mon, Nov 26, 2018 at 4:32 PM Kevin Lu  wrote:
> >
> > Hi All,
> >
> > I'm bumping this thread as it has been a couple weeks with no activity.
> >
> > The proposed changes in this KIP are minor, but are extremely helpful for
> > operators to immediately identify partitions under min ISR. Please take a
> > couple minutes to review and provide a vote.
> >
> > Thanks~
> >
> > Regards,
> > Kevin
> >
> > On Thu, Nov 8, 2018 at 1:07 AM Kevin Lu  wrote:
> >
> > > Hi All,
> > >
> > > I'm starting the vote thread for KIP-351: Add --under-min-isr option to
> > > describe topics command.
> > >
> > > KIP:
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-351%3A+Add+--under-min-isr+option+to+describe+topics+command
> > >
> > > Discussion thread:
> > >
> https://lists.apache.org/thread.html/90d1652ebc03a7be4100dd101b92a7dcefe63d144856c5f6c132381b@%3Cdev.kafka.apache.org%3E
> > >
> > > Thanks!
> > >
> > > Regards,
> > > Kevin
> > >
> > >
>

Re: Requesting permission to create KIP

[Manikumar]

Thanks for your interest. I have given you the permissions.

On Thu, Nov 29, 2018 at 8:24 PM Brandon Kirchner 
wrote:

> Hello,
>
> Could you please grant me (brandon.kirchner) permission to create a KIP?
>
> Thanks!
> Brandon K.
>

Re: I'd like permissions to publish a KIP

[Manikumar]

Thanks for your interest. I have given you the wiki permissions.

On Thu, Nov 29, 2018 at 10:03 PM Noa Resare  wrote:

> Greetings friends,
>
> I have a proposal for a tiny change that would alter configuration
> semantics slightly, so it seems like something that should be handled
> through the KIP process. Could I get permission? My account on cwiki is
> ‘noa’
>
> cheers
> noa

Re: kafka Build fail - jdk 8 - #18041

[Manikumar]

HI Srinivas,

They are mostly transient test failures. We can rerun the test to confirm
the failures.
You can re-trigger the jenkins tests by commenting  "retest this please" in
PR comments.


On Fri, Nov 30, 2018 at 12:04 PM Srinivas, Kaushik (Nokia - IN/Bangalore) <
kaushik.srini...@nokia.com> wrote:

> Hi All,
>
> Facing below build failure.
>
>
> 20:19:43
>
> 20:19:43 > Task :streams:test-utils:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-0100:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-0101:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-0102:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-0110:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-10:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-11:integrationTest
>
> 20:19:43 > Task :streams:upgrade-system-tests-20:integrationTest
>
> 20:19:43
>
> 20:19:43 FAILURE: Build failed with an exception.
>
> 20:19:43
>
> 20:19:43 * What went wrong:
>
> 20:19:43 Execution failed for task ':core:integrationTest'.
>
> 20:19:43 > There were failing tests. See the report at:
> file:///home/jenkins/jenkins-slave/workspace/kafka-pr-jdk8-scala2.11@2
> /core/build/reports/tests/integrationTest/index.html /core/build/reports/tests/integrationTest/index.html>
>
> Build failure report :
> https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/18041/testReport/
>
> Need support.
>
> -kaushik
>

Re: Request for wiki permission to create KIP

[Manikumar]

Hi Kamal,

I just gave you wiki permissions;

Thanks,

On Sat, Dec 1, 2018 at 11:29 AM Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:

> Hi,
>
> I would like to propose a KIP for KAFKA-7499
> . Please grant me
> permission to create KIP.
>
> ID: ckamal
>

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

[Manikumar]

Bump up! to get some attention.

BTW, recently Apache Spark added for Kafka delegation token support.
https://issues.apache.org/jira/browse/SPARK-25501

On Tue, Sep 25, 2018 at 9:56 PM Manikumar  wrote:

> Hi all,
>
> I have created a KIP that proposes to allow users to create delegation
> tokens for other users.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-373%3A+Allow+users+to+create+delegation+tokens+for+other+users
>
> Please take a look when you get a chance.
>
> Thanks,
> Manikumar
>

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

[Manikumar]

Bump up! to get some attention.

BTW, recently Apache Spark added  support for Kafka delegation token.
https://issues.apache.org/jira/browse/SPARK-25501

On Fri, Dec 7, 2018 at 5:27 PM Manikumar  wrote:

> Bump up! to get some attention.
>
> BTW, recently Apache Spark added for Kafka delegation token support.
> https://issues.apache.org/jira/browse/SPARK-25501
>
> On Tue, Sep 25, 2018 at 9:56 PM Manikumar 
> wrote:
>
>> Hi all,
>>
>> I have created a KIP that proposes to allow users to create delegation
>> tokens for other users.
>>
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-373%3A+Allow+users+to+create+delegation+tokens+for+other+users
>>
>> Please take a look when you get a chance.
>>
>> Thanks,
>> Manikumar
>>
>

Re: [DISCUSS] KIP-395: Encypt-then-MAC Delegation token metadata

[Manikumar]

Hi,

Thanks for the KIP.

Currently, master/secret key is stored as plain text in server.properties
config file.
Using master secret key as shared secret is again a security risk. We have
raised KAFKA-7694
to implement a ZooKeeper based master/secret key management to automate
secret key rotation.

As you mentioned in the alternatives sections, it is good to have pluggable
mechanism for
token storage and master key generation. We can implement pluggable
interfaces for token storage
and master key generation as part of KAFKA-7694. This will provide us out
of the box implementation
using ZooKeeper and pluggable interfaces for custom implementations.

What do you think?

Thanks,
Manikumar

On Sat, Dec 1, 2018 at 9:37 PM Attila Sasvári  wrote:

> Hi All,
>
> I have a proposal to allow Kafka brokers to encrypt sensitive metadata
> information about delegation tokens.
>
> As of now, delegation token metadata is stored in an unencrypted format in
> Zookeeper. Having the possibility to encrypt-then-MAC token information
> would be beneficial in Kafka installations where Zookeeper is not on a
> private network.
>
> Please take a look at
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-395%3A+Encypt-then-MAC+Delegation+token+metadata
> and let me know what you think.
>
> - Attila
>

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

[Manikumar]

Hi Harsha,

Thanks for the review.

With this KIP a designated superuser can create tokens without requiring
individual user credentials.
Any client can authenticate brokers using the created tokens. We may not
call this as impersonation,
since the clients API calls are executing on their own authenticated
connections.

Thanks,
Manikumar

On Fri, Dec 7, 2018 at 11:56 PM Harsha  wrote:

> Hi Mani,
>  Overall KIP looks good to me. Can we call this Impersonation
> support, which is what the KIP is doing?
> Also instead of using super.uses as the config which essentially giving
> cluster-wide support to the users, we can introduce impersonation.users as
> a config and users listed in the config are allowed to impersonate other
> users.
>
> Thanks,
> Harsha
>
>
> On Fri, Dec 7, 2018, at 3:58 AM, Manikumar wrote:
> > Bump up! to get some attention.
> >
> > BTW, recently Apache Spark added  support for Kafka delegation token.
> > https://issues.apache.org/jira/browse/SPARK-25501
> >
> > On Fri, Dec 7, 2018 at 5:27 PM Manikumar 
> wrote:
> >
> > > Bump up! to get some attention.
> > >
> > > BTW, recently Apache Spark added for Kafka delegation token support.
> > > https://issues.apache.org/jira/browse/SPARK-25501
> > >
> > > On Tue, Sep 25, 2018 at 9:56 PM Manikumar 
> > > wrote:
> > >
> > >> Hi all,
> > >>
> > >> I have created a KIP that proposes to allow users to create delegation
> > >> tokens for other users.
> > >>
> > >>
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-373%3A+Allow+users+to+create+delegation+tokens+for+other+users
> > >>
> > >> Please take a look when you get a chance.
> > >>
> > >> Thanks,
> > >> Manikumar
> > >>
> > >
>

Re: Permissions to edit KIP page and ...

[Manikumar]

I just gave you permissions. Thanks.

On Wed, Dec 12, 2018 at 7:50 PM Per Steffensen  wrote:

> Hi
>
> I used to have permissions to edit KIP page
> (
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals).
>
> Now suddenly I do not. I created KIP-381 and now wanted to change to put
> it into vote mode. I cannot edit neither KIP page nor KIP-381 page.
> Please give me permissions again.
>
> Username: steff1193
>
> Kind regards, Per Steffensen
>
>

Re: Fail-fast builds?

[Manikumar]

+1 fo the suggestion.

On Fri, Dec 21, 2018 at 2:38 AM David Arthur  wrote:

> In the jenkins.sh file, we have the following comment:
>
> "In order to provide faster feedback, the tasks are ordered so that faster
> tasks are executed in every module before slower tasks (if possible)"
>
>
> but then we proceed to use the Gradle --continue flag. This means PRs won't
> get notified of problems until the whole build finishes.
>
>
> What do folks think about splitting the build invocation into a validation
> step and a test step? The validation step would omit the continue flag, but
> the test step would include it. This would allow for fast failure on
> compilation and checkstyle problems, but let the whole test suite run in
> spite of test failures.
>
>
> Cheers,
> David
>

Re: [ANNOUNCE] New Committer: Vahid Hashemian

[Manikumar]

Congrats, Vahid!

On Wed 16 Jan, 2019, 6:53 AM Ismael Juma  Congratulations Vahid!
>
> On Tue, Jan 15, 2019, 2:45 PM Jason Gustafson 
> > Hi All,
> >
> > The PMC for Apache Kafka has invited Vahid Hashemian as a project
> > committer and
> > we are
> > pleased to announce that he has accepted!
> >
> > Vahid has made numerous contributions to the Kafka community over the
> past
> > few years. He has authored 13 KIPs with core improvements to the consumer
> > and the tooling around it. He has also contributed nearly 100 patches
> > affecting all parts of the codebase. Additionally, Vahid puts a lot of
> > effort into community engagement, helping others on the mail lists and
> > sharing his experience at conferences and meetups.
> >
> > We appreciate the contributions and we are looking forward to more.
> > Congrats Vahid!
> >
> > Jason, on behalf of the Apache Kafka PMC
> >
>

Re: [VOTE] KIP-294 - Enable TLS hostname verification by default

[Manikumar]

+1 (non-binding)

Thanks.

On Wed, May 9, 2018 at 10:09 PM, Mickael Maison 
wrote:

> +1, thanks for the KIP!
>
> On Wed, May 9, 2018 at 4:41 PM, Ted Yu  wrote:
> > +1
> >
> > On Wed, May 9, 2018 at 8:28 AM, Rajini Sivaram 
> > wrote:
> >
> >> Hi all,
> >>
> >> Since there have been no objections on this straightforward KIP, I would
> >> like to initiate the voting process. KIP-294 proposes to use a secure
> >> default value for endpoint identification when using SSL as the security
> >> protocol. The KIP Is here:
> >>
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> 294+-+Enable+TLS+hostname+verification+by+default
> >>
> >> If there are any concerns, please add them to this thread or the
> discussion
> >> thread (https://www.mail-archive.com/dev@kafka.apache.org/msg87549.html
> )
> >>
> >> Regards,
> >>
> >> Rajini
> >>
>

Re: [VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position

[Manikumar]

+1 (non-binding).
Thanks.

On Thu, May 10, 2018 at 2:33 PM, Mickael Maison 
wrote:

> +1 (non binding)
> Thanks
>
> On Thu, May 10, 2018 at 9:39 AM, Rajini Sivaram 
> wrote:
> > Hi Richard, Thanks for the KIP.
> >
> > +1 (binding)
> >
> > Regards,
> >
> > Rajini
> >
> > On Wed, May 9, 2018 at 10:54 PM, Guozhang Wang 
> wrote:
> >
> >> +1 from me, thanks!
> >>
> >>
> >> Guozhang
> >>
> >> On Wed, May 9, 2018 at 10:46 AM, Jason Gustafson 
> >> wrote:
> >>
> >> > Thanks for the KIP, +1 (binding).
> >> >
> >> > One small correction: the KIP mentions that close() will be
> deprecated,
> >> but
> >> > we do not want to do this because it is needed by the Closeable
> >> interface.
> >> > We only want to deprecate close(long, TimeUnit) in favor of
> >> > close(Duration).
> >> >
> >> > -Jason
> >> >
> >> > On Tue, May 8, 2018 at 12:43 AM, khaireddine Rezgui <
> >> > khaireddine...@gmail.com> wrote:
> >> >
> >> > > +1
> >> > >
> >> > > 2018-05-07 20:35 GMT+01:00 Bill Bejeck :
> >> > >
> >> > > > +1
> >> > > >
> >> > > > Thanks,
> >> > > > Bill
> >> > > >
> >> > > > On Fri, May 4, 2018 at 7:21 PM, Richard Yu <
> >> yohan.richard...@gmail.com
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hi all, I would like to bump this thread since discussion in the
> >> KIP
> >> > > > > appears to be reaching its conclusion.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Thu, Mar 15, 2018 at 3:30 PM, Richard Yu <
> >> > > yohan.richard...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi all,
> >> > > > > >
> >> > > > > > Since there does not seem to be too much discussion in
> KIP-266, I
> >> > > will
> >> > > > be
> >> > > > > > starting a voting thread.
> >> > > > > > Here is the link to KIP-266 for reference:
> >> > > > > >
> >> > > > > > https://cwiki.apache.org/confluence/pages/viewpage.
> >> > > > > action?pageId=75974886
> >> > > > > >
> >> > > > > > Recently, I have made some updates to the KIP. To reiterate, I
> >> have
> >> > > > > > included KafkaConsumer's commitSync,
> >> > > > > > poll, and committed in the KIP. (we will be adding to a
> >> > > > TimeoutException
> >> > > > > > to them as well, in a similar manner
> >> > > > > > to what we will be doing for position())
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > Richard Yu
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Ingénieur en informatique
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
>

Re: [VOTE] KIP-278: Add version option to Kafka's commands

[Manikumar]

+1 (non-binding)

Thanks for the KIP.

On Fri, May 11, 2018 at 12:56 PM, zhenya Sun  wrote:

> +1 building
> > 在 2018年5月11日，上午9:51，Ted Yu  写道：
> >
> > +1
> >
> > On Thu, May 10, 2018 at 6:42 PM, Sasaki Toru 
> > wrote:
> >
> >> Hi all,
> >>
> >> I would like to start the vote on KIP-278: Add version option to Kafka's
> >> commands.
> >>
> >> The link to this KIP is here:
> >>  >> +Add+version+option+to+Kafka%27s+commands>
> >>
> >> The discussion thread is here:
> >> 
> >>
> >>
> >> Many thanks,
> >> Sasaki
> >>
> >> --
> >> Sasaki Toru(sasaki...@oss.nttdata.com) NTT DATA CORPORATION
> >>
> >>
>
>

Re: [VOTE] KIP-255: OAuth Authentication via SASL/OAUTHBEARER

[Manikumar]

+1 (non-binding)

Thanks for the detailed KIP.

On Wed, May 16, 2018 at 5:59 PM, Mickael Maison 
wrote:

> Thanks for the KIP,
> +1 (non binding)
>
> On Wed, May 16, 2018 at 2:51 AM, Ron Dagostino  wrote:
> > Hi Jun.  I think you are getting at the fact that OAuth 2 is a flexible
> > framework that allows different installations to do things differently.
> It
> > is true that the principal name in Kafka could come from any claim in the
> > token.  Most of the time it would come from the 'sub' claim, but it could
> > certainly come from another claim, or it could be only indirectly based
> on
> > a claim value (maybe certain text would be trimmed or prefixed/suffixed).
> > The point, which I think you are getting at, is that because the
> framework
> > is flexible we need to accommodate that flexibility.  The callback
> handler
> > class defined by the listener.name.sasl_ssl.oauthbearer.sasl.server.
> > callback.handler.class configuration value gives us the required
> > flexibility.  As an example, I have an implementation that leverages a
> > popular open source JOSE library to parse the compact serialization,
> > retrieve the public key if it has not yet been retrieved, verify the
> > digital signature, and map the 'sub' claim to the OAuthBearerToken's
> > principal name (which becomes the SASL authorization ID, which becomes
> the
> > Kafka principal name).  I could just as easily have mapped a different
> > claim to the OAuthBearerToken's principal name, manipulated the 'sub'
> claim
> > value in some way, etc.  I write the callback handler code, so I complete
> > flexibility to do whatever my OAuth 2 installation requires me to do.
> >
> > Ron
> >
> > On Tue, May 15, 2018 at 1:39 PM, Jun Rao  wrote:
> >
> >> Hi, Ron,
> >>
> >> Thanks for the reply. I understood your answers to #2 and #3.
> >>
> >> For #1, will the server map all clients' principal name to the value
> >> associated with "sub" claim? How do we support mapping different
> clients to
> >> different principal names?
> >>
> >> Jun
> >>
> >> On Mon, May 14, 2018 at 7:02 PM, Ron Dagostino 
> wrote:
> >>
> >> > Hi Jun.  Thanks for the +1 vote.
> >> >
> >> > Regarding the first question about token claims, yes, you have it
> correct
> >> > about translating the OAuth token to a principle name via a JAAS
> module
> >> > option in the default unsecured case.  Specifically, the OAuth SASL
> >> Server
> >> > implementation is responsible for setting the authorization ID, and it
> >> gets
> >> > the authorization ID from the OAuthBearerToken's principalName()
> method.
> >> > The listener.name.sasl_ssl.oauthbearer.sasl.server.
> >> callback.handler.class
> >> > is responsible for handling an instance of
> OAuthBearerValidatorCallback
> >> to
> >> > accept a token compact serialization from the client and return an
> >> instance
> >> > of OAuthBearerToken (assuming the compact serialization validates),
> and
> >> in
> >> > the default unsecured case the builtin unsecured validator callback
> >> handler
> >> > defines the OAuthBearerToken.principalName() method to return the
> 'sub'
> >> > claim value by default (with the actual claim it uses being
> configurable
> >> > via the unsecuredValidatorPrincipalClaimName JAAS module option).  So
> >> that
> >> > is how we translate from a token to a principal name in the default
> >> > unsecured (out-of-the-box) case.
> >> >
> >> > For production use cases, the implementation associated with
> >> > listener.name.sasl_ssl.oauthbearer.sasl.server.callback.handler.class
> >> can
> >> > do whatever it wants.  As an example, I have written a class that
> wraps a
> >> > com.nimbusds.jwt.SignedJWT instance (see
> >> > https://connect2id.com/products/nimbus-jose-jwt/) and presents it as
> an
> >> > OAuthBearerToken:
> >> >
> >> > public class NimbusSignedJwtOAuthBearerToken implements
> >> OAuthBearerToken {
> >> > private final SignedJWT signedJwt;
> >> > private final String principalName;
> >> > private final Set scope;
> >> > private final Long startTimeMs;
> >> > private final long lifetimeMs;
> >> >
> >> > /**
> >> >  * Constructor
> >> >  *
> >> >  * @param signedJwt
> >> >  *the mandatory signed JWT
> >> >  * @param principalClaimName
> >> >  *the mandatory claim name identifying the claim from
> >> which
> >> > the
> >> >  *principal name will be extracted. The claim must
> exist
> >> > and must be
> >> >  *a String.
> >> >  * @param optionalScopeClaimName
> >> >  *the optional claim name identifying the claim from
> >> which
> >> > any scope
> >> >  *will be extracted. If specified and the claim exists
> >> then
> >> > the
> >> >  *value must be either a String or a String List.
> >> >  * @throws ParseException
> >> >  * if the principal claim does not exist or is not a
> >> > String; the
> >> >  * scope claim is neit

Re: [VOTE] 1.1.1 RC0

[Manikumar]

+1 (non-binding)  Ran test,  Verified quick start,  producer/consumer perf
tests

On Thu, Jun 21, 2018 at 11:58 AM zhenya Sun  wrote:

> +1 non-binding
>
> > 在 2018年6月21日，下午2:18，Andras Beni  写道：
> >
> > +1 (non-binding)
> >
> > Built .tar.gz, created a cluster from it and ran a basic end-to-end test:
> > performed a rolling restart while console-producer and console-consumer
> ran
> > at around 20K messages/sec. No errors or data loss.
> >
> > Ran unit and integration tests successfully 3 out of 5 times. Encountered
> > some flakies:
> > -
> DescribeConsumerGroupTest.testDescribeGroupWithShortInitializationTimeout
> > - LogDirFailureTest.testIOExceptionDuringCheckpoint
> > - SimpleAclAuthorizerTest.testHighConcurrencyModificationOfResourceAcls
> >
> >
> > Andras
> >
> >
> > On Wed, Jun 20, 2018 at 4:59 AM Ted Yu  wrote:
> >
> >> +1
> >>
> >> Ran unit test suite which passed.
> >>
> >> Checked signatures.
> >>
> >> On Tue, Jun 19, 2018 at 4:47 PM, Dong Lin  wrote:
> >>
> >>> Re-send to kafka-clie...@googlegroups.com
> >>>
> >>> On Tue, Jun 19, 2018 at 4:29 PM, Dong Lin  wrote:
> >>>
>  Hello Kafka users, developers and client-developers,
> 
>  This is the first candidate for release of Apache Kafka 1.1.1.
> 
>  Apache Kafka 1.1.1 is a bug-fix release for the 1.1 branch that was
> >> first
>  released with 1.1.0 about 3 months ago. We have fixed about 25 issues
> >>> since
>  that release. A few of the more significant fixes include:
> 
>  KAFKA-6925  - Fix
>  memory leak in StreamsMetricsThreadImpl
>  KAFKA-6937  -
> >> In-sync
>  replica delayed during fetch if replica throttle is exceeded
>  KAFKA-6917  -
> >> Process
>  txn completion asynchronously to avoid deadlock
>  KAFKA-6893  -
> Create
>  processors before starting acceptor to avoid ArithmeticException
>  KAFKA-6870  -
>  Fix ConcurrentModificationException in SampledStat
>  KAFKA-6878  - Fix
>  NullPointerException when querying global state store
>  KAFKA-6879  -
> Invoke
>  session init callbacks outside lock to avoid Controller deadlock
>  KAFKA-6857  -
> >> Prevent
>  follower from truncating to the wrong offset if undefined leader epoch
> >> is
>  requested
>  KAFKA-6854  - Log
>  cleaner fails with transaction markers that are deleted during clean
>  KAFKA-6747  - Check
>  whether there is in-flight transaction before aborting transaction
>  KAFKA-6748  -
> Double
>  check before scheduling a new task after the punctuate call
>  KAFKA-6739  -
>  Fix IllegalArgumentException when down-converting from V2 to V0/V1
>  KAFKA-6728  -
>  Fix NullPointerException when instantiating the HeaderConverter
> 
>  Kafka 1.1.1 release plan:
>  https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.1.1
> 
>  Release notes for the 1.1.1 release:
>  http://home.apache.org/~lindong/kafka-1.1.1-rc0/RELEASE_NOTES.html
> 
>  *** Please download, test and vote by Thursday, Jun 22, 12pm PT ***
> 
>  Kafka's KEYS file containing PGP keys we use to sign the release:
>  http://kafka.apache.org/KEYS
> 
>  * Release artifacts to be voted upon (source and binary):
>  http://home.apache.org/~lindong/kafka-1.1.1-rc0/
> 
>  * Maven artifacts to be voted upon:
>  https://repository.apache.org/content/groups/staging/
> 
>  * Tag to be voted upon (off 1.1 branch) is the 1.1.1-rc0 tag:
>  https://github.com/apache/kafka/tree/1.1.1-rc0
> 
>  * Documentation:
>  http://kafka.apache.org/11/documentation.html
> 
>  * Protocol:
>  http://kafka.apache.org/11/protocol.html
> 
>  * Successful Jenkins builds for the 1.1 branch:
>  Unit/integration tests: https://builds.apache.org/job/
> >>> kafka-1.1-jdk7/150/
> 
>  Please test and verify the release artifacts and submit a vote for
> this
> >>> RC,
>  or report any issues so we can fix them and get a new RC out ASAP.
> >>> Although
>  this release vote requires PMC votes to pass, testing, votes, and bug
>  reports are valuable and appreciated from everyone.
> 
>  Cheers,
>  Dong
> 
> 
> 
> >>>
> >>
>
>

[DISCUSS] KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides configs

[Manikumar]

Hi all,

I have created a KIP to add support for dynamic update of
max.connections.per.ip/max.connections.per.ip.overrides configs

*https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
*

Any feedback is appreciated.

Thanks

Re: [kafka-clients] Re: [VOTE] 1.1.1 RC1

[Manikumar]

+1 (non-binding)  Ran tests,  Verified quick start,  producer/consumer perf
tests


On Sat, Jun 23, 2018 at 8:11 AM Dong Lin  wrote:

> Thank you for testing and voting the release!
>
> I noticed that the date for 1.1.1-rc1 is wrong. Please kindly test and
> vote by Tuesday, June 26, 12 pm PT.
>
> Thanks,
> Dong
>
> On Fri, Jun 22, 2018 at 10:09 AM, Dong Lin  wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the second candidate for release of Apache Kafka 1.1.1.
>>
>> Apache Kafka 1.1.1 is a bug-fix release for the 1.1 branch that was
>> first released with 1.1.0 about 3 months ago. We have fixed about 25 issues
>> since that release. A few of the more significant fixes include:
>>
>> KAFKA-6925  - Fix
>> memory leak in StreamsMetricsThreadImpl
>> KAFKA-6937  - In-sync
>> replica delayed during fetch if replica throttle is exceeded
>> KAFKA-6917  - Process
>> txn completion asynchronously to avoid deadlock
>> KAFKA-6893  - Create
>> processors before starting acceptor to avoid ArithmeticException
>> KAFKA-6870  -
>> Fix ConcurrentModificationException in SampledStat
>> KAFKA-6878  - Fix
>> NullPointerException when querying global state store
>> KAFKA-6879  - Invoke
>> session init callbacks outside lock to avoid Controller deadlock
>> KAFKA-6857  - Prevent
>> follower from truncating to the wrong offset if undefined leader epoch is
>> requested
>> KAFKA-6854  - Log
>> cleaner fails with transaction markers that are deleted during clean
>> KAFKA-6747  - Check
>> whether there is in-flight transaction before aborting transaction
>> KAFKA-6748  - Double
>> check before scheduling a new task after the punctuate call
>> KAFKA-6739  -
>> Fix IllegalArgumentException when down-converting from V2 to V0/V1
>> KAFKA-6728  -
>> Fix NullPointerException when instantiating the HeaderConverter
>>
>> Kafka 1.1.1 release plan:
>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.1.1
>>
>> Release notes for the 1.1.1 release:
>> http://home.apache.org/~lindong/kafka-1.1.1-rc1/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Thursday, Jun 22, 12pm PT ***
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> http://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> http://home.apache.org/~lindong/kafka-1.1.1-rc1/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/
>>
>> * Javadoc:
>> http://home.apache.org/~lindong/kafka-1.1.1-rc1/javadoc/
>>
>> * Tag to be voted upon (off 1.1 branch) is the 1.1.1-rc1 tag:
>> https://github.com/apache/kafka/tree/1.1.1-rc1
>>
>> * Documentation:
>> http://kafka.apache.org/11/documentation.html
>>
>> * Protocol:
>> http://kafka.apache.org/11/protocol.html
>>
>> * Successful Jenkins builds for the 1.1 branch:
>> Unit/integration tests: *https://builds.apache.org/job/kafka-1.1-jdk7/152/
>> *
>> System tests:
>> https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1817
>>
>>
>> Please test and verify the release artifacts and submit a vote for this
>> RC,
>> or report any issues so we can fix them and get a new RC out ASAP.
>> Although
>> this release vote requires PMC votes to pass, testing, votes, and bug
>> reports are valuable and appreciated from everyone.
>>
>> Cheers,
>> Dong
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAAaarBZCqdUPK8asaZS0ws0yr_vjFw0o8RxFcdRv07%3Df_7g%3DkQ%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

Re: [VOTE] 1.0.2 RC0

[Manikumar]

+1 (non-binding) Verified tests, quick start, producer/consumer perf tests.

On Sat, Jun 23, 2018 at 2:25 AM Ted Yu  wrote:

> +1
>
> Ran test suite.
>
> Checked signatures.
>
> On Fri, Jun 22, 2018 at 11:42 AM, Vahid S Hashemian <
> vahidhashem...@us.ibm.com> wrote:
>
> > +1 (non-binding)
> >
> > Built from source and ran quickstart successfully on Ubuntu (with Java
> 8).
> >
> > Thanks for running the release Matthias!
> > --Vahid
> >
> >
> >
> >
> > From:   "Matthias J. Sax" 
> > To: dev@kafka.apache.org, us...@kafka.apache.org,
> > kafka-clie...@googlegroups.com
> > Date:   06/22/2018 10:42 AM
> > Subject:[VOTE] 1.0.2 RC0
> >
> >
> >
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 1.0.2.
> >
> > This is a bug fix release closing 26 tickets:
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.0.2
> >
> > Release notes for the 1.0.2 release:
> > http://home.apache.org/~mjsax/kafka-1.0.2-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Tuesday, 6/26/18 end-of-day, so we
> > can close the vote on Wednesday.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~mjsax/kafka-1.0.2-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~mjsax/kafka-1.0.2-rc0/javadoc/
> >
> > * Tag to be voted upon (off 1.0 branch) is the 1.0.2 tag:
> > https://github.com/apache/kafka/releases/tag/1.0.2-rc0
> >
> > * Documentation:
> > http://kafka.apache.org/10/documentation.html
> >
> > * Protocol:
> > http://kafka.apache.org/10/protocol.html
> >
> > * Successful Jenkins builds for the 1.0 branch:
> > Unit/integration tests:
> https://builds.apache.org/job/kafka-1.0-jdk7/211/
> > System tests:
> > https://jenkins.confluent.io/job/system-test-kafka/job/1.0/217/
> >
> > /**
> >
> > Thanks,
> >   -Matthias
> >
> > [attachment "signature.asc" deleted by Vahid S Hashemian/Silicon
> > Valley/IBM]
> >
> >
> >
> >
>

[DISCUSS] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Hi all,

I have created a minor KIP to return new error code for DeleteTopics API
when topic deletion disabled.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87295558

Please take a look.

Thanks,

Re: [DISCUSS] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

 Hi,

Thanks for the review.


> nit: you are going to fill in an actual value below in your PR, right ?
> private static final long serialVersionUID = 1L;
>
>
 We normally set the serialVersionUID = 1L in Kafka exception classes.

In Motivation, please mention the IllegalStateException scenario.
>

 IllegalStateException mentioned in the JIRA is specific to user
implementaion.
 Main issue is we wont get response form the server. In this case, admin
client times out the requests.

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

[Manikumar]

+1 (non-binding)

Thanks.

On Wed, Jun 27, 2018 at 10:15 PM Matthias J. Sax 
wrote:

> +1 (binding)
>
> On 6/26/18 2:33 PM, Guozhang Wang wrote:
> > +1. Thanks.
> >
> > On Tue, Jun 26, 2018 at 2:31 PM, Yishun Guan  wrote:
> >
> >> Hi All,
> >>
> >> I am starting a vote on this KIP:
> >>
> >> https://cwiki.apache.org/confluence/x/lQg0BQ
> >>
> >> Thanks,
> >> Yishun
> >>
> >
> >
> >
>
>

Re: [DISCUSS] KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides configs

[Manikumar]

Thanks for the responses. If there are no concerns, I will start the voting
process soon.


On Tue, Jun 26, 2018 at 10:14 PM Harsha  wrote:

> This is very useful. LGTM.
>
> Thanks,
> Harsha
>
> On Mon, Jun 25th, 2018 at 10:20 AM, Dong Lin  wrote:
>
> >
> >
> >
> > Hey Manikumar,
> >
> > Thanks much for the KIP. It looks pretty good.
> >
> > Thanks,
> > Dong
> >
> > On Thu, Jun 21, 2018 at 11:38 PM, Manikumar < manikumar.re...@gmail.com
> >
> > wrote:
> >
> > > Hi all,
> > >
> > > I have created a KIP to add support for dynamic update of
> > > max.connections.per.ip/max.connections.per.ip.overrides configs
> > >
> > > *
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
> >
> > > <
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
> >
> > > >*
> > >
> > > Any feedback is appreciated.
> > >
> > > Thanks
> > >
> >
> >
> >
> >

[VOTE] KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides configs

[Manikumar]

Hi All,

I would like to start voting on KIP-308 which would add support for dynamic
update of max.connections.per.ip/max.connections.per.ip.overrides configs

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993

Thanks,

Re: [kafka-clients] [VOTE] 2.0.0 RC1

[Manikumar]

looks like maven artifacts are not updated in the staging repo. They are
still at old timestamp.
https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.11/2.0.0/

On Sat, Jun 30, 2018 at 12:06 AM Rajini Sivaram 
wrote:

> Hello Kafka users, developers and client-developers,
>
>
> This is the second candidate for release of Apache Kafka 2.0.0.
>
>
> This is a major version release of Apache Kafka. It includes 40 new  KIPs
> and
>
> several critical bug fixes. Please see the 2.0.0 release plan for more
> details:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=80448820
>
>
> A few notable highlights:
>
>- Prefixed wildcard ACLs (KIP-290), Fine grained ACLs for CreateTopics
>(KIP-277)
>- SASL/OAUTHBEARER implementation (KIP-255)
>- Improved quota communication and customization of quotas (KIP-219,
>KIP-257)
>- Efficient memory usage for down conversion (KIP-283)
>- Fix log divergence between leader and follower during fast leader
>failover (KIP-279)
>- Drop support for Java 7 and remove deprecated code including old
>scala clients
>- Connect REST extension plugin, support for externalizing secrets and
>improved error handling (KIP-285, KIP-297, KIP-298 etc.)
>- Scala API for Kafka Streams and other Streams API improvements
>(KIP-270, KIP-150, KIP-245, KIP-251 etc.)
>
> Release notes for the 2.0.0 release:
>
> http://home.apache.org/~rsivaram/kafka-2.0.0-rc1/RELEASE_NOTES.html
>
>
>
> *** Please download, test and vote by Tuesday, July 3rd, 4pm PT
>
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
>
> http://kafka.apache.org/KEYS
>
>
> * Release artifacts to be voted upon (source and binary):
>
> http://home.apache.org/~rsivaram/kafka-2.0.0-rc1/
>
>
> * Maven artifacts to be voted upon:
>
> https://repository.apache.org/content/groups/staging/
>
>
> * Javadoc:
>
> http://home.apache.org/~rsivaram/kafka-2.0.0-rc1/javadoc/
>
>
> * Tag to be voted upon (off 2.0 branch) is the 2.0.0 tag:
>
> https://github.com/apache/kafka/tree/2.0.0-rc1
>
>
> * Documentation:
>
> http://kafka.apache.org/20/documentation.html
>
>
> * Protocol:
>
> http://kafka.apache.org/20/protocol.html
>
>
> * Successful Jenkins builds for the 2.0 branch:
>
> Unit/integration tests: https://builds.apache.org/job/kafka-2.0-jdk8/66/
>
> System tests:
> https://jenkins.confluent.io/job/system-test-kafka/job/2.0/15/
>
>
>
> Please test and verify the release artifacts and submit a vote for this RC
> or report any issues so that we can fix them and roll out a new RC ASAP!
>
> Although this release vote requires PMC votes to pass, testing, votes,
> and bug
> reports are valuable and appreciated from everyone.
>
>
> Thanks,
>
>
> Rajini
>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAOJcB39GdTWOaK4qysvyPyGU8Ldm82t_TA364x1MP8a8OAod6A%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

Re: [kafka-clients] Re: [VOTE] 0.11.0.3 RC0

[Manikumar]

+1 (non-binding)   Verified quick start, ran producer/consumer perf tests,

Thanks


On Sun, Jul 1, 2018 at 8:29 PM Ismael Juma  wrote:

> +1 (binding)
>
> Verified signature of source artifact, ran tests and verified quickstart
> on source artifact with Java 7, verified quickstart on binary artifact (Scala
> 2.11) with Java 7, sanity checked release notes and Maven staging
> repository.
>
> Thanks for running the release!
>
> Ismael
>
> On Fri, Jun 22, 2018 at 3:14 PM Matthias J. Sax 
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the first candidate for release of Apache Kafka 0.11.0.3.
>>
>> This is a bug fix release closing 27 tickets:
>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.11.0.3
>>
>> Release notes for the 0.11.0.3 release:
>> http://home.apache.org/~mjsax/kafka-0.11.0.3-rc0/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Tuesday, 6/26/18 end-of-day, so we
>> can close the vote on Wednesday.
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> http://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> http://home.apache.org/~mjsax/kafka-0.11.0.3-rc0/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/
>>
>> * Javadoc:
>> http://home.apache.org/~mjsax/kafka-0.11.0.3-rc0/javadoc/
>>
>> * Tag to be voted upon (off 0.11.0 branch) is the 0.11.0.3 tag:
>> https://github.com/apache/kafka/releases/tag/0.11.0.3-rc0
>>
>> * Documentation:
>> http://kafka.apache.org/0110/documentation.html
>>
>> * Protocol:
>> http://kafka.apache.org/0110/protocol.html
>>
>> * Successful Jenkins builds for the 0.11.0 branch:
>> Unit/integration tests:
>> https://builds.apache.org/job/kafka-0.11.0-jdk7/385/
>> System tests:
>> https://jenkins.confluent.io/job/system-test-kafka/job/0.11.0/217/
>>
>> /**
>>
>> Thanks,
>>   -Matthias
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAD5tkZb38rnx99Ab%2BTbK8O3j%3D3xZLuqoPH2%2BdWxOGetzSpCmpQ%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

Re: [kafka-clients] Re: [VOTE] 1.0.2 RC1

[Manikumar]

+1 (non-binding) Verified tests, quick start, producer/consumer perf tests.



On Sun, Jul 1, 2018 at 11:27 PM Ismael Juma  wrote:

> +1 (binding)
>
> Verified signature of source artifact, ran tests and verified quickstart
> on source artifact with Java 7, verified quickstart on binary artifact (Scala
> 2.12) with Java 8, sanity checked release notes and Maven staging
> repository.
>
> Ismael
>
> On Fri, Jun 29, 2018 at 10:02 PM Matthias J. Sax 
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the second candidate for release of Apache Kafka 1.0.2.
>>
>> This is a bug fix release addressing 27 tickets:
>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.0.2
>>
>> Release notes for the 1.0.2 release:
>> http://home.apache.org/~mjsax/kafka-1.0.2-rc1/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by end of next week (7/6/18).
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> http://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> http://home.apache.org/~mjsax/kafka-1.0.2-rc1/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/
>>
>> * Javadoc:
>> http://home.apache.org/~mjsax/kafka-1.0.2-rc1/javadoc/
>>
>> * Tag to be voted upon (off 1.0 branch) is the 1.0.2 tag:
>> https://github.com/apache/kafka/releases/tag/1.0.2-rc1
>>
>> * Documentation:
>> http://kafka.apache.org/10/documentation.html
>>
>> * Protocol:
>> http://kafka.apache.org/10/protocol.html
>>
>> * Successful Jenkins builds for the 1.0 branch:
>> Unit/integration tests: https://builds.apache.org/job/kafka-1.0-jdk7/214/
>> System tests:
>> https://jenkins.confluent.io/job/system-test-kafka/job/1.0/225/
>>
>> /**
>>
>> Thanks,
>>   -Matthias
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAD5tkZZUnorMq4mgKnY5v6t0sAi4TNtMcQPjyYd18R_FC%3DGzzg%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

Re: [DISCUSS] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Since this is a minor KIP, If there are no concerns I will start voting
thread.

On Wed, Jun 27, 2018 at 2:24 PM Ted Yu  wrote:

> bq. set the serialVersionUID = 1L in Kafka exception classes
>
> Indeed. Should have checked earlier.
>
> On Wed, Jun 27, 2018 at 1:38 AM, Manikumar 
> wrote:
>
> >  Hi,
> >
> > Thanks for the review.
> >
> >
> > > nit: you are going to fill in an actual value below in your PR, right ?
> > > private static final long serialVersionUID = 1L;
> > >
> > >
> >  We normally set the serialVersionUID = 1L in Kafka exception classes.
> >
> > In Motivation, please mention the IllegalStateException scenario.
> > >
> >
> >  IllegalStateException mentioned in the JIRA is specific to user
> > implementaion.
> >  Main issue is we wont get response form the server. In this case, admin
> > client times out the requests.
> >
>

[VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Manikumar 
Fri, Jun 29, 7:59 PM (4 days ago)
to dev
Hi All,

I would like to start voting on KIP-322 which would return new error code
for DeleteTopics API when topic deletion disabled.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87295558

Thanks,

Re: [VOTE] 2.0.0 RC1

[Manikumar]

+1 (non-binding)  Verified the release notes, src, binary artifacts,  Ran
the test suite,
Verified quick start, Ran producer/consumer perf test, log compaction tests

Thanks


On Wed, Jul 4, 2018 at 8:33 AM Brett Rann  wrote:

> +1 tentative
> rolling upgrade of tiny shared staging multitenacy (200+ consumer groups)
> cluster from 1.1 to 2.0.0-rc1. cluster looks healthy. Will monitor.
>
> On Tue, Jul 3, 2018 at 8:18 AM Harsha  wrote:
>
> > +1.
> >
> > 1) Ran unit tests
> > 2) 3 node cluster , tested basic operations.
> >
> > Thanks,
> > Harsha
> >
> > On Mon, Jul 2nd, 2018 at 11:13 AM, "Vahid S Hashemian" <
> > vahidhashem...@us.ibm.com> wrote:
> >
> > >
> > >
> > >
> > > +1 (non-binding)
> > >
> > > Built from source and ran quickstart successfully on Ubuntu (with Java
> > 8).
> > >
> > >
> > > Minor: It seems this doc update PR is not included in the RC:
> > > https://github.com/apache/kafka/pull/5280
> > 
> > > Guozhang seems to have wanted to cherry-pick it to 2.0.
> > >
> > > Thanks Rajini!
> > > --Vahid
> > >
> > >
> > >
> > >
> > > From: Rajini Sivaram < rajinisiva...@gmail.com >
> > > To: dev < dev@kafka.apache.org >, Users < us...@kafka.apache.org >,
> > > kafka-clients < kafka-clie...@googlegroups.com >
> > > Date: 06/29/2018 11:36 AM
> > > Subject: [VOTE] 2.0.0 RC1
> > >
> > >
> > >
> > > Hello Kafka users, developers and client-developers,
> > >
> > >
> > > This is the second candidate for release of Apache Kafka 2.0.0.
> > >
> > >
> > > This is a major version release of Apache Kafka. It includes 40 new
> KIPs
> > > and
> > >
> > > several critical bug fixes. Please see the 2.0.0 release plan for more
> > > details:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=80448820
> > <
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=80448820>
> > >
> > >
> > >
> > > A few notable highlights:
> > >
> > > - Prefixed wildcard ACLs (KIP-290), Fine grained ACLs for CreateTopics
> > > (KIP-277)
> > > - SASL/OAUTHBEARER implementation (KIP-255)
> > > - Improved quota communication and customization of quotas (KIP-219,
> > > KIP-257)
> > > - Efficient memory usage for down conversion (KIP-283)
> > > - Fix log divergence between leader and follower during fast leader
> > > failover (KIP-279)
> > > - Drop support for Java 7 and remove deprecated code including old
> > > scala
> > > clients
> > > - Connect REST extension plugin, support for externalizing secrets and
> > > improved error handling (KIP-285, KIP-297, KIP-298 etc.)
> > > - Scala API for Kafka Streams and other Streams API improvements
> > > (KIP-270, KIP-150, KIP-245, KIP-251 etc.)
> > >
> > > Release notes for the 2.0.0 release:
> > >
> > > http://home.apache.org/~rsivaram/kafka-2.0.0-rc1/RELEASE_NOTES.html
> > 
> > >
> > >
> > >
> > >
> > > *** Please download, test and vote by Tuesday, July 3rd, 4pm PT
> > >
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > >
> > > http://kafka.apache.org/KEYS
> > 
> > >
> > >
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > >
> > > http://home.apache.org/~rsivaram/kafka-2.0.0-rc1/
> > 
> > >
> > >
> > >
> > > * Maven artifacts to be voted upon:
> > >
> > > https://repository.apache.org/content/groups/staging/
> > 
> > >
> > >
> > >
> > > * Javadoc:
> > >
> > > http://home.apache.org/~rsivaram/kafka-2.0.0-rc1/javadoc/
> > 
> > >
> > >
> > >
> > > * Tag to be voted upon (off 2.0 branch) is the 2.0.0 tag:
> > >
> > > https://github.com/apache/kafka/tree/2.0.0-rc1
> > 
> > >
> > >
> > >
> > > * Documentation:
> > >
> > > http://kafka.apache.org/20/documentation.html
> > 
> > >
> > >
> > >
> > > * Protocol:
> > >
> > > http://kafka.apache.org/20/protocol.html
> > 
> > >
> > >
> > >
> > > * Successful Jenkins builds for the 2.0 branch:
> > >
> > > Unit/integration tests:
> > > https://builds.apache.org/job/kafka-2.0-jdk8/66/
> > 
> > >
> > >
> > > System tests:
> > > https://jenkins.confluent.io/job/system-test-kafka/job/2.0/15/
> > 
> > >
> > >
> > >
> > >
> > > Please test and verify the release artifacts and submit a vote for this
> > RC
> > >
> > > or report any issues so that we can fix them and roll out a new RC
> ASAP!
> > >
> > > Although this release vote requires PMC votes to pass, testing, votes,
> > and
> > >
> > > bug
> > > reports are valuable and appreciated from everyone.
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Rajini
> > >
> > >
> > >
> > >

Re: [VOTE] KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides configs

[Manikumar]

*Hi All,The vote has passed with 3 binding votes (Dong Lin, Rajini, Jason,)
and one non-binding vote(Ted).Thanks everyone for the
votes.Thanks,Manikumar*

On Fri, Jun 29, 2018 at 9:34 PM Jason Gustafson  wrote:

> +1
>
> Thanks Manikumar!
>
> On Fri, Jun 29, 2018 at 8:37 AM, Rajini Sivaram 
> wrote:
>
> > +1 (binding)
> >
> > Thanks for the KIP, Manikumar!
> >
> > On Fri, Jun 29, 2018 at 4:23 PM, Dong Lin  wrote:
> >
> > > +1
> > >
> > > Thanks!
> > >
> > > On Fri, 29 Jun 2018 at 7:36 AM Ted Yu  wrote:
> > >
> > > > +1
> > > >
> > > > On Fri, Jun 29, 2018 at 7:29 AM, Manikumar <
> manikumar.re...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > I would like to start voting on KIP-308 which would add support for
> > > > dynamic
> > > > > update of max.connections.per.ip/max.connections.per.ip.overrides
> > > configs
> > > > >
> > > > >
> > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > > action?pageId=85474993
> > > > >
> > > > > Thanks,
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Waiting for one more binding vote to pass this minor KIP.  Appreciate your
vote.

On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska  wrote:

> +1 (non binding)
>
> On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash <
> kamal.chandraprak...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > On Wed, Jul 4, 2018 at 5:22 PM Magnus Edenhill 
> wrote:
> >
> > > +1 (non-binding)
> > >
> > > 2018-07-04 13:40 GMT+02:00 Satish Duggana :
> > >
> > > > +1
> > > >
> > > > Thanks,
> > > > Satish.
> > > >
> > > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione  >
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Thanks,
> > > > > Daniele
> > > > >
> > > > > Il giorno mar 3 lug 2018 alle ore 23:55 Harsha 
> ha
> > > > > scritto:
> > > > >
> > > > > > +1.
> > > > > >
> > > > > > Thanks,
> > > > > > Harsha
> > > > > >
> > > > > > On Tue, Jul 3rd, 2018 at 9:22 AM, Ted Yu 
> > > wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > On Tue, Jul 3, 2018 at 9:05 AM, Mickael Maison <
> > > > > > mickael.mai...@gmail.com >
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1 (non binding)
> > > > > > > > Thanks for the KIP
> > > > > > > >
> > > > > > > > On Tue, Jul 3, 2018 at 4:59 PM, Vahid S Hashemian
> > > > > > > > < vahidhashem...@us.ibm.com > wrote:
> > > > > > > > > +1 (non-binding)
> > > > > > > > >
> > > > > > > > > --Vahid
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > From: Gwen Shapira < g...@confluent.io >
> > > > > > > > > To: dev < dev@kafka.apache.org >
> > > > > > > > > Date: 07/03/2018 08:49 AM
> > > > > > > > > Subject: Re: [VOTE] KIP-322: Return new error code for
> > > > > > > > DeleteTopics
> > > > > > > > > API when topic deletion disabled.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > +1
> > > > > > > > >
> > > > > > > > > On Tue, Jul 3, 2018 at 8:24 AM, Manikumar <
> > > > > > manikumar.re...@gmail.com >
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Manikumar < manikumar.re...@gmail.com >
> > > > > > > > >> Fri, Jun 29, 7:59 PM (4 days ago)
> > > > > > > > >> to dev
> > > > > > > > >> Hi All,
> > > > > > > > >>
> > > > > > > > >> I would like to start voting on KIP-322 which would return
> > new
> > > > > error
> > > > > > > > > code
> > > > > > > > >> for DeleteTopics API when topic deletion disabled.
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > > > > > > > action?pageId=87295558
> > > > > > > > >
> > > > > > > > >>
> > > > > > > > >> Thanks,
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > *Gwen Shapira*
> > > > > > > > > Product Manager | Confluent
> > > > > > > > > 650.450.2760 | @gwenshap
> > > > > > > > > Follow us: Twitter <
> > > > > > > > > https://twitter.com/ConfluentInc
> > > > > > > > >> | blog
> > > > > > > > > <
> > > > > > > > > http://www.confluent.io/blog
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

[Manikumar]

LGTM. Thanks for the KIP.

Thanks.

On Wed, Jul 11, 2018 at 3:31 PM Chia-Ping Tsai  wrote:

> hi Kafka,
>
> KIP-338 is trying to make us exclude internal topics easily when using the
> kafka-topics.sh. An new option will be added to kafka-topics.sh
> (TopicCommand) and user can use the option to exclude all internal topics
> when running the list or describe command.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-338+Support+to+exclude+the+internal+topics+in+kafka-topics.sh+command
>
> feedback is welcome
>
> Cheers,
> chia-ping
>

Re: [VOTE] 1.1.1 RC3

[Manikumar]

+1 (non-binding)  Ran tests,  Verified quick start,  producer/consumer perf
tests



On Thu, Jul 12, 2018 at 11:06 AM Brett Rann 
wrote:

> +1 (non binding)
> rolling upgrade of shared staging multitenacy (200+ consumer groups)
> cluster from 1.1.0 to 1.1.1-rc3 using the kafka_2.11-1.1.1.tgz artifact.
> cluster looks healthy after upgrade. Lack of burrow lag suggests consumers
> are still happy, and incoming messages remains the same.
>
> On Mon, Jul 9, 2018 at 8:36 AM Dong Lin  wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the fourth candidate for release of Apache Kafka 1.1.1.
> >
> > Apache Kafka 1.1.1 is a bug-fix release for the 1.1 branch that was first
> > released with 1.1.0 about 3 months ago. We have fixed about 25 issues
> since
> > that release. A few of the more significant fixes include:
> >
> > KAFKA-6925  > > - Fix memory
> > leak in StreamsMetricsThreadImpl
> > KAFKA-6937  > > - In-sync
> > replica delayed during fetch if replica throttle is exceeded
> > KAFKA-6917  > > - Process txn
> > completion asynchronously to avoid deadlock
> > KAFKA-6893  > > - Create
> > processors before starting acceptor to avoid ArithmeticException
> > KAFKA-6870  > > -
> > Fix ConcurrentModificationException in SampledStat
> > KAFKA-6878  > > - Fix
> > NullPointerException when querying global state store
> > KAFKA-6879  > > - Invoke
> > session init callbacks outside lock to avoid Controller deadlock
> > KAFKA-6857  > > - Prevent
> > follower from truncating to the wrong offset if undefined leader epoch is
> > requested
> > KAFKA-6854  > > - Log cleaner
> > fails with transaction markers that are deleted during clean
> > KAFKA-6747  > > - Check
> > whether there is in-flight transaction before aborting transaction
> > KAFKA-6748  > > - Double
> > check before scheduling a new task after the punctuate call
> > KAFKA-6739  > > -
> > Fix IllegalArgumentException when down-converting from V2 to V0/V1
> > KAFKA-6728  > > -
> > Fix NullPointerException when instantiating the HeaderConverter
> >
> > Kafka 1.1.1 release plan:
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.1.1
> > 
> >
> > Release notes for the 1.1.1 release:
> > http://home.apache.org/~lindong/kafka-1.1.1-rc3/RELEASE_NOTES.html
> > 
> >
> > *** Please download, test and vote by Thursday, July 12, 12pm PT ***
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> > 
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~lindong/kafka-1.1.1-rc3/
> > 
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> > 
> >
> > * Javadoc:
> > http://home.apache.org/~lindong/kafka-1.1.1-rc3/javadoc/
> > 
> >
> > * Tag to be voted upon (off 1.1 branch) is the 1.1.1-rc3 tag:
> > https://github.com/apache/kafka/tree/1.1.1-rc3
> > 
> >
> > * Documentation:
> > http://kafka.apache.org/11/documentation.html
> > 
> >
> > * Protocol:
> > http://kafka.apache.org/11/protocol.html
> > 
> >
> > * Successful Jenkins builds for the 1.1 branch:
> > Unit/integration tests: *
> https://builds.apache.org/job/kafka-1.1-jdk7/162
> > 

KIP-327: Add describe all topics API to AdminClient

[Manikumar]

Hi all,

I have created a KIP to add describe all topics API to AdminClient .

https://cwiki.apache.org/confluence/display/KAFKA/KIP-327%3A+Add+describe+all+topics+API+to+AdminClient

Please take a look.

Thanks,

Re: KIP-327: Add describe all topics API to AdminClient

[Manikumar]

Hi Jason and Colin,

Thanks for the feedback. I agree that having filtering support to the
Metadata API would be useful and solves
the scalability issues.

But to implement specific use case of "describe all topics", regex support
won't help. In any case user needs to
call listTopics() to get topic list, and then make describeTopics() calls
with a subset of the topics set.
This leads to improving existing listTopics() API performance. Colin
already raised a KIP for this: KIP-142
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-142%3A+Add+ListTopicsRequest+to+efficiently+list+all+the+topics+in+a+cluster>
 .
May be we should consider implementing KIP-142.

Since we have support wildcard ACLs, Initially, I can explore
prefixed/wildcards patterns support to Metadata API.
We can later extend support for regular expressions.

Thanks



On Sat, Jul 14, 2018 at 2:42 PM Ted Yu  wrote:

> What if broker crashes before all the pages can be returned ?
>
> Cheers
>
> On Sat, Jul 14, 2018 at 1:07 AM Stephane Maarek <
> steph...@simplemachines.com.au> wrote:
>
> > Why not paginate ? Then one can retrieve as many topics as desired ?
> >
> > On Sat., 14 Jul. 2018, 4:15 pm Colin McCabe,  wrote:
> >
> > > Good point.  We should probably have a maximum number of results like
> > > 1000 or something.  That can go in the request RPC as well...
> > > Cheers,
> > > Colin
> > >
> > > On Fri, Jul 13, 2018, at 18:15, Ted Yu wrote:
> > > > bq. describe topics by a regular expression on the server side
> > > >
> > > > Should caution be taken if the regex doesn't filter ("*") ?
> > > >
> > > > Cheers
> > > >
> > > > On Fri, Jul 13, 2018 at 6:02 PM Colin McCabe
> > > >  wrote:>
> > > > > As Jason wrote, this won't scale as the number of partitions
> > > > > increases.> > We already have users who have tens of thousands of
> > > topics, or
> > > > > more.  If> > you multiply that by 100x over the next few years, you
> > > end up with
> > > > > this API> > returning full information about millions of topics,
> > which
> > > clearly
> > > > > doesn't> > work.
> > > > >
> > > > > We discussed this a lot in the original KIP-117 DISCUSS thread
> > > > > which added> > the Java AdminClient.  ListTopics and DescribeTopics
> > > were
> > > > > deliberately kept> > separate because we understood that
> eventually a
> > > single RPC would
> > > > > not be> > able to return information about all the topics in the
> > > cluster.  So
> > > > > I have> > to vote -1 for this proposal as it stands.
> > > > >
> > > > > I do agree that adding a way to describe topics by a regular
> > > > > expression on> > the server side would be very useful.  This would
> > > also fix a major
> > > > > scalability problem we have now, which is that when
> > > > > subscribing via a> > regular expression, clients need to fetch the
> > > full list of all
> > > > > topics in> > the cluster and filter locally.
> > > > >
> > > > > I think a regular expression library like re2 would be ideal
> > > > > for this> > purpose.  re2 is standardized and language-agnostic
> (it's
> > > not tied
> > > > > only to> > Java).  In contrast, Java regular expression change with
> > > different
> > > > > releases> > of the JDK (there were some changes in java 8, for
> > > example).
> > > > > Also, re2> > regular expressions are linear time, never exponential
> > > time.  See
> > > > > https://github.com/google/re2j
> > > > >
> > > > > regards,
> > > > > Colin
> > > > >
> > > > >
> > > > > On Fri, Jul 13, 2018, at 05:00, Andras Beni wrote:
> > > > > > The KIP looks good to me.
> > > > > > However, if there is willingness in the community to work on
> > > > > > metadata> > > request with patterns, the feature proposed here
> and
> > > filtering by
> > > > > > '*' or> > > '.*' would be redundant.
> > > > > >
> > > > > > Andras
> > > > > >
> > > > > >
> > > > > >
> > 

Re: KIP-327: Add describe all topics API to AdminClient

[Manikumar]

Closing this KIP in favor of adding filtering support to the Metadata API
and KIP-142. Will open a new KIP when ready.
Thanks for your reviews.

On Mon, Jul 16, 2018 at 8:38 AM Colin McCabe  wrote:

> Thanks, Manikumar.  I've been meaning to bring up KIP-142 again.  It would
> definitely be a nice improvement.
>
> best,
> Colin
>
>
> On Sat, Jul 14, 2018, at 08:51, Manikumar wrote:
> > Hi Jason and Colin,
> >
> > Thanks for the feedback. I agree that having filtering support to the
> > Metadata API would be useful and solves
> > the scalability issues.
> >
> > But to implement specific use case of "describe all topics", regex
> > support
> > won't help. In any case user needs to
> > call listTopics() to get topic list, and then make describeTopics()
> > calls
> > with a subset of the topics set.
> > This leads to improving existing listTopics() API performance. Colin
> > already raised a KIP for this: KIP-142
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-142%3A+Add+ListTopicsRequest+to+efficiently+list+all+the+topics+in+a+cluster
> >
> >  .
> > May be we should consider implementing KIP-142.
> >
> > Since we have support wildcard ACLs, Initially, I can explore
> > prefixed/wildcards patterns support to Metadata API.
> > We can later extend support for regular expressions.
> >
> > Thanks
> >
> >
> >
> > On Sat, Jul 14, 2018 at 2:42 PM Ted Yu  wrote:
> >
> > > What if broker crashes before all the pages can be returned ?
> > >
> > > Cheers
> > >
> > > On Sat, Jul 14, 2018 at 1:07 AM Stephane Maarek <
> > > steph...@simplemachines.com.au> wrote:
> > >
> > > > Why not paginate ? Then one can retrieve as many topics as desired ?
> > > >
> > > > On Sat., 14 Jul. 2018, 4:15 pm Colin McCabe, 
> wrote:
> > > >
> > > > > Good point.  We should probably have a maximum number of results
> like
> > > > > 1000 or something.  That can go in the request RPC as well...
> > > > > Cheers,
> > > > > Colin
> > > > >
> > > > > On Fri, Jul 13, 2018, at 18:15, Ted Yu wrote:
> > > > > > bq. describe topics by a regular expression on the server side
> > > > > >
> > > > > > Should caution be taken if the regex doesn't filter ("*") ?
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > On Fri, Jul 13, 2018 at 6:02 PM Colin McCabe
> > > > > >  wrote:>
> > > > > > > As Jason wrote, this won't scale as the number of partitions
> > > > > > > increases.> > We already have users who have tens of thousands
> of
> > > > > topics, or
> > > > > > > more.  If> > you multiply that by 100x over the next few
> years, you
> > > > > end up with
> > > > > > > this API> > returning full information about millions of
> topics,
> > > > which
> > > > > clearly
> > > > > > > doesn't> > work.
> > > > > > >
> > > > > > > We discussed this a lot in the original KIP-117 DISCUSS thread
> > > > > > > which added> > the Java AdminClient.  ListTopics and
> DescribeTopics
> > > > > were
> > > > > > > deliberately kept> > separate because we understood that
> > > eventually a
> > > > > single RPC would
> > > > > > > not be> > able to return information about all the topics in
> the
> > > > > cluster.  So
> > > > > > > I have> > to vote -1 for this proposal as it stands.
> > > > > > >
> > > > > > > I do agree that adding a way to describe topics by a regular
> > > > > > > expression on> > the server side would be very useful.  This
> would
> > > > > also fix a major
> > > > > > > scalability problem we have now, which is that when
> > > > > > > subscribing via a> > regular expression, clients need to fetch
> the
> > > > > full list of all
> > > > > > > topics in> > the cluster and filter locally.
> > > > > > >
> > > > > > > I think a regular expression library like re2 would be ideal
> > > > > > > for this&

Re: [VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

[Manikumar]

+1 (non-binding)

On Thu, Jul 19, 2018 at 11:47 PM Colin McCabe  wrote:

> +1 (non-binding)
>
> cheers,
> Colin
>
> On Mon, Jul 16, 2018, at 01:10, Chia-Ping Tsai wrote:
> > hi folks,
> >
> > The discussion[1] of KIP-338[2] did not get any objection for last 6
> > days so it is time to start the voting thread.
> >
> > Thanks for your time!
> >
> > [1]
> >
> https://lists.apache.org/thread.html/9bd4e61b73c901b51132ada49743b9b703d40b85fc4eeaa5c9099900@%3Cdev.kafka.apache.org%3E
> >
> > [2]
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-338+Support+to+exclude+the+internal+topics+in+kafka-topics.sh+command
> >
> > Cheers,
> > chia-ping
>

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Bumping this thread.

On Tue, Jul 10, 2018 at 10:33 PM Manikumar 
wrote:

> Waiting for one more binding vote to pass this minor KIP.  Appreciate your
> vote.
>
> On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska 
> wrote:
>
>> +1 (non binding)
>>
>> On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash <
>> kamal.chandraprak...@gmail.com> wrote:
>>
>> > +1 (non-binding)
>> >
>> > On Wed, Jul 4, 2018 at 5:22 PM Magnus Edenhill 
>> wrote:
>> >
>> > > +1 (non-binding)
>> > >
>> > > 2018-07-04 13:40 GMT+02:00 Satish Duggana :
>> > >
>> > > > +1
>> > > >
>> > > > Thanks,
>> > > > Satish.
>> > > >
>> > > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione <
>> ascion...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > +1
>> > > > >
>> > > > > Thanks,
>> > > > > Daniele
>> > > > >
>> > > > > Il giorno mar 3 lug 2018 alle ore 23:55 Harsha 
>> ha
>> > > > > scritto:
>> > > > >
>> > > > > > +1.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Harsha
>> > > > > >
>> > > > > > On Tue, Jul 3rd, 2018 at 9:22 AM, Ted Yu 
>> > > wrote:
>> > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > +1
>> > > > > > >
>> > > > > > > On Tue, Jul 3, 2018 at 9:05 AM, Mickael Maison <
>> > > > > > mickael.mai...@gmail.com >
>> > > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > +1 (non binding)
>> > > > > > > > Thanks for the KIP
>> > > > > > > >
>> > > > > > > > On Tue, Jul 3, 2018 at 4:59 PM, Vahid S Hashemian
>> > > > > > > > < vahidhashem...@us.ibm.com > wrote:
>> > > > > > > > > +1 (non-binding)
>> > > > > > > > >
>> > > > > > > > > --Vahid
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > From: Gwen Shapira < g...@confluent.io >
>> > > > > > > > > To: dev < dev@kafka.apache.org >
>> > > > > > > > > Date: 07/03/2018 08:49 AM
>> > > > > > > > > Subject: Re: [VOTE] KIP-322: Return new error code for
>> > > > > > > > DeleteTopics
>> > > > > > > > > API when topic deletion disabled.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > +1
>> > > > > > > > >
>> > > > > > > > > On Tue, Jul 3, 2018 at 8:24 AM, Manikumar <
>> > > > > > manikumar.re...@gmail.com >
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >> Manikumar < manikumar.re...@gmail.com >
>> > > > > > > > >> Fri, Jun 29, 7:59 PM (4 days ago)
>> > > > > > > > >> to dev
>> > > > > > > > >> Hi All,
>> > > > > > > > >>
>> > > > > > > > >> I would like to start voting on KIP-322 which would
>> return
>> > new
>> > > > > error
>> > > > > > > > > code
>> > > > > > > > >> for DeleteTopics API when topic deletion disabled.
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.
>> > > > > > > > action?pageId=87295558
>> > > > > > > > >
>> > > > > > > > >>
>> > > > > > > > >> Thanks,
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > *Gwen Shapira*
>> > > > > > > > > Product Manager | Confluent
>> > > > > > > > > 650.450.2760 | @gwenshap
>> > > > > > > > > Follow us: Twitter <
>> > > > > > > > > https://twitter.com/ConfluentInc
>> > > > > > > > >> | blog
>> > > > > > > > > <
>> > > > > > > > > http://www.confluent.io/blog
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Hi Jason,

Yes, we can return UNKNOWN_SERVER_ERROR  for older versions.
This will give better developer experience. Updated the KIP.

Thanks,

On Fri, Jul 20, 2018 at 11:02 PM Jason Gustafson  wrote:

> Hey Manikumar,
>
> This looks good. Do we have to keep the current behavior when handling old
> DeleteTopics versions? I'm wondering if it would be better to return an
> UNKNOWN_ERROR (say) and let the client raise that to the user instead of
> just timing out.
>
> Thanks,
> Jason
>
> On Fri, Jul 20, 2018 at 10:00 AM, Manikumar 
> wrote:
>
> > Bumping this thread.
> >
> > On Tue, Jul 10, 2018 at 10:33 PM Manikumar 
> > wrote:
> >
> > > Waiting for one more binding vote to pass this minor KIP.  Appreciate
> > your
> > > vote.
> > >
> > > On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska 
> > > wrote:
> > >
> > >> +1 (non binding)
> > >>
> > >> On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash <
> > >> kamal.chandraprak...@gmail.com> wrote:
> > >>
> > >> > +1 (non-binding)
> > >> >
> > >> > On Wed, Jul 4, 2018 at 5:22 PM Magnus Edenhill 
> > >> wrote:
> > >> >
> > >> > > +1 (non-binding)
> > >> > >
> > >> > > 2018-07-04 13:40 GMT+02:00 Satish Duggana <
> satish.dugg...@gmail.com
> > >:
> > >> > >
> > >> > > > +1
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Satish.
> > >> > > >
> > >> > > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione <
> > >> ascion...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > +1
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Daniele
> > >> > > > >
> > >> > > > > Il giorno mar 3 lug 2018 alle ore 23:55 Harsha <
> ka...@harsha.io
> > >
> > >> ha
> > >> > > > > scritto:
> > >> > > > >
> > >> > > > > > +1.
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > Harsha
> > >> > > > > >
> > >> > > > > > On Tue, Jul 3rd, 2018 at 9:22 AM, Ted Yu <
> yuzhih...@gmail.com
> > >
> > >> > > wrote:
> > >> > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > +1
> > >> > > > > > >
> > >> > > > > > > On Tue, Jul 3, 2018 at 9:05 AM, Mickael Maison <
> > >> > > > > > mickael.mai...@gmail.com >
> > >> > > > > > >
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > > +1 (non binding)
> > >> > > > > > > > Thanks for the KIP
> > >> > > > > > > >
> > >> > > > > > > > On Tue, Jul 3, 2018 at 4:59 PM, Vahid S Hashemian
> > >> > > > > > > > < vahidhashem...@us.ibm.com > wrote:
> > >> > > > > > > > > +1 (non-binding)
> > >> > > > > > > > >
> > >> > > > > > > > > --Vahid
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > From: Gwen Shapira < g...@confluent.io >
> > >> > > > > > > > > To: dev < dev@kafka.apache.org >
> > >> > > > > > > > > Date: 07/03/2018 08:49 AM
> > >> > > > > > > > > Subject: Re: [VOTE] KIP-322: Return new error code for
> > >> > > > > > > > DeleteTopics
> > >> > > > > > > > > API when topic deletion disabled.
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > +1
> > >> > > > > > > > >
> > >> > > > > > > > > On Tue, Jul 3, 2018 at 8:24 AM, Manikumar <
> > >> > > > > > manikumar.re...@gmail.com >
> > >> > > > > > >
> > >> > > > > > > > > wrote:
> > >> > > > > > > > >
> > >> > > > > > > > >> Manikumar < manikumar.re...@gmail.com >
> > >> > > > > > > > >> Fri, Jun 29, 7:59 PM (4 days ago)
> > >> > > > > > > > >> to dev
> > >> > > > > > > > >> Hi All,
> > >> > > > > > > > >>
> > >> > > > > > > > >> I would like to start voting on KIP-322 which would
> > >> return
> > >> > new
> > >> > > > > error
> > >> > > > > > > > > code
> > >> > > > > > > > >> for DeleteTopics API when topic deletion disabled.
> > >> > > > > > > > >>
> > >> > > > > > > > >>
> > >> > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > >> > > > > > > > action?pageId=87295558
> > >> > > > > > > > >
> > >> > > > > > > > >>
> > >> > > > > > > > >> Thanks,
> > >> > > > > > > > >>
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > --
> > >> > > > > > > > > *Gwen Shapira*
> > >> > > > > > > > > Product Manager | Confluent
> > >> > > > > > > > > 650.450.2760 | @gwenshap
> > >> > > > > > > > > Follow us: Twitter <
> > >> > > > > > > > > https://twitter.com/ConfluentInc
> > >> > > > > > > > >> | blog
> > >> > > > > > > > > <
> > >> > > > > > > > > http://www.confluent.io/blog
> > >> > > > > > > > >>
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

Thanks for the feedback, sounds good to me. Updated the KIP.

On Sat, Jul 21, 2018 at 12:04 AM Jason Gustafson  wrote:

> After thinking about it, perhaps a better option is INVALID_REQUEST? I
> think this is usually used for requests which are malformed in some way, so
> it's a slight abuse, but at least it suggests an incompatibility between
> what the client is requesting to do and what the server supports.
> UNKNOWN_SERVER_ERROR, on the other hand, suggests an unexpected condition
> on the broker.
>
> -Jason
>
> On Fri, Jul 20, 2018 at 10:57 AM, Manikumar 
> wrote:
>
> > Hi Jason,
> >
> > Yes, we can return UNKNOWN_SERVER_ERROR  for older versions.
> > This will give better developer experience. Updated the KIP.
> >
> > Thanks,
> >
> > On Fri, Jul 20, 2018 at 11:02 PM Jason Gustafson 
> > wrote:
> >
> > > Hey Manikumar,
> > >
> > > This looks good. Do we have to keep the current behavior when handling
> > old
> > > DeleteTopics versions? I'm wondering if it would be better to return an
> > > UNKNOWN_ERROR (say) and let the client raise that to the user instead
> of
> > > just timing out.
> > >
> > > Thanks,
> > > Jason
> > >
> > > On Fri, Jul 20, 2018 at 10:00 AM, Manikumar  >
> > > wrote:
> > >
> > > > Bumping this thread.
> > > >
> > > > On Tue, Jul 10, 2018 at 10:33 PM Manikumar <
> manikumar.re...@gmail.com>
> > > > wrote:
> > > >
> > > > > Waiting for one more binding vote to pass this minor KIP.
> Appreciate
> > > > your
> > > > > vote.
> > > > >
> > > > > On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska <
> eno.there...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> +1 (non binding)
> > > > >>
> > > > >> On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash <
> > > > >> kamal.chandraprak...@gmail.com> wrote:
> > > > >>
> > > > >> > +1 (non-binding)
> > > > >> >
> > > > >> > On Wed, Jul 4, 2018 at 5:22 PM Magnus Edenhill <
> > mag...@edenhill.se>
> > > > >> wrote:
> > > > >> >
> > > > >> > > +1 (non-binding)
> > > > >> > >
> > > > >> > > 2018-07-04 13:40 GMT+02:00 Satish Duggana <
> > > satish.dugg...@gmail.com
> > > > >:
> > > > >> > >
> > > > >> > > > +1
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > > Satish.
> > > > >> > > >
> > > > >> > > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione <
> > > > >> ascion...@gmail.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > +1
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > Daniele
> > > > >> > > > >
> > > > >> > > > > Il giorno mar 3 lug 2018 alle ore 23:55 Harsha <
> > > ka...@harsha.io
> > > > >
> > > > >> ha
> > > > >> > > > > scritto:
> > > > >> > > > >
> > > > >> > > > > > +1.
> > > > >> > > > > >
> > > > >> > > > > > Thanks,
> > > > >> > > > > > Harsha
> > > > >> > > > > >
> > > > >> > > > > > On Tue, Jul 3rd, 2018 at 9:22 AM, Ted Yu <
> > > yuzhih...@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > > +1
> > > > >> > > > > > >
> > > > >> > > > > > > On Tue, Jul 3, 2018 at 9:05 AM, Mickael Maison <
> > > > >> > > > > > mickael.mai...@gmail.com >
> > > > >> > > > > > >
> > > 

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

[Manikumar]

*Hi All,The vote has passed with 3 binding votes (Gwen,  Harsha, Jason) and
8 non-binding votes (Vahid, Mickael, Ted, Daniele, Satish, Magnus, Kamal,
Eno).Thanks everyone for the votes.Thanks,Manikumar*

On Sat, Jul 21, 2018 at 12:26 AM Jason Gustafson  wrote:

> Thanks, +1 from me.
>
> On Fri, Jul 20, 2018 at 11:41 AM, Manikumar 
> wrote:
>
> > Thanks for the feedback, sounds good to me. Updated the KIP.
> >
> > On Sat, Jul 21, 2018 at 12:04 AM Jason Gustafson 
> > wrote:
> >
> > > After thinking about it, perhaps a better option is INVALID_REQUEST? I
> > > think this is usually used for requests which are malformed in some
> way,
> > so
> > > it's a slight abuse, but at least it suggests an incompatibility
> between
> > > what the client is requesting to do and what the server supports.
> > > UNKNOWN_SERVER_ERROR, on the other hand, suggests an unexpected
> condition
> > > on the broker.
> > >
> > > -Jason
> > >
> > > On Fri, Jul 20, 2018 at 10:57 AM, Manikumar  >
> > > wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > Yes, we can return UNKNOWN_SERVER_ERROR  for older versions.
> > > > This will give better developer experience. Updated the KIP.
> > > >
> > > > Thanks,
> > > >
> > > > On Fri, Jul 20, 2018 at 11:02 PM Jason Gustafson  >
> > > > wrote:
> > > >
> > > > > Hey Manikumar,
> > > > >
> > > > > This looks good. Do we have to keep the current behavior when
> > handling
> > > > old
> > > > > DeleteTopics versions? I'm wondering if it would be better to
> return
> > an
> > > > > UNKNOWN_ERROR (say) and let the client raise that to the user
> instead
> > > of
> > > > > just timing out.
> > > > >
> > > > > Thanks,
> > > > > Jason
> > > > >
> > > > > On Fri, Jul 20, 2018 at 10:00 AM, Manikumar <
> > manikumar.re...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Bumping this thread.
> > > > > >
> > > > > > On Tue, Jul 10, 2018 at 10:33 PM Manikumar <
> > > manikumar.re...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Waiting for one more binding vote to pass this minor KIP.
> > > Appreciate
> > > > > > your
> > > > > > > vote.
> > > > > > >
> > > > > > > On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska <
> > > eno.there...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> +1 (non binding)
> > > > > > >>
> > > > > > >> On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash <
> > > > > > >> kamal.chandraprak...@gmail.com> wrote:
> > > > > > >>
> > > > > > >> > +1 (non-binding)
> > > > > > >> >
> > > > > > >> > On Wed, Jul 4, 2018 at 5:22 PM Magnus Edenhill <
> > > > mag...@edenhill.se>
> > > > > > >> wrote:
> > > > > > >> >
> > > > > > >> > > +1 (non-binding)
> > > > > > >> > >
> > > > > > >> > > 2018-07-04 13:40 GMT+02:00 Satish Duggana <
> > > > > satish.dugg...@gmail.com
> > > > > > >:
> > > > > > >> > >
> > > > > > >> > > > +1
> > > > > > >> > > >
> > > > > > >> > > > Thanks,
> > > > > > >> > > > Satish.
> > > > > > >> > > >
> > > > > > >> > > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione <
> > > > > > >> ascion...@gmail.com>
> > > > > > >> > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > +1
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks,
> > > > > > >> > > > > Daniele
> > > > > > >> &g

[DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi all,

I have created a KIP to use AdminClient API in AclCommand (kafka-acls.sh)

*https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API*


Please take a look.

Thanks,

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Manikumar]

is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Bumping this up!

On Mon, Jul 23, 2018 at 8:00 PM Manikumar  wrote:

> Hi all,
>
> I have created a KIP to use AdminClient API in AclCommand (kafka-acls.sh)
>
>
> *https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API*
>
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API>
>
> Please take a look.
>
> Thanks,
>

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi Colin,

Yes,  "--authorizer-properties" option is not required with
"--bootstrap-server" option. Updated the KIP.


Thanks,

On Tue, Jul 31, 2018 at 1:30 AM Ted Yu  wrote:

> Look good to me.
>
> On Mon, Jul 23, 2018 at 7:30 AM Manikumar 
> wrote:
>
> > Hi all,
> >
> > I have created a KIP to use AdminClient API in AclCommand (kafka-acls.sh)
> >
> > *
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API*
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API
> > >
> >
> > Please take a look.
> >
> > Thanks,
> >
>

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi all,

If there are no concerns, I will start the voting process soon.

Thanks

On Tue, Jul 31, 2018 at 9:08 AM Manikumar  wrote:

> Hi Colin,
>
> Yes,  "--authorizer-properties" option is not required with
> "--bootstrap-server" option. Updated the KIP.
>
>
> Thanks,
>
> On Tue, Jul 31, 2018 at 1:30 AM Ted Yu  wrote:
>
>> Look good to me.
>>
>> On Mon, Jul 23, 2018 at 7:30 AM Manikumar 
>> wrote:
>>
>> > Hi all,
>> >
>> > I have created a KIP to use AdminClient API in AclCommand
>> (kafka-acls.sh)
>> >
>> > *
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API*
>> > <
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API
>> > >
>> >
>> > Please take a look.
>> >
>> > Thanks,
>> >
>>
>

[VOTE] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi All,

I would like to start voting on KIP-332 which allows AclCommand to use
AdminClient API for acl management.

KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-332%3A+Update+AclCommand+to+use+AdminClient+API

Discussion Thread:
https://www.mail-archive.com/dev@kafka.apache.org/msg90049.html

Thanks,
Manikumar

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi Rajini,

Yes, I too prefer using  "--command-config" . Recently in one of the other
KIPs (KIP-340),  it was suggested to use
"--config-file", So I just followed the recent suggestion. But I think we
should use same name in all tools (at least in core tools).

If there are no concerns, I will change the option to   "--command-config".
Since KIP-340 PR is not yet merged, we can also change there.

Thanks,


On Fri, Aug 3, 2018 at 1:57 PM Rajini Sivaram 
wrote:

> Hi Manikumar,
>
> We have some tools already (ConfigCommand, ConsumerGroupCommand,
> DelegationTokenCommand) which use "--command-config" option to specify
> config file. Perhaps use should use the same name for AclCommand as well?
>
> On Thu, Aug 2, 2018 at 7:23 PM, Colin McCabe  wrote:
>
> > +1 for starting the vote
> >
> > cheers,
> > Colin
> >
> >
> > On Wed, Aug 1, 2018, at 08:46, Manikumar wrote:
> > > Hi all,
> > >
> > > If there are no concerns, I will start the voting process soon.
> > >
> > > Thanks
> > >
> > > On Tue, Jul 31, 2018 at 9:08 AM Manikumar 
> > wrote:
> > >
> > > > Hi Colin,
> > > >
> > > > Yes,  "--authorizer-properties" option is not required with
> > > > "--bootstrap-server" option. Updated the KIP.
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > On Tue, Jul 31, 2018 at 1:30 AM Ted Yu  wrote:
> > > >
> > > >> Look good to me.
> > > >>
> > > >> On Mon, Jul 23, 2018 at 7:30 AM Manikumar <
> manikumar.re...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi all,
> > > >> >
> > > >> > I have created a KIP to use AdminClient API in AclCommand
> > > >> (kafka-acls.sh)
> > > >> >
> > > >> > *
> > > >> >
> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 332%3A+Update+AclCommand+to+use+AdminClient+API*
> > > >> > <
> > > >> >
> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 332%3A+Update+AclCommand+to+use+AdminClient+API
> > > >> > >
> > > >> >
> > > >> > Please take a look.
> > > >> >
> > > >> > Thanks,
> > > >> >
> > > >>
> > > >
> >
>

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

[Manikumar]

Hi Dong,

In KIP-332 discussion, It was agreed to use "--command-config" option name
for passing config property file.
We can also use same name in here,  to make it consistent across all  tools.

Thanks,

On Thu, Jul 12, 2018 at 9:20 AM Dong Lin  wrote:

> wiki page is currently read-only and it is unavailable for write operation.
> Will update it later.
>
> On Wed, Jul 11, 2018 at 8:48 PM, Dong Lin  wrote:
>
> > Ah I see. Thanks for the suggestion. It is updated now.
> >
> > On Wed, Jul 11, 2018 at 8:13 PM, Ted Yu  wrote:
> >
> >> bq. the same approach used by "--config-file" in ConfigCommand.
> >>
> >> I should have copied more from the KIP.
> >> What I meant was that ConfigCommand doesn't use "--config-file" option.
> So
> >> 'same approach' implies StreamsResetter class, not ConfigCommand.
> >>
> >> I didn't mean to change ConfigCommand w.r.t. name of the option.
> >>
> >> Cheers
> >>
> >> On Wed, Jul 11, 2018 at 8:06 PM Dong Lin  wrote:
> >>
> >> > Do you mean we should replace "--command-config" with "--config-file"
> in
> >> > ConfigCommand? There is backward compatibility concern with the
> change.
> >> I
> >> > am not sure the benefit of this change is worth the effort to
> deprecate
> >> the
> >> > old config. Maybe we should do it separately if more people thing it
> is
> >> > necessary?
> >> >
> >> > On Wed, Jul 11, 2018 at 8:01 PM, Ted Yu  wrote:
> >> >
> >> > > bq. "--config-file" in ConfigCommand.
> >> > >
> >> > > Please update the above - it should be StreamsResetter
> >> > >
> >> > >
> >> > > On Wed, Jul 11, 2018 at 7:59 PM Dong Lin 
> wrote:
> >> > >
> >> > > > Hey Ted,
> >> > > >
> >> > > > Thanks much for the suggestion. Yeah "config-file" looks better
> than
> >> > > > "command-config". I have updated the KIP as suggested.
> >> > > >
> >> > > > Thanks,
> >> > > > Dong
> >> > > >
> >> > > > On Wed, Jul 11, 2018 at 5:57 PM, Ted Yu 
> >> wrote:
> >> > > >
> >> > > > > Looking at StreamsResetter.java :
> >> > > > >
> >> > > > >commandConfigOption = optionParser.accepts("config-file",
> >> > > > "Property
> >> > > > > file containing configs to be passed to admin cl
> >> > > > >
> >> > > > > Not sure you have considered naming the option in the above
> >> fashion.
> >> > > > >
> >> > > > > Probably add the above to Alternative section.
> >> > > > >
> >> > > > > Cheers
> >> > > > >
> >> > > > > On Wed, Jul 11, 2018 at 2:04 PM Dong Lin 
> >> > wrote:
> >> > > > >
> >> > > > > > Hi all,
> >> > > > > >
> >> > > > > > I have created KIP-340: Allow kafka-reassign-partitions.sh and
> >> > > > > > kafka-log-dirs.sh to take admin client property file. See
> >> > > > > >
> >> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> > > > > 340%3A+Allow+kafka-reassign-partitions.sh+and+kafka-log-
> >> > > > > dirs.sh+to+take+admin+client+property+file
> >> > > > > > .
> >> > > > > >
> >> > > > > > This KIP provides a way to allow kafka-reassign-partitions.sh
> >> and
> >> > > > > > kafka-log-dirs.sh to talk to broker over SSL. Please review
> the
> >> KIP
> >> > > if
> >> > > > > you
> >> > > > > > have time.
> >> > > > > >
> >> > > > > >
> >> > > > > > Thanks!
> >> > > > > > Dong
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Updated the KIP.  added a note to KIP-340 discussion thread.

On Fri, Aug 3, 2018 at 2:52 PM Rajini Sivaram 
wrote:

> Thanks Manikumar. Can you add a note to the KIP-340 discussion thread?
>
> On Fri, Aug 3, 2018 at 10:04 AM, Manikumar 
> wrote:
>
> > Hi Rajini,
> >
> > Yes, I too prefer using  "--command-config" . Recently in one of the
> other
> > KIPs (KIP-340),  it was suggested to use
> > "--config-file", So I just followed the recent suggestion. But I think we
> > should use same name in all tools (at least in core tools).
> >
> > If there are no concerns, I will change the option to
>  "--command-config".
> > Since KIP-340 PR is not yet merged, we can also change there.
> >
> > Thanks,
> >
> >
> > On Fri, Aug 3, 2018 at 1:57 PM Rajini Sivaram 
> > wrote:
> >
> > > Hi Manikumar,
> > >
> > > We have some tools already (ConfigCommand, ConsumerGroupCommand,
> > > DelegationTokenCommand) which use "--command-config" option to specify
> > > config file. Perhaps use should use the same name for AclCommand as
> well?
> > >
> > > On Thu, Aug 2, 2018 at 7:23 PM, Colin McCabe 
> wrote:
> > >
> > > > +1 for starting the vote
> > > >
> > > > cheers,
> > > > Colin
> > > >
> > > >
> > > > On Wed, Aug 1, 2018, at 08:46, Manikumar wrote:
> > > > > Hi all,
> > > > >
> > > > > If there are no concerns, I will start the voting process soon.
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Tue, Jul 31, 2018 at 9:08 AM Manikumar <
> manikumar.re...@gmail.com
> > >
> > > > wrote:
> > > > >
> > > > > > Hi Colin,
> > > > > >
> > > > > > Yes,  "--authorizer-properties" option is not required with
> > > > > > "--bootstrap-server" option. Updated the KIP.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > On Tue, Jul 31, 2018 at 1:30 AM Ted Yu 
> > wrote:
> > > > > >
> > > > > >> Look good to me.
> > > > > >>
> > > > > >> On Mon, Jul 23, 2018 at 7:30 AM Manikumar <
> > > manikumar.re...@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi all,
> > > > > >> >
> > > > > >> > I have created a KIP to use AdminClient API in AclCommand
> > > > > >> (kafka-acls.sh)
> > > > > >> >
> > > > > >> > *
> > > > > >> >
> > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 332%3A+Update+AclCommand+to+use+AdminClient+API*
> > > > > >> > <
> > > > > >> >
> > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 332%3A+Update+AclCommand+to+use+AdminClient+API
> > > > > >> > >
> > > > > >> >
> > > > > >> > Please take a look.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> >
> > > > > >>
> > > > > >
> > > >
> > >
> >
>

[DISCUSS] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

Hi all,

I have created a KIP to add delegation token operations to Java Admin
Client.
This KIP proposes new API additions to admin client. There are no new wire
protocol changes.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient


Feedback and suggestions are welcome.

Thanks
Manikumar

Re: [DISCUSS] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

Hi Ted,

Thanks for the review.

1.  default value for maxLifeTimeMs is -1. with this,  token MaxLifeTime
will default to a server side
 config value (delegation.token.max.lifetime.ms).  Added a comment to the
KIP.

2. Yes, DelegationTokenOwnerMismatchException will be thrown, updated the
KIP with possible exceptions.


Thanks

On Wed, Jan 17, 2018 at 6:45 AM, Jun Rao  wrote:

> Hi, Mani,
>
> Thanks for the KIP. Looks good to me overhead. Just a couple of minor
> comments below.
>
> 1. Should hmac be of type ByteBuffer? We return hmac as byte[] in
> DelegationToken.
> So, it seems it's more consistent to pass in hmac as byte[] too.
> 2. Does describeDelegationToken() return all tokens?
> 3. As Ted mentioned, it would be useful to include the exceptions that can
> be thrown in the new apis.
>
> Jun
>
> On Tue, Jan 16, 2018 at 10:03 AM, Manikumar 
> wrote:
>
> > Hi all,
> >
> > I have created a KIP to add delegation token operations to Java Admin
> > Client.
> > This KIP proposes new API additions to admin client. There are no new
> wire
> > protocol changes.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient
> >
> >
> > Feedback and suggestions are welcome.
> >
> > Thanks
> > Manikumar
> >
>

Re: [DISCUSS] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

Hi, Jun,

Thanks for the review.

1.  Yes,  We can pass hmac  as byte[]. Updated the KIP
2.  Yes,  describeDelegationToken() returns all the user owned tokens and
tokens where user have Describe permission.
 Added a comment to KIP.
3.  updated the KIP with possible exceptions.


Thanks,


On Wed, Jan 17, 2018 at 6:45 AM, Jun Rao  wrote:

> Hi, Mani,
>
> Thanks for the KIP. Looks good to me overhead. Just a couple of minor
> comments below.
>
> 1. Should hmac be of type ByteBuffer? We return hmac as byte[] in
> DelegationToken.
> So, it seems it's more consistent to pass in hmac as byte[] too.
> 2. Does describeDelegationToken() return all tokens?
> 3. As Ted mentioned, it would be useful to include the exceptions that can
> be thrown in the new apis.
>
> Jun
>
> On Tue, Jan 16, 2018 at 10:03 AM, Manikumar 
> wrote:
>
> > Hi all,
> >
> > I have created a KIP to add delegation token operations to Java Admin
> > Client.
> > This KIP proposes new API additions to admin client. There are no new
> wire
> > protocol changes.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient
> >
> >
> > Feedback and suggestions are welcome.
> >
> > Thanks
> > Manikumar
> >
>

Re: [DISCUSS] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

Hi all,

We want to include this KIP in the upcoming 1.1.0 release.
Please let me know if there are any other comments.

If there are no more comments, I'd like to start vote on this KIP.

Thanks,

On Wed, Jan 17, 2018 at 8:20 AM, Manikumar 
wrote:

> Hi, Jun,
>
> Thanks for the review.
>
> 1.  Yes,  We can pass hmac  as byte[]. Updated the KIP
> 2.  Yes,  describeDelegationToken() returns all the user owned tokens and
> tokens where user have Describe permission.
>  Added a comment to KIP.
> 3.  updated the KIP with possible exceptions.
>
>
> Thanks,
>
>
> On Wed, Jan 17, 2018 at 6:45 AM, Jun Rao  wrote:
>
>> Hi, Mani,
>>
>> Thanks for the KIP. Looks good to me overhead. Just a couple of minor
>> comments below.
>>
>> 1. Should hmac be of type ByteBuffer? We return hmac as byte[] in
>> DelegationToken.
>> So, it seems it's more consistent to pass in hmac as byte[] too.
>> 2. Does describeDelegationToken() return all tokens?
>> 3. As Ted mentioned, it would be useful to include the exceptions that can
>> be thrown in the new apis.
>>
>> Jun
>>
>> On Tue, Jan 16, 2018 at 10:03 AM, Manikumar 
>> wrote:
>>
>> > Hi all,
>> >
>> > I have created a KIP to add delegation token operations to Java Admin
>> > Client.
>> > This KIP proposes new API additions to admin client. There are no new
>> wire
>> > protocol changes.
>> >
>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > 249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient
>> >
>> >
>> > Feedback and suggestions are welcome.
>> >
>> > Thanks
>> > Manikumar
>> >
>>
>
>

[VOTE] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

Hi All,

I would like to start a vote on KIP-249 which would add delegation token
operations
to Java Admin Client.

We have merged DelegationToken API PR recently. We want to include admin
client changes in the upcoming release. This will make the feature complete.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient

Thanks,

Re: [VOTE] KIP-86: Configurable SASL callback handlers

[Manikumar]

Hi,

+1 (non-binding)

Thanks for the KIP.

On Wed, Jan 24, 2018 at 5:00 AM, Jun Rao  wrote:

> Hi, Rajini,
>
> Thanks for the KIP. +1 from me.
>
> Jun
>
> On Thu, Jan 18, 2018 at 8:58 AM, tao xiao  wrote:
>
> >  +1 (non-binding)
> >
> > On Fri, 19 Jan 2018 at 00:47 Rajini Sivaram 
> > wrote:
> >
> > > Hi all,
> > >
> > > I would like to restart the vote for KIP-86:
> > >https://cwiki.apache.org/confluence/display/KAFKA/KIP-86
> > > %3A+Configurable+SASL+callback+handlers
> > >
> > > The KIP makes callback handlers for SASL configurable to make it
> simpler
> > to
> > > integrate with custom authentication database or custom authentication
> > > servers. This is particularly useful for SASL/PLAIN where the
> > > implementation in Kafka based on credentials stored in jaas.conf is not
> > > suitable for production use. It is also useful for SCRAM in
> environments
> > > where ZooKeeper is not secure. The KIP has also been updated to
> simplify
> > > addition of new SASL mechanisms by making the Login class configurable.
> > >
> > > The PR for the KIP has been rebased and updated (
> > > https://github.com/apache/kafka/pull/2022)
> > >
> > > Thank you,
> > >
> > > Rajini
> > >
> > >
> > >
> > > On Mon, Dec 11, 2017 at 2:22 PM, Ted Yu  wrote:
> > >
> > > > +1
> > > >  Original message From: Tom Bentley <
> > > t.j.bent...@gmail.com>
> > > > Date: 12/11/17  6:06 AM  (GMT-08:00) To: dev@kafka.apache.org
> Subject:
> > > > Re: [VOTE] KIP-86: Configurable SASL callback handlers
> > > > +1 (non-binding)
> > > >
> > > > On 5 May 2017 at 11:57, Mickael Maison 
> > wrote:
> > > >
> > > > > Thanks for the KIP Rajini, this will significantly simplify
> providing
> > > > > custom credential providers
> > > > > +1 (non binding)
> > > > >
> > > > > On Wed, May 3, 2017 at 8:25 AM, Rajini Sivaram <
> > > rajinisiva...@gmail.com>
> > > > > wrote:
> > > > > > Can we have some more reviews or votes for this KIP to include in
> > > > > 0.11.0.0?
> > > > > > It is not a breaking change and the code is ready for
> integration,
> > so
> > > > it
> > > > > > will be good to get it in if possible.
> > > > > >
> > > > > > Ismael/Jun, since you had reviewed the KIP earlier, can you let
> me
> > > know
> > > > > if
> > > > > > I can do anything more to get your votes?
> > > > > >
> > > > > >
> > > > > > Thank you,
> > > > > >
> > > > > > Rajini
> > > > > >
> > > > > >
> > > > > > On Mon, Apr 10, 2017 at 12:18 PM, Edoardo Comar <
> eco...@uk.ibm.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> +1 (non binding)
> > > > > >> many thanks Rajini !
> > > > > >>
> > > > > >> --
> > > > > >> Edoardo Comar
> > > > > >> IBM MessageHub
> > > > > >> eco...@uk.ibm.com
> > > > > >> IBM UK Ltd, Hursley Park, SO21 2JN
> > > > > >>
> > > > > >> IBM United Kingdom Limited Registered in England and Wales with
> > > number
> > > > > >> 741598 Registered office: PO Box 41, North Harbour, Portsmouth,
> > > Hants.
> > > > > PO6
> > > > > >> 3AU
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> From:   Rajini Sivaram 
> > > > > >> To: dev@kafka.apache.org
> > > > > >> Date:   06/04/2017 10:53
> > > > > >> Subject:[VOTE] KIP-86: Configurable SASL callback
> handlers
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> Hi all,
> > > > > >>
> > > > > >> I would like to start the voting process for KIP-86:
> > > > > >>
> > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > >> 86%3A+Configurable+SASL+callback+handlers
> > > > > >>
> > > > > >>
> > > > > >> The KIP makes callback handlers for SASL configurable to make it
> > > > simpler
> > > > > >> to
> > > > > >> integrate with custom authentication database or custom
> > > authentication
> > > > > >> servers. This is particularly useful for SASL/PLAIN where the
> > > > > >> implementation in Kafka based on credentials stored in jaas.conf
> > is
> > > > not
> > > > > >> suitable for production use. It is also useful for SCRAM in
> > > > environments
> > > > > >> where ZooKeeper is not secure.
> > > > > >>
> > > > > >> Thank you...
> > > > > >>
> > > > > >> Regards,
> > > > > >>
> > > > > >> Rajini
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> Unless stated otherwise above:
> > > > > >> IBM United Kingdom Limited - Registered in England and Wales
> with
> > > > number
> > > > > >> 741598.
> > > > > >> Registered office: PO Box 41, North Harbour, Portsmouth,
> Hampshire
> > > PO6
> > > > > 3AU
> > > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] KIP-206: Add support for UUID serialization and deserialization

[Manikumar]

+1 (non-binding)

On Tue, Jan 30, 2018 at 11:50 AM, Ewen Cheslack-Postava 
wrote:

> +1 (binding)
>
> On Fri, Jan 26, 2018 at 9:16 AM, Colin McCabe  wrote:
>
> > +1 (non-binding)
> >
> >
> >
> > On Fri, Jan 26, 2018, at 08:29, Ted Yu wrote:
> > > +1
> > >
> > > On Fri, Jan 26, 2018 at 7:00 AM, Brandon Kirchner <
> > > brandon.kirch...@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I would like to (re)start the voting process for KIP-206:
> > > >
> > > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 206%3A+Add+support+for+UUID+serialization+and+deserialization
> > > >  > > > 206%3A+Add+support+for+UUID+serialization+and+deserialization>*
> > > >
> > > > The KIP adds a UUID serializer and deserializer. Possible
> > implementation
> > > > can be seen here --
> > > >
> > > > https://github.com/apache/kafka/pull/4438
> > > >
> > > > Original discussion and voting thread can be seen here --
> > > > http://search-hadoop.com/m/Kafka/uyzND1dlgePJY7l9?subj=+
> > > > DISCUSS+KIP+206+Add+support+for+UUID+serialization+and+
> deserialization
> > > >
> > > >
> > > > Thanks!
> > > > Brandon K.
> > > >
> >
>

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

[Manikumar]

Hi,

1. Do we support IPv6 CIDR/ranges?

2. KAFKA-5713 is mentioned in Related JIRAs section. But there is no
mention of wildcard support in the KIP.


Thanks,

On Thu, Feb 1, 2018 at 4:05 AM, Sönke Liebau <
soenke.lie...@opencore.com.invalid> wrote:

> Hey everybody,
>
> following a brief inital discussion a couple of days ago on this list
> I'd like to get a discussion going on KIP-252 which would allow
> specifying ip ranges and subnets for the -allow-host and --deny-host
> parameters of the acl tool.
>
> The KIP can be found at
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 252+-+Extend+ACLs+to+allow+filtering+based+on+ip+ranges+and+subnets
>
> Best regards,
> Sönke
>

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

[Manikumar]

Hi,

They are few deployments using IPv6.  It is good to support IPv6 also.

I think KAFKA-5713 is about adding regular expression support to resource
names (topic. consumer etc..).
Yes, wildcards (*) in hostname doesn't makes sense. Range and subnet
support will give us the flexibility.

On Thu, Feb 1, 2018 at 5:56 PM, Sönke Liebau <
soenke.lie...@opencore.com.invalid> wrote:

> Hi Manikumar,
>
> the current proposal indeed leaves out IPv6 addresses, as I was unsure
> whether Kafka fully supports that yet to be honest. But it would be
> fairly easy to add these to the proposal - I'll update it over the
> weekend.
>
> Regarding KAFKA-5713, I simply listed it as related, since it is
> similar in spirit, if not exact wording.  Parts of that issue
> (wildcards in hosts) would be covered by this kip - just in a slightly
> different way. Do we really need wildcard support in IP addresses if
> we can specify ranges and subnets? I considered it, but only came up
> with scenarios that seemed fairly academic to me, like allowing the
> same host from multiple subnets (10.0.*.1) for example.
>
> Allowing wildcards has the potential to make the code more complex,
> depending on how we decide to implement this feature, hance I decided
> to leave wildcards out for now.
>
> What do you think?
>
> Best regards,
> Sönke
>
> On Thu, Feb 1, 2018 at 10:14 AM, Manikumar 
> wrote:
> > Hi,
> >
> > 1. Do we support IPv6 CIDR/ranges?
> >
> > 2. KAFKA-5713 is mentioned in Related JIRAs section. But there is no
> > mention of wildcard support in the KIP.
> >
> >
> > Thanks,
> >
> > On Thu, Feb 1, 2018 at 4:05 AM, Sönke Liebau <
> > soenke.lie...@opencore.com.invalid> wrote:
> >
> >> Hey everybody,
> >>
> >> following a brief inital discussion a couple of days ago on this list
> >> I'd like to get a discussion going on KIP-252 which would allow
> >> specifying ip ranges and subnets for the -allow-host and --deny-host
> >> parameters of the acl tool.
> >>
> >> The KIP can be found at
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> 252+-+Extend+ACLs+to+allow+filtering+based+on+ip+ranges+and+subnets
> >>
> >> Best regards,
> >> Sönke
> >>
>
>
>
> --
> Sönke Liebau
> Partner
> Tel. +49 179 7940878
> OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
>

Re: Kafka Log deletion Problem

[Manikumar]

looks like log segment is not rotated.  You can send more data or adjust
log.roll.ms/log.segment.bytes configs to rotate segments.


On Fri, Feb 2, 2018 at 7:08 PM, SenthilKumar K 
wrote:

> Hello Experts , We have a Kafka Setup running for our analytics pipeline
> ...Below is the broker config ..
>
> max.message.bytes = 67108864
> replica.fetch.max.bytes = 67108864
> zookeeper.session.timeout.ms = 7000
> replica.socket.timeout.ms = 3
> offsets.commit.timeout.ms = 5000
> request.timeout.ms = 4
> zookeeper.connection.timeout.ms = 7000
> controller.socket.timeout.ms = 3
> num.partitions = 24
> listeners = SSL://23.212.237.10:9093
> broker.id = 1
> socket.receive.buffer.bytes = 102400
> message.max.bytes = 2621440
> auto.create.topics.enable = true
> auto.leader.rebalance.enable = true
> zookeeper.connect = zk1:2181,zk2:2181
> log.retention.ms=8640
> #log.retention.hours = 24
> socket.request.max.bytes = 104857600
> default.replication.factor = 2
> log.dirs = /data/kafka_logs
> compression.codec = 3
>
> Kafka Version : *0.11*
>
> The retention period is set to 24 Hours , but i could see the data in disk
> after 24 hours.. *What could be the problem here ?*
>
> Note : There is no topic specific configuration.
>
> --Senthil
>

Re: [VOTE] 1.0.1 RC2

[Manikumar]

+1 (non-binding)
Built src and ran tests
Ran core quick start

On Sat, Feb 24, 2018 at 8:44 PM, Jakub Scholz  wrote:

> +1 (non-binding) ... I used the Scala 2.12 binaries and run my tests with
> producers / consumers.
>
> On Thu, Feb 22, 2018 at 1:06 AM, Ewen Cheslack-Postava 
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the third candidate for release of Apache Kafka 1.0.1.
> >
> > This is a bugfix release for the 1.0 branch that was first released with
> > 1.0.0 about 3 months ago. We've fixed 49 issues since that release. Most
> of
> > these are non-critical, but in aggregate these fixes will have
> significant
> > impact. A few of the more significant fixes include:
> >
> > * KAFKA-6277: Make loadClass thread-safe for class loaders of Connect
> > plugins
> > * KAFKA-6185: Selector memory leak with high likelihood of OOM in case of
> > down conversion
> > * KAFKA-6269: KTable state restore fails after rebalance
> > * KAFKA-6190: GlobalKTable never finishes restoring when consuming
> > transactional messages
> > * KAFKA-6529: Stop file descriptor leak when client disconnects with
> staged
> > receives
> > * KAFKA-6238: Issues with protocol version when applying a rolling
> upgrade
> > to 1.0.0
> >
> > Release notes for the 1.0.1 release:
> > http://home.apache.org/~ewencp/kafka-1.0.1-rc2/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Saturday Feb 24, 9pm PT ***
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~ewencp/kafka-1.0.1-rc2/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~ewencp/kafka-1.0.1-rc2/javadoc/
> >
> > * Tag to be voted upon (off 1.0 branch) is the 1.0.1 tag:
> > https://github.com/apache/kafka/tree/1.0.1-rc2
> >
> > * Documentation:
> > http://kafka.apache.org/10/documentation.html
> >
> > * Protocol:
> > http://kafka.apache.org/10/protocol.html
> >
> > /**
> >
> > Thanks,
> > Ewen Cheslack-Postava
> >
>

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

[Manikumar]

Can we retain total RequestsPerSec metric and add new version tag metric?
When monitoring with simple jconsole/jmx based tools, It is useful to have
total metric
to monitor request rate.


Thanks,

On Wed, Mar 21, 2018 at 11:01 AM, Gwen Shapira  wrote:

> I love this. Not much to add - it is an elegant solution, clean
> implementation and it addresses a real need, especially during upgrades.
>
> On Tue, Mar 20, 2018 at 2:49 PM, Ted Yu  wrote:
>
> > Thanks for the response.
> >
> > Assuming number of client versions is limited in a cluster, memory
> > consumption is not a concern.
> >
> > Cheers
> >
> > On Tue, Mar 20, 2018 at 10:47 AM, Allen Wang 
> wrote:
> >
> > > Hi Ted,
> > >
> > > The additional hash map is very small, possibly a few KB. Each request
> > type
> > > ("produce", "fetch", etc.) will have such a map which have a few
> entries
> > > depending on the client API versions the broker will encounter. So if
> > > broker encounters two client versions for "produce", there will be two
> > > entries in the map for "produce" requests mapping from version to
> meter.
> > Of
> > > course, hash map always have additional memory overhead.
> > >
> > > Thanks,
> > > Allen
> > >
> > >
> > > On Mon, Mar 19, 2018 at 3:49 PM, Ted Yu  wrote:
> > >
> > > > bq. *additional hash lookup is needed when updating the metric to
> > locate
> > > > the metric *
> > > >
> > > > *Do you have estimate how much memory is needed for maintaining the
> > hash
> > > > map ?*
> > > >
> > > > *Thanks*
> > > >
> > > > On Mon, Mar 19, 2018 at 3:19 PM, Allen Wang 
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I have created KIP-272: Add API version tag to broker's
> > RequestsPerSec
> > > > > metric.
> > > > >
> > > > > Here is the link to the KIP:
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 272%3A+Add+API+version+tag+to+broker%27s+RequestsPerSec+metric
> > > > >
> > > > > Looking forward to the discussion.
> > > > >
> > > > > Thanks,
> > > > > Allen
> > > > >
> > > >
> > >
> >
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter  | blog
> 
>

Re: Producer timeout and retry behavior

[Manikumar]

Hi,

a) If there are no brokers available, producer send() will block up to
max_block_ms (default: 6)
after which it will throw a TimeoutException. No retries in this
case. by-default
console producer
uses Callback and just logs the error after max_block_ms.

b) Current behavior is, If metadata is not available and producer batches
will Timedout after request.timeout.ms.
retries are applicable to failed produce requests with
transient/retriable exceptions.


Thanks,


On Tue, Mar 27, 2018 at 3:56 PM, Sandor Murakozi 
wrote:

> Hi All,
>
> I have a question about intended producer behavior if the broker is lost -
> do I see a bug or the code works as specified?
>
> What I do and see using trunk:
>
> *a) No message send timeout at all if there is no available broker*
> - no broker is started
> - consoleproducer is started using --broker-list localhost:9092 --topic
> test --request-timeout-ms 2000 --property retries=3
> - I try to send some messages. The request doesn't time out, I just see an
> endless amount of log entries like this:
> [2018-03-27 10:04:42,084] WARN [Producer clientId=console-producer]
> Connection to node -1 could not be established. Broker may not be
> available. (org.apache.kafka.clients.NetworkClient)
>
> I would expect that the producer eventually times out, preferably retrying
> sending 3 times.
>
> *b) No retries if the last broker disappears after successful
> communication*
> - broker is started
> - consoleproducer is started with the same config as in a)
> - 1 message is sent successfully
> - broker is stopped - producer starts to add log entries like
> [2018-03-27 10:09:11,597] WARN [Producer clientId=console-producer]
> Connection to node 0 could not be established. Broker may not be available.
> (org.apache.kafka.clients.NetworkClient)
> - I try to send another message
> - it times out after ~2 seconds
> [2018-03-27 10:09:17,442] ERROR Error when sending message to topic test
> with key: null, value: 1 bytes with error:
> (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> test-0: 2018 ms has passed since batch creation plus linger time
> - the producer *does not retry* sending
>
> Some debugging have shown that after the connection was broken the producer
> will attempt to fetch metadata. As there are no more available brokers this
> will not be possible.
> In this case it does not retry fetching of the metadata but it considers
> the batch expired.
>
> My expectation would be that the producer times out but it retries sending.
>
> I've spent quite some time trying to derive what's the correct behavior
> according to KIP-19
>  19+-+Add+a+request+timeout+to+NetworkClient>
> and
> jiras/pull requests related to the topic (e.g. KAFKA-2805
>  and KAFKA-3388
> ) but it's still not
> 100%
> clear to me if the code works as intended.
>
> Could someone help me decide this question?
>
> Many thanks in advance,
> Sandor
>

Re: [kafka-clients] Re: [VOTE] 1.1.0 RC4

[Manikumar]

+1 (non-binding)

- Verified src, binary artifacts and basic quick start
- Verified delegation token operations and docs
- Verified dynamic broker configuration and docs.


On Tue, Mar 27, 2018 at 6:52 PM, Rajini Sivaram 
wrote:

> Can we get some more votes for this RC so that the release can be rolled
> out soon?
>
> Many thanks,
>
> Rajini
>
> On Sat, Mar 24, 2018 at 6:54 PM, Ted Yu  wrote:
>
>> I wasn't able to reproduce the test failure when it is run alone.
>>
>> This seems to be flaky test.
>>
>> +1 from me.
>>
>> On Sat, Mar 24, 2018 at 11:49 AM, Rajini Sivaram > >
>> wrote:
>>
>> > Hi Ted,
>> >
>> > Thank you for testing the RC. I haven't been able to recreate that
>> failure
>> > after running the test a 100 times. Was it a one-off transient failure
>> or
>> > does it fail consistently for you?
>> >
>> >
>> > On Sat, Mar 24, 2018 at 2:51 AM, Ted Yu  wrote:
>> >
>> > > When I ran test suite, I got one failure:
>> > >
>> > > kafka.api.PlaintextConsumerTest > testAsyncCommit FAILED
>> > > java.lang.AssertionError: expected:<5> but was:<1>
>> > > at org.junit.Assert.fail(Assert.java:88)
>> > > at org.junit.Assert.failNotEquals(Assert.java:834)
>> > > at org.junit.Assert.assertEquals(Assert.java:645)
>> > > at org.junit.Assert.assertEquals(Assert.java:631)
>> > > at
>> > > kafka.api.BaseConsumerTest.awaitCommitCallback(
>> > BaseConsumerTest.scala:214)
>> > > at
>> > > kafka.api.PlaintextConsumerTest.testAsyncCommit(
>> > > PlaintextConsumerTest.scala:513)
>> > >
>> > > Not sure if anyone else saw similar error.
>> > >
>> > > Cheers
>> > >
>> > > On Fri, Mar 23, 2018 at 4:37 PM, Rajini Sivaram <
>> rajinisiva...@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > Hello Kafka users, developers and client-developers,
>> > > >
>> > > > This is the fifth candidate for release of Apache Kafka 1.1.0.
>> > > >
>> > > > https://cwiki.apache.org/confluence/pages/viewpage.
>> > > action?pageId=75957546
>> > > >
>> > > > A few highlights:
>> > > >
>> > > > * Significant Controller improvements (much faster and session
>> > expiration
>> > > > edge
>> > > > cases fixed)
>> > > > * Data balancing across log directories (JBOD)
>> > > > * More efficient replication when the number of partitions is large
>> > > > * Dynamic Broker Configs
>> > > > * Delegation tokens (KIP-48)
>> > > > * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
>> > > >
>> > > > Release notes for the 1.1.0 release:
>> > > >
>> > > > http://home.apache.org/~rsivaram/kafka-1.1.0-rc4/RELEASE_NOTES.html
>> > > >
>> > > >
>> > > > *** Please download, test and vote by Tuesday March 27th 4pm PT.
>> > > >
>> > > >
>> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > > >
>> > > > http://kafka.apache.org/KEYS
>> > > >
>> > > >
>> > > > * Release artifacts to be voted upon (source and binary):
>> > > >
>> > > > http://home.apache.org/~rsivaram/kafka-1.1.0-rc4/
>> > > >
>> > > >
>> > > > * Maven artifacts to be voted upon:
>> > > >
>> > > > https://repository.apache.org/content/groups/staging/
>> > > >
>> > > >
>> > > > * Javadoc:
>> > > >
>> > > > http://home.apache.org/~rsivaram/kafka-1.1.0-rc4/javadoc/
>> > > >
>> > > >
>> > > > * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
>> > > >
>> > > > https://github.com/apache/kafka/tree/1.1.0-rc4
>> > > >
>> > > >
>> > > >
>> > > > * Documentation:
>> > > >
>> > > > http://kafka.apache.org/11/documentation.html
>> > > >
>> > > >
>> > > > * Protocol:
>> > > >
>> > > > http://kafka.apache.org/11/protocol.html
>> > > >
>> > > >
>> > > >
>> > > > Thanks,
>> > > >
>> > > >
>> > > > Rajini
>> > > >
>> > >
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscr...@googlegroups.com.
> To post to this group, send email to kafka-clie...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/kafka-clients/CAOJcB3_6Eyr9knQZ6Sg%2BqyDbAizyqTNcbR%
> 3D7R_%2BA%2BD8c_VWeKg%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

Re: [ANNOUNCE] Apache Kafka 1.1.0 Released

[Manikumar]

> >>
> >>
> >>
> >> ** The Streams API allows an application to act as a stream processor,
> >>
> >> consuming an input stream from one or more topics and producing an
> output
> >>
> >> stream to one or more output topics, effectively transforming the input
> >>
> >> streams to output streams.
> >>
> >>
> >>
> >> ** The Connector API allows building and running reusable producers or
> >>
> >> consumers that connect Kafka topics to existing applications or data
> >>
> >> systems. For example, a connector to a relational database might capture
> >>
> >> every change to a table.three key capabilities:
> >>
> >>
> >>
> >>
> >> With these APIs, Kafka can be used for two broad classes of application:
> >>
> >> ** Building real-time streaming data pipelines that reliably get data
> >>
> >> between systems or applications.
> >>
> >>
> >>
> >> ** Building real-time streaming applications that transform or react to
> the
> >>
> >> streams of data.
> >>
> >>
> >>
> >>
> >> Apache Kafka is in use at large and small companies worldwide, including
> >>
> >> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> >>
> >> Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >>
> >>
> >>
> >>
> >> A big thank you for the following 120 contributors to this release!
> >>
> >>
> >> Adem Efe Gencer, Alex Good, Andras Beni, Andy Bryant, Antony Stubbs,
> >>
> >> Apurva Mehta, Arjun Satish, bartdevylder, Bill Bejeck, Charly Molter,
> >>
> >> Chris Egerton, Clemens Valiente, cmolter, Colin P. Mccabe,
> >>
> >> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
> >>
> >> Daniel Wojda, Derrick Or, Dmitry Minkovsky, Dong Lin, Edoardo Comar,
> >>
> >> ekenny, Elyahou, Eugene Sevastyanov, Ewen Cheslack-Postava, Filipe
> Agapito,
> >>
> >> fredfp, Gavrie Philipson, Gunnar Morling, Guozhang Wang, hmcl, Hugo
> Louro,
> >>
> >> huxi, huxihx, Igor Kostiakov, Ismael Juma, Ivan Babrou, Jacek Laskowski,
> >>
> >> Jakub Scholz, Jason Gustafson, Jeff Klukas, Jeff Widman, Jeremy
> >> Custenborder,
> >>
> >> Jeyhun Karimov, Jiangjie (Becket) Qin, Jiangjie Qin, Jimin Hsieh, Joel
> >> Hamill,
> >>
> >> John Roesler, Jorge Quilcate Otoya, Jun Rao, Kamal C, Kamil Szymański,
> >>
> >> Koen De Groote, Konstantine Karantasis, lisa2lisa, Logan Buckley,
> >>
> >> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> >> manjuapu,
> >>
> >> Manjula K, Mats Julian Olsen, Matt Farmer, Matthias J. Sax,
> >>
> >> Matthias Wessendorf, Max Zheng, Maytee Chinavanichkit, Mickael Maison,
> >> Mikkin,
> >>
> >> mulvenna, Narendra kumar, Nick Chiu, Onur Karaman, Panuwat
> Anawatmongkhon,
> >>
> >> Paolo Patierno, parafiend, ppatierno, Prasanna Gautam, Radai Rosenblatt,
> >>
> >> Rajini Sivaram, Randall Hauch, Richard Yu, RichardYuSTUG, Robert Yokota,
> >>
> >> Rohan, Rohan Desai, Romain Hardouin, Ron Dagostino, sachinbhalekar,
> >>
> >> Sagar Chavan, Sandor Murakozi, Satish Duggana, Scott, Sean McCauliff,
> >>
> >> Siva Santhalingam, siva santhalingam, Soenke Liebau, Steven Aerts,
> Study,
> >>
> >> Tanvi Jaywant, tedyu, Tobias Gies, Tom Bentley, Tommy Becker, Travis
> >> Wellman,
> >>
> >> umesh chaudhary, Vahid Hashemian, Viktor Somogyi, Wladimir Schmidt,
> >>
> >> wushujames, Xavier Léauté, Xin Li, Yaswanth Kumar, ying-zheng, Yu,
> Yu-Jhe
> >>
> >>
> >>
> >> Many thanks to Damian Guy for driving this release.
> >>
> >>
> >> We welcome your help and feedback. For more information on how to
> >>
> >> report problems, and to get involved, visit the project website at
> >>
> >> http://kafka.apache.org/
> >>
> >>
> >>
> >> Thank you!
> >>
> >>
> >> Rajini
>
>

Re: [VOTE] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

I'm bumping this up to get some attention.


On Wed, Jan 24, 2018 at 3:36 PM, Satish Duggana 
wrote:

>  +1, thanks for the KIP.
>
> ~Satish.
>
> On Wed, Jan 24, 2018 at 5:09 AM, Jun Rao  wrote:
>
> > Hi, Mani,
> >
> > Thanks for the KIP. +1
> >
> > Jun
> >
> > On Sun, Jan 21, 2018 at 7:44 AM, Manikumar 
> > wrote:
> >
> > > Hi All,
> > >
> > > I would like to start a vote on KIP-249 which would add delegation
> token
> > > operations
> > > to Java Admin Client.
> > >
> > > We have merged DelegationToken API PR recently. We want to include
> admin
> > > client changes in the upcoming release. This will make the feature
> > > complete.
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient
> > >
> > > Thanks,
> > >
> >
>

Re: [VOTE] KIP-272: Add API version tag to broker's RequestsPerSec metric

[Manikumar]

+1 (non-binding)

On Thu, Mar 29, 2018 at 6:16 PM, Thomas Crayford 
wrote:

> +1 (non-binding)
>
> On Wed, Mar 28, 2018 at 9:15 PM, Ted Yu  wrote:
>
> > +1
> >
> > On Wed, Mar 28, 2018 at 12:05 PM, Mickael Maison <
> mickael.mai...@gmail.com
> > >
> > wrote:
> >
> > > +1 (non binding)
> > > Thanks for the KIP
> > >
> > > On Wed, Mar 28, 2018 at 6:25 PM, Gwen Shapira 
> wrote:
> > > > +1 (binding)
> > > >
> > > > On Wed, Mar 28, 2018 at 9:55 AM, Allen Wang 
> > > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> I would like to start voting for KIP-272:  Add API version tag to
> > > broker's
> > > >> RequestsPerSec metric.
> > > >>
> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > >> 272%3A+Add+API+version+tag+to+broker%27s+RequestsPerSec+metric
> > > >>
> > > >> Thanks,
> > > >> Allen
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Gwen Shapira*
> > > > Product Manager | Confluent
> > > > 650.450.2760 | @gwenshap
> > > > Follow us: Twitter  | blog
> > > > 
> > >
> >
>

Re: [VOTE] KIP-249: Add Delegation Token Operations to Kafka Admin Client

[Manikumar]

Hi All,

The vote has passed with 3 binding votes (Jun, Gwen, Rajini) and 2
non-binding votes (Satish, Viktor).

Thanks everyone for the votes.

Thanks,
Manikumar

On Tue, Apr 3, 2018 at 2:46 PM, Rajini Sivaram 
wrote:

> +1 (binding)
>
> Thanks for the KIP, Manikumar!
>
> On Thu, Mar 29, 2018 at 5:34 PM, Gwen Shapira  wrote:
>
> > +1
> >
> > Thank you and sorry for missing it the first time around.
> >
> > On Thu, Mar 29, 2018 at 3:05 AM, Manikumar 
> > wrote:
> >
> > > I'm bumping this up to get some attention.
> > >
> > >
> > > On Wed, Jan 24, 2018 at 3:36 PM, Satish Duggana <
> > satish.dugg...@gmail.com>
> > > wrote:
> > >
> > > >  +1, thanks for the KIP.
> > > >
> > > > ~Satish.
> > > >
> > > > On Wed, Jan 24, 2018 at 5:09 AM, Jun Rao  wrote:
> > > >
> > > > > Hi, Mani,
> > > > >
> > > > > Thanks for the KIP. +1
> > > > >
> > > > > Jun
> > > > >
> > > > > On Sun, Jan 21, 2018 at 7:44 AM, Manikumar <
> > manikumar.re...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I would like to start a vote on KIP-249 which would add
> delegation
> > > > token
> > > > > > operations
> > > > > > to Java Admin Client.
> > > > > >
> > > > > > We have merged DelegationToken API PR recently. We want to
> include
> > > > admin
> > > > > > client changes in the upcoming release. This will make the
> feature
> > > > > > complete.
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > 249%3A+Add+Delegation+Token+Operations+to+KafkaAdminClient
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > <http://www.confluent.io/blog>
> >
>

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

[Manikumar]

Hi,

Substitution mechanism can be useful to configure regular password configs
liken ssl.keystore.password, ssl.truststore.password, etc.
This is can be good alternative to previously proposed KIP-76 and will give
more options to the user.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-
76+Enable+getting+password+from+executable+rather+than+
passing+as+plaintext+in+config+files


Thanks,

On Fri, Apr 6, 2018 at 4:29 AM, Rajini Sivaram 
wrote:

> Hi Ron,
>
> For the password example, you could define a login CallbackHandler that
> processes PasswordCallback to provide passwords. We don't currently do this
> with PLAIN/SCRAM because login callback handlers were not configurable
> earlier and we haven't updated the login modules to do this. But that could
> be one way of providing passwords and integrating with other password
> sources, now that we have configurable login callback handlers. I was
> wondering whether similar approach could be used for the parameters that
> OAuth needed to obtain at runtime. We could still have this KIP with
> built-in substitutable types to handle common cases like getting options
> from a file without writing any code. But I wasn't sure if there were OAuth
> options that couldn't be handled as callbacks using the login callback
> handler.
>
> On Thu, Apr 5, 2018 at 10:25 PM, Ron Dagostino  wrote:
>
> > Hi Rajini.  Thanks for the questions.  I could see someone wanting to
> > retrieve a password from a vended password vault solution (for example);
> > that is the kind of scenario that the ability to add new substitutable
> > types would be meant for.  I do still consider this KIP 269 to be a
> > prerequisite for the SASL/OAUTHBEARER KIP 255.  I am open to a different
> > perspective in case I missed or misunderstood your point.
> >
> > Ron
> >
> > On Thu, Apr 5, 2018 at 8:13 AM, Rajini Sivaram 
> > wrote:
> >
> > > Hi Ron,
> > >
> > > Now that login callback handlers are configurable, is this KIP still a
> > > pre-req for OAuth? I was wondering whether we still need the ability to
> > add
> > > new substitutable types or whether it would be sufficient to add the
> > > built-in ones to read from file etc.
> > >
> > >
> > > On Thu, Mar 29, 2018 at 6:48 AM, Ron Dagostino 
> > wrote:
> > >
> > > > Hi everyone.  There have been no comments on this KIP, so I intend to
> > put
> > > > it to a vote next week if there are no comments that might entail
> > changes
> > > > between now and then.  Please take a look in the meantime if you
> wish.
> > > >
> > > > Ron
> > > >
> > > > On Thu, Mar 15, 2018 at 2:36 PM, Ron Dagostino 
> > > wrote:
> > > >
> > > > > Hi everyone.
> > > > >
> > > > > I created KIP-269: Substitution Within Configuration Values
> > > > >  > > > 269+Substitution+Within+Configuration+Values>
> > > > > (https://cwiki.apache.org/confluence/display/KAFKA/KIP+269+
> > > > > Substitution+Within+Configuration+Values
> > > > >  > > > action?pageId=75968876>
> > > > > ).
> > > > >
> > > > > This KIP proposes adding support for substitution within client
> JAAS
> > > > > configuration values for PLAIN and SCRAM-related SASL mechanisms
> in a
> > > > > backwards-compatible manner and making the functionality available
> to
> > > > other
> > > > > existing (or future) configuration contexts where it is deemed
> > > > appropriate.
> > > > >
> > > > > This KIP was extracted from (and is now a prerequisite for)
> KIP-255:
> > > > > OAuth Authentication via SASL/OAUTHBEARER
> > > > >  > > > action?pageId=75968876>
> > > > > based on discussion of that KIP.
> > > > >
> > > > > Ron
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

[Manikumar]

+1 (non-binding)

Thanks for the detailed KIP.

On Fri, Apr 6, 2018 at 8:54 AM, Guozhang Wang  wrote:

> Thanks for the KIP!
>
> +1 (binding)
>
> On Thu, Apr 5, 2018 at 3:11 PM, Matthias J. Sax 
> wrote:
>
> > +1 (binding)
> >
> >
> > -Matthias
> >
> > On 4/5/18 4:36 AM, Ted Yu wrote:
> > > +1
> > >  Original message From: Mickael Maison <
> > mickael.mai...@gmail.com> Date: 4/5/18  1:42 AM  (GMT-08:00) To: dev <
> > dev@kafka.apache.org> Subject: Re: [VOTE] KIP-211: Revise Expiration
> > Semantics of Consumer Group Offsets
> > > +1 (non-binding)
> > > Thanks for the KIP!
> > >
> > > On Thu, Apr 5, 2018 at 8:08 AM, Jason Gustafson 
> > wrote:
> > >> +1 Thanks Vahid!
> > >>
> > >> On Wed, Mar 28, 2018 at 7:27 PM, James Cheng 
> > wrote:
> > >>
> > >>> +1 (non-binding)
> > >>>
> > >>> Thanks for all the hard work on this, Vahid!
> > >>>
> > >>> -James
> > >>>
> >  On Mar 28, 2018, at 10:34 AM, Vahid S Hashemian <
> > >>> vahidhashem...@us.ibm.com> wrote:
> > 
> >  Hi all,
> > 
> >  As I believe the feedback and suggestions on this KIP have been
> > addressed
> >  so far, I'd like to start a vote.
> >  The KIP can be found at
> >  https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>> 211%3A+Revise+Expiration+Semantics+of+Consumer+Group+Offsets
> > 
> >  Thanks in advance for voting :)
> > 
> >  --Vahid
> > 
> > >>>
> > >>>
> >
> >
>
>
> --
> -- Guozhang
>

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

[Manikumar]

lso need to be concerned about valid
> > > > > > substitutions
> > > > > > > >>> that
> > > > > > > >>>weren't intended to be substitutions? I am thinking
> that we
> > > > > don't
> > > > > > > >>> need to
> > > > > > > >>>worry about the latter if substitutions are only by
> opt-in.
> > > > > > > >>>- It will be good to get more feedback on this KIP
> before
> > > > > updating
> > > > > > > the
> > > > > > > >>>code to use it for all configs since the code may need
> to
> > > change
> > > > > > > >>> quite a
> > > > > > > >>>bit to fit in with the config classes.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> For the callbacks, I agree that we want a LoginModule for
> OAuth
> > > > > that
> > > > > > > can
> > > > > > > >>> be
> > > > > > > >>> reused. But to use OAuth, you will probably have your own
> > > callback
> > > > > > > >>> handler
> > > > > > > >>> implementation to process OAuthBearerLoginCallback . From
> the
> > > > > > example,
> > > > > > > it
> > > > > > > >>> is not clear to me why the callback handler that processes
> > > > > > > >>> OAuthBearerLoginCallback cannot do whatever a custom
> > > substitution
> > > > > > class
> > > > > > > >>> would do, e,g. read some options like passwordVaultUrl
> from the
> > > > > JAAS
> > > > > > > >>> config
> > > > > > > >>> (which it has access to) and retrieve passwords from a
> password
> > > > > > vault?
> > > > > > > If
> > > > > > > >>> we are going to have extensible substitution anyway, then
> > > > > obviously,
> > > > > > we
> > > > > > > >>> could use that as an option here too.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> On Fri, Apr 6, 2018 at 2:47 PM, Ron Dagostino <
> > > rndg...@gmail.com>
> > > > > > > wrote:
> > > > > > > >>>
> > > > > > > >>> > Hi folks.  I think there are a couple of issues that were
> > > just
> > > > > > raised
> > > > > > > >>> in
> > > > > > > >>> > this thread.  One is whether the ability to use
> > > PasswordCallback
> > > > > > > >>> exists,
> > > > > > > >>> > and if so whether that impacts the applicability of this
> KIP
> > > to
> > > > > the
> > > > > > > >>> > SASL/OAUTHBEARER KIP-255.  The second issue is related to
> > > how we
> > > > > > > might
> > > > > > > >>> > leverage this KIP more broadly (including as an
> alternative
> > > to
> > > > > > > KIP-76)
> > > > > > > >>> > while maintaining forward compatibility and not causing
> > > > > unexpected
> > > > > > > >>> > substitutions/parsing exceptions.
> > > > > > > >>> >
> > > > > > > >>> > Let me address the second issue (more broad use) first,
> > > since I
> > > > > > think
> > > > > > > >>> > Rajini hit on a good possibility.  Currently this KIP
> > > addresses
> > > > > the
> > > > > > > >>> > possibility of an unexpected substitution by saying "This
> > > would
> > > > > > > cause a
> > > > > > > >>> > substitution to be attempted, which of course would fail
> and
> > > > > raise
> > > > > > an
> > > > > > > >>> > exception."  I think Rajini's idea is to explicitly state
> > > that
> > > > &g

Re: [VOTE] Kafka 2.0.0 in June 2018

[Manikumar]

+1 (non-binding).

Thanks.

On Thu, Apr 19, 2018 at 3:07 PM, Stephane Maarek <
steph...@simplemachines.com.au> wrote:

> +1 (non binding). Thanks Ismael!
>
> On Thu., 19 Apr. 2018, 2:47 pm Gwen Shapira,  wrote:
>
> > +1 (binding)
> >
> > On Wed, Apr 18, 2018 at 11:35 AM, Ismael Juma  wrote:
> >
> > > Hi all,
> > >
> > > I started a discussion last year about bumping the version of the June
> > 2018
> > > release to 2.0.0[1]. To reiterate the reasons in the original post:
> > >
> > > 1. Adopt KIP-118 (Drop Support for Java 7), which requires a major
> > version
> > > bump due to semantic versioning.
> > >
> > > 2. Take the chance to remove deprecated code that was deprecated prior
> to
> > > 1.0.0, but not removed in 1.0.0 (e.g. old Scala clients) so that we can
> > > move faster.
> > >
> > > One concern that was raised is that we still do not have a rolling
> > upgrade
> > > path for the old ZK-based consumers. Since the Scala clients haven't
> been
> > > updated in a long time (they don't support security or the latest
> message
> > > format), users who need them can continue to use 1.1.0 with no loss of
> > > functionality.
> > >
> > > Since it's already mid-April and people seemed receptive during the
> > > discussion last year, I'm going straight to a vote, but we can discuss
> > more
> > > if needed (of course).
> > >
> > > Ismael
> > >
> > > [1]
> > > https://lists.apache.org/thread.html/dd9d3e31d7e9590c1f727ef5560c93
> > > 3281bad0de3134469b7b3c4257@%3Cdev.kafka.apache.org%3E
> > >
> >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter  | blog
> > 
> >
>

Re: [VOTE] KIP-282: Add the listener name to the authentication context

[Manikumar]

+1 (non-binding)

On Wed, Apr 25, 2018 at 9:05 PM, Ted Yu  wrote:

> +1
>
> On Wed, Apr 25, 2018 at 8:33 AM, Rajini Sivaram 
> wrote:
>
> > Hi Mickael,
> >
> > Thanks for the KIP.
> >
> > +1 (binding)
> >
> > On Wed, Apr 25, 2018 at 9:51 AM, Mickael Maison <
> mickael.mai...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > There has been no objections in the DISCUSS thread so I'd like to
> > > start a vote on KIP-282:
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 282%3A+Add+the+listener+name+to+the+authentication+context
> > >
> > > Thanks
> > >
> >
>

Re: [VOTE] KIP-277 - Fine Grained ACL for CreateTopics API

[Manikumar]

+1 (non-binding)

Thanks

On Thu, Apr 26, 2018 at 9:59 PM, Colin McCabe  wrote:

> +1 (non-binding)
>
> best,
> Colin
>
>
> On Wed, Apr 25, 2018, at 02:45, Edoardo Comar wrote:
> > Hi,
> >
> > The discuss thread on KIP-277 (
> > https://www.mail-archive.com/dev@kafka.apache.org/msg86540.html )
> > seems to have been fruitful and concerns have been addressed, please
> allow
> > me start a vote on it:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 277+-+Fine+Grained+ACL+for+CreateTopics+API
> >
> > I will update the small PR to the latest KIP semantics if the vote
> passes
> > (as I hope :-).
> >
> > cheers
> > Edo
> > --
> >
> > Edoardo Comar
> >
> > IBM Message Hub
> >
> > IBM UK Ltd, Hursley Park, SO21 2JN
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
>

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

[Manikumar]

I assume the intent of the KIP to find out the Kafka broker version.  In
this case, maybe we should expose
version using a Kafka request. This will help the remote scripts/tools to
query the Kafka version.
scripts (kafka-topics.sh, kafka-configs.sh, etc..)  may run from remote
machines  and may use
older Kafka versions. In this case, current proposal prints on the older
version.

On Tue, May 1, 2018 at 7:47 PM, Colin McCabe  wrote:

> Thanks, Sasaki.
>
> Colin
>
> On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> > Hi Colin, Jason,
> >
> > Thank you for your beneficial comment.
> > I have updated my Pull Request to show git commit hash in version
> > information.> In my current Pull Request, we cat get the result such
> below:
> >
> >$ bin/kafka-topics.sh --version
> >(snip)
> >2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> >
> >
> > I have also updated to accept double-dash for this option (--
> > version) only.>
> >
> > Many thanks,
> > Sasaki
> >
> > > From: Jason Gustafson 
> > > Date: 2018-04-25 9:42 GMT+09:00
> > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > commands> > To: dev 
> > >
> > >
> > > +1 on adding the git commit id to the output. We often encounter
> > > environments which are testing off of trunk or have modifications on
> > > top of> > an existing release.
> > >
> > > -Jason
> > >
> > > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe 
> > > wrote:> >
> > >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > >>> Hi Jason, Colin,
> > >>>
> > >>> Thank you for your comment, and I'm sorry for late reply.
> > >>>
> > >>>   > we refactored all of the tools so that we could use a common
> > >>>   > set of> >>> base options,
> > >>>   > it might be a little annoying to have to continue supporting
> > >>>   > both> >>> variations.
> > >>>
> > >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> > >>> it), and> >>> I think it's no longer necessary both single-dash and
> double-
> > >>> dash are> >>> supported when this KIP will be accepted, as you said.
> > >>> I revise my Pull Request to support single-dash option only.
> > >>>
> > >>> Some my code in kafka-run-class.sh will become unnecessary when
> > >>> KIP-14> >>> is accepted.
> > >>> But I will keep this as temporary, because I seem that it's useful>
> >>> before KIP-14 accepted.
> > >>>
> > >>>
> > >>>   > Can you give a little more detail about what would be
> > >>>   > displayed when> >>> the version command was used?
> > >>>
> > >>> As Ismael said, the version string is got from
> > >>> AppInfoParser#getVersion.> >>>
> > >>> In my Pull Request, we can get the result such as below::
> > >>>
> > >>> $ bin/kafka-topics.sh --version
> > >>> (snip)
> > >>> Kafka 1.2.0-SNAPSHOT
> > >> Hi Sasaki,
> > >>
> > >> Thanks for the info.  Can you add this to the KIP?
> > >>
> > >> Also, I really think we should include the git hash.
> > >>
> > >> best,
> > >> Colin
> > >>
> > >>
> > >>>
> > >>> Many thanks,
> > >>> Sasaki
> > >>>
> > >>>
> >  From: Ismael Juma 
> >  Date: 2018-04-24 3:44 GMT+09:00
> >  Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> >  commands>  To: dev 
> > 
> > 
> >  FYI, the injection via the build process that is mentioned here
> > > already
> >  happens. See AppInfoParser.
> > 
> >  Ismael
> > 
> >  On Mon, Apr 23, 2018 at 9:39 AM, Colin McCabe
> >  > >> wrote:
> > > Hi Sasaki,
> > >
> > > Thanks for the KIP.  I think a version flag is a good idea.
> > >
> > > Can you give a little more detail about what would be displayed
> > > when> >> the
> > > version command was used?
> > >
> > > We clearly want the version number, but we probably also want to
> > > know> >> if
> > > this is an official release, or a random SNAPSHOT from a branch.
> > > If> >> this
> > > is a release candidate, we probably want the RC number as well,
> > > like> > "1.1-rc3"  We also want a git hash.  This can be
> injected by the> > build
> > > process.  In the case of an official release, where the source
> > > code> >> is not
> > > under git, we can pull it from a file.
> > >
> > > For example, hadoop's version output looks like this:
> > >
> > >> cmccabe@aurora:~/Downloads/hadoop-2.8.3> ./bin/hadoop
> > >> version> >> Hadoop 2.8.3
> > >> Subversion
> > >> https://git-wip-us.apache.org/repos/asf/hadoop.git -r> >
> b3fe56402d908019d99af1f1f4fc65cb1d1436a2
> > >> Compiled by jdu on 2017-12-05T03:43Z
> > >> Compiled with protoc 2.5.0
> > >> From source with checksum 9ff4856d824e983fa510d3f843e3f19d>
> >> This command was run using /home/cmccabe/Downloads/
> > > hadoop-2.8.3/share/hadoop/common/hadoop-common-2.8.3.jar
> > >
> > > (The "subversion" line here is a little weird -- it now refers
> > > to> >> git, not
> > > svn)
> > >
> > > On Wed

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

[Manikumar]

Hi Colin,

Thanks for explanation. It's definitely useful to have  --version flag.

kafka-broker-api-versions.sh gives the API versions, not Kafka release
version.
Is not easy to figure out release version from API versions. Currently
release version is available via metric/JMX.
If required, we can implement this in future.


Thanks,

On Wed, May 2, 2018 at 10:58 PM, Colin McCabe  wrote:

> Hi Manikumar,
>
> We already have a tool for getting the Kafka broker API versions,
> "./bin/kafka-broker-api-versions.sh".  It was added as part of KIP-97.
>
> What Saski is proposing here is having a way of getting the version of
> locally installed Kafka software, which may be different from the server
> version.  Many pieces of software offer a --version flag, and it's always
> understood to refer to the local version of the software, not a version
> running somewhere else.  The user has no way to get this information now,
> other than perhaps trying to look at he names of jar files.
>
> cheers,
> Colin
>
> On Tue, May 1, 2018, at 08:20, Manikumar wrote:
> > I assume the intent of the KIP to find out the Kafka broker version.  In
> > this case, maybe we should expose
> > version using a Kafka request. This will help the remote scripts/tools to
> > query the Kafka version.
> > scripts (kafka-topics.sh, kafka-configs.sh, etc..)  may run from remote
> > machines  and may use
> > older Kafka versions. In this case, current proposal prints on the older
> > version.
> >
> > On Tue, May 1, 2018 at 7:47 PM, Colin McCabe  wrote:
> >
> > > Thanks, Sasaki.
> > >
> > > Colin
> > >
> > > On Sat, Apr 28, 2018, at 00:55, Sasaki Toru wrote:
> > > > Hi Colin, Jason,
> > > >
> > > > Thank you for your beneficial comment.
> > > > I have updated my Pull Request to show git commit hash in version
> > > > information.> In my current Pull Request, we cat get the result such
> > > below:
> > > >
> > > >$ bin/kafka-topics.sh --version
> > > >(snip)
> > > >2.0.0-SNAPSHOT (Commit:f3876cd9617faf7e)
> > > >
> > > >
> > > > I have also updated to accept double-dash for this option (--
> > > > version) only.>
> > > >
> > > > Many thanks,
> > > > Sasaki
> > > >
> > > > > From: Jason Gustafson 
> > > > > Date: 2018-04-25 9:42 GMT+09:00
> > > > > Subject: Re: [DISCUSS] KIP-278: Add version option to Kafka's
> > > > > commands> > To: dev 
> > > > >
> > > > >
> > > > > +1 on adding the git commit id to the output. We often encounter
> > > > > environments which are testing off of trunk or have modifications
> on
> > > > > top of> > an existing release.
> > > > >
> > > > > -Jason
> > > > >
> > > > > On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe  >
> > > > > wrote:> >
> > > > >> On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote:
> > > > >>> Hi Jason, Colin,
> > > > >>>
> > > > >>> Thank you for your comment, and I'm sorry for late reply.
> > > > >>>
> > > > >>>   > we refactored all of the tools so that we could use a common
> > > > >>>   > set of> >>> base options,
> > > > >>>   > it might be a little annoying to have to continue supporting
> > > > >>>   > both> >>> variations.
> > > > >>>
> > > > >>> I feel KIP-14 is good idea (I'm sorry, I didn't know about
> > > > >>> it), and> >>> I think it's no longer necessary both single-dash
> and
> > > double-
> > > > >>> dash are> >>> supported when this KIP will be accepted, as you
> said.
> > > > >>> I revise my Pull Request to support single-dash option only.
> > > > >>>
> > > > >>> Some my code in kafka-run-class.sh will become unnecessary when
> > > > >>> KIP-14> >>> is accepted.
> > > > >>> But I will keep this as temporary, because I seem that it's
> useful>
> > > >>> before KIP-14 accepted.
> > > > >>>
> > > > >>>
> > > > >>>   > Can you give a little more detail about what would be
> > >

Re: [VOTE] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi All,

The vote has passed with 3 binding votes (Rajini, Jason, Dong) and 2
non-binding votes (Ted, Colin).

Thanks everyone for the votes.

Thanks,
Manikumar


On Sat, Aug 4, 2018 at 5:09 AM Dong Lin  wrote:

> Thanks Manikumar!
>
> +1
>
> On Fri, Aug 3, 2018 at 4:33 PM, Jason Gustafson 
> wrote:
>
> > +1 Thanks Manikumar!
> >
> > On Fri, Aug 3, 2018 at 4:24 PM, Colin McCabe  wrote:
> >
> > > +1 (non-binding)
> > >
> > > regards,
> > > Colin
> > >
> > > On Fri, Aug 3, 2018, at 02:27, Rajini Sivaram wrote:
> > > > Hi Manikumar,
> > > >
> > > > +1 (binding)
> > > >
> > > > Thanks for the KIP!
> > > >
> > > > On Fri, Aug 3, 2018 at 3:46 AM, Ted Yu  wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Thu, Aug 2, 2018 at 7:33 PM Manikumar <
> manikumar.re...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I would like to start voting on KIP-332 which allows AclCommand
> to
> > > use
> > > > > > AdminClient API for acl management.
> > > > > >
> > > > > > KIP:
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 332%3A+Update+AclCommand+to+use+AdminClient+API
> > > > > >
> > > > > > Discussion Thread:
> > > > > > https://www.mail-archive.com/dev@kafka.apache.org/msg90049.html
> > > > > >
> > > > > > Thanks,
> > > > > > Manikumar
> > > > > >
> > > > >
> > >
> >
>

Re: [DISCUSS] KIP-332: Update AclCommand to use AdminClient API

[Manikumar]

Hi All,

For this KIP, I will be using "--command-config" option name to specify
config file.
We can handle the naming discrepancies as part of KIP-14.

Thanks,





On Sat, Aug 4, 2018 at 4:53 AM Jason Gustafson  wrote:

> Here's hoping someone has time to pick up KIP-14 again:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-14+-+Tools+Standardization
> .
> The inconsistencies are so irritating.
>
> -Jason
>
> On Fri, Aug 3, 2018 at 10:27 AM, Rajini Sivaram 
> wrote:
>
> > Hi Dong,
> >
> > I don't have a preference for the option itself, but we are using "
> > --command-config" in ConfigCommand, ConsumerGroupCommand,
> > BrokerApiVersionsCommand, DeleteRecordsCommand. So it feels like we
> should
> > use the same here. I think we are currently only using "--config-file"
> > StreamsResetter. I will let others comment on whether we should change to
> > this when we add the option to other tools.
> >
> > Regards,
> >
> > Rajini
> >
> > On Fri, Aug 3, 2018 at 5:41 PM, Dong Lin  wrote:
> >
> > > Hey Rajini, Manikumar,
> > >
> > > Currently kafka-streams-application-reset.sh uses "--config-file" and a
> > > few
> > > other tools uses "--command-config". So the config name will be
> > > inconsistent no matter which config name we choose. Not sure we
> currently
> > > distinquish between core commands and non-core commands. I am wondering
> > if
> > > it is better to choose the name that is better in the long term. Do you
> > > think `config-file` would be more intuitive than `command-config` here?
> > >
> > > Thanks,
> > > Dong
> > >
> > > On Fri, Aug 3, 2018 at 2:40 AM, Manikumar 
> > > wrote:
> > >
> > > > Updated the KIP.  added a note to KIP-340 discussion thread.
> > > >
> > > > On Fri, Aug 3, 2018 at 2:52 PM Rajini Sivaram <
> rajinisiva...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Thanks Manikumar. Can you add a note to the KIP-340 discussion
> > thread?
> > > > >
> > > > > On Fri, Aug 3, 2018 at 10:04 AM, Manikumar <
> > manikumar.re...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Rajini,
> > > > > >
> > > > > > Yes, I too prefer using  "--command-config" . Recently in one of
> > the
> > > > > other
> > > > > > KIPs (KIP-340),  it was suggested to use
> > > > > > "--config-file", So I just followed the recent suggestion. But I
> > > think
> > > > we
> > > > > > should use same name in all tools (at least in core tools).
> > > > > >
> > > > > > If there are no concerns, I will change the option to
> > > > >  "--command-config".
> > > > > > Since KIP-340 PR is not yet merged, we can also change there.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > >
> > > > > > On Fri, Aug 3, 2018 at 1:57 PM Rajini Sivaram <
> > > rajinisiva...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Manikumar,
> > > > > > >
> > > > > > > We have some tools already (ConfigCommand,
> ConsumerGroupCommand,
> > > > > > > DelegationTokenCommand) which use "--command-config" option to
> > > > specify
> > > > > > > config file. Perhaps use should use the same name for
> AclCommand
> > as
> > > > > well?
> > > > > > >
> > > > > > > On Thu, Aug 2, 2018 at 7:23 PM, Colin McCabe <
> cmcc...@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1 for starting the vote
> > > > > > > >
> > > > > > > > cheers,
> > > > > > > > Colin
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 1, 2018, at 08:46, Manikumar wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > If there are no concerns, I will start the voting process
> > soon.
>

Re: [VOTE] KIP-346 - Improve LogCleaner behavior on error

[Manikumar]

+1 (non-binding)

Thanks for the KIP.

On Tue, Aug 7, 2018 at 10:42 PM Ray Chiang  wrote:

> +1 (non-binding)
>
> -Ray
>
> On 8/7/18 9:26 AM, Ted Yu wrote:
> > +1
> >
> > On Tue, Aug 7, 2018 at 5:25 AM Thomas Becker 
> wrote:
> >
> >> +1 (non-binding)
> >>
> >> We've hit issues with the log cleaner in the past, and this would be a
> >> great improvement.
> >> On Tue, 2018-08-07 at 12:19 +0100, Stanislav Kozlovski wrote:
> >>
> >> Hey everybody,
> >>
> >> I'm starting a vote on KIP-346
> >>
> >> <
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-346+-+Improve+LogCleaner+behavior+on+error
> >>
> >>
> >> 
> >>
> >> This email and any attachments may contain confidential and privileged
> >> material for the sole use of the intended recipient. Any review,
> copying,
> >> or distribution of this email (or any attachments) by others is
> prohibited.
> >> If you are not the intended recipient, please contact the sender
> >> immediately and permanently delete this email and any attachments. No
> >> employee or agent of TiVo Inc. is authorized to conclude any binding
> >> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> >> Inc. may only be made by a signed written agreement.
> >>
>
>

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

[Manikumar]

+1 (non-binding).

Thanks for the KIP.

On Wed, Aug 8, 2018 at 7:36 PM Ted Yu  wrote:

> +1
>
> On Wed, Aug 8, 2018 at 4:09 AM Mickael Maison 
> wrote:
>
> > +1 (non-binding)
> > Thanks Vahid
> > On Wed, Aug 8, 2018 at 11:26 AM Kamal Chandraprakash
> >  wrote:
> > >
> > > +1 (non-binding)
> > >
> > > Thanks for the KIP.
> > >
> > > On Wed, Aug 8, 2018 at 3:11 PM Stanislav Kozlovski <
> > stanis...@confluent.io>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > > Thanks!
> > > >
> > > > On Tue, Aug 7, 2018 at 11:47 PM Jason Gustafson 
> > > > wrote:
> > > >
> > > > > +1 Thanks Vahid.
> > > > >
> > > > > On Tue, Aug 7, 2018 at 11:14 AM, Vahid S Hashemian <
> > > > > vahidhashem...@us.ibm.com> wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I'd like to start a vote on KIP-289 to modify the default group
> id
> > of
> > > > > > KafkaConsumer.
> > > > > > The KIP:
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > 289%3A+Improve+the+default+group+id+behavior+in+KafkaConsumer
> > > > > > The discussion thread:
> > > > > > https://www.mail-archive.com/dev@kafka.apache.org/msg87379.html
> > > > > >
> > > > > > Thanks!
> > > > > > --Vahid
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best,
> > > > Stanislav
> > > >
> >
>

Re: Java 11 support

[Manikumar]

check this: https://issues.apache.org/jira/browse/KAFKA-7264

On Thu, Aug 16, 2018 at 2:40 PM Manoj Jain  wrote:

> Hi All,
>
>   I have just subscribed to this group. I don't know what is format of
> questions to be submitted yet but in hurry thought of getting more details
> on kafka java support.
> We are planning to use kafka in our project but since our whole env/infra
> is moving to java 11. I wanted to know the support for java 11 in kafka.
>
> --
> Thanks and Regards
> Manoj Jain
> call @ +918097122214
>

[DISCUSS] KIP-357: Add support to list ACLs per principal

[Manikumar]

Hi all,

I have created a minor KIP to add support to list ACLs per principal using
AclCommand (kafka-acls.sh)

https://cwiki.apache.org/confluence/display/KAFKA/KIP-357%3A++Add+support+to+list+ACLs+per+principal

Please take a look.

Thanks,