Kafka SASL handshake takes too long

2018-07-10 Thread Majed Al Zayer
Hello everyone, I hope that someone could help me with this issue. I have already posted this on: - StackOverflow: https://stackoverflow.com/questions/51249835/kafka-sasl-handshake-takes-too-long - Confluent Kafka .Net @ Github: https://github.com/confluentinc/confluent-kafka-dotnet/issues/564

[jira] [Created] (KAFKA-7145) Consumer thread getting stuck in hasNext() method

2018-07-10 Thread Lovenish goyal (JIRA)
Lovenish goyal created KAFKA-7145: - Summary: Consumer thread getting stuck in hasNext() method Key: KAFKA-7145 URL: https://issues.apache.org/jira/browse/KAFKA-7145 Project: Kafka Issue Type:

Re: [DISCUSSION] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

2018-07-10 Thread Viktor Somogyi
Hi Ismael, Well, yes. If we care about headers only then you'd need to add a dummy implementation for the 2 parameter method as well. Although it is not ideal, we're using the Serializer interface everywhere and convert it to extended with ensureExtended(serializer) and delegate to the 2 parameter

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression (Updated)

2018-07-10 Thread Bobby Evans
I there any update on this. The performance improvements are quite impressive and I really would like to stop forking kafka just to get this in. Thanks, Bobby On Wed, Jun 13, 2018 at 8:56 PM Dongjin Lee wrote: > Ismael, > > Oh, I forgot all of you are on working frenzy for 2.0! No problem, ta

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

2018-07-10 Thread John Roesler
Hi Guozhang, That sounds good to me. I'll include that in the KIP. Thanks, -John On Mon, Jul 9, 2018 at 6:33 PM Guozhang Wang wrote: > Let me clarify a bit on what I meant about moving `retentionPeriod` to > WindowStoreBuilder: > > In another discussion we had around KIP-319 / 330, that the "r

[jira] [Resolved] (KAFKA-7080) WindowStoreBuilder incorrectly initializes CachingWindowStore

2018-07-10 Thread John Roesler (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-7080. - Resolution: Fixed Fixed in https://github.com/apache/kafka/pull/5257 > WindowStoreBuilder incorr

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

2018-07-10 Thread Manikumar
Waiting for one more binding vote to pass this minor KIP. Appreciate your vote. On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska wrote: > +1 (non binding) > > On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > +1 (non-binding) > > > > On Wed, Jul 4,

[VOTE] 2.0.0 RC2

2018-07-10 Thread Rajini Sivaram
Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 2.0.0. This is a major version release of Apache Kafka. It includes 40 new KIPs and several critical bug fixes. Please see the 2.0.0 release plan for more details: https://cwiki.apach

Re: Stream config in StreamPartitionAssignor

2018-07-10 Thread Guozhang Wang
Boyang, I see. That makes sense. We should at least update the docs to reduce confusions. Guozhang On Mon, Jul 9, 2018 at 10:22 PM, Boyang Chen wrote: > Hey Guozhang, > > > thanks for the reply. Actually I'm not trying to pass config into > StreamPartitionAssignor. My debugging was about som

Re: Kafka system tests contribution

2018-07-10 Thread Colin McCabe
Hi Andriy, Try looking at the logs to see why the test failed. best, Colin On Wed, May 16, 2018, at 07:08, Andriy Sorokhtey wrote: > Hi, > > Did anyone had a chance to take a look at this issue? > > 2018-05-08 15:01 GMT+03:00 Andriy Sorokhtey : > > > Hello Kafka team > > > > I’d like to contr

[jira] [Created] (KAFKA-7146) Grouping consumer requests per consumer coordinator in admin client in describeConsumerGroups

2018-07-10 Thread Yishun Guan (JIRA)
Yishun Guan created KAFKA-7146: -- Summary: Grouping consumer requests per consumer coordinator in admin client in describeConsumerGroups Key: KAFKA-7146 URL: https://issues.apache.org/jira/browse/KAFKA-7146

Re: [VOTE] KIP-264: Add a consumer metric to record raw fetch size

2018-07-10 Thread Vahid S Hashemian
Bump! From: "Vahid S Hashemian" To: dev Date: 05/25/2018 10:51 AM Subject:[VOTE] KIP-264: Add a consumer metric to record raw fetch size In the absence of additional feedback on this KIP I'd like to start a vote. To summarize, the KIP simply proposes to add a consumer me

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

2018-07-10 Thread Dong Lin
Hey Anna, Thanks much for your detailed explanation and example! It does help me understand the difference between our understanding. So it seems that the solution based on findOffsets() currently focuses mainly on the scenario that consumer has cached leaderEpoch -> offset mapping whereas I was

Stream processing meetup at LinkedIn (Sunnyvale) on Thursday, July 19 at 6pm

2018-07-10 Thread Dong Lin
Hi everyone, We would like to invite you to a Stream Processing Meetup at LinkedIn’s Sunnyvale campus, LinkedIn Building F (LSNF) 605 West Maude Avenuee, on Thursday, July 19 at 6pm. Please RSVP here (if you intend to attend in person): *https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/ev

[jira] [Created] (KAFKA-7147) ReassignPartitionsCommand should be able to connect to broker over SSL

2018-07-10 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-7147: --- Summary: ReassignPartitionsCommand should be able to connect to broker over SSL Key: KAFKA-7147 URL: https://issues.apache.org/jira/browse/KAFKA-7147 Project: Kafka I

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

2018-07-10 Thread Anna Povzner
Hi Dong, Thanks for the follow up! I finally have much more clear understanding of where you are coming from. You are right. The success of findOffsets()/finding a point of non-divergence depends on whether we have enough entries in the consumer's leader epoch cache. However, I think this is a fu

Re: Stream config in StreamPartitionAssignor

2018-07-10 Thread Boyang Chen
Sounds good. I will take a look at the doc and get back to you! From: Guozhang Wang Sent: Wednesday, July 11, 2018 2:07 AM To: dev Subject: Re: Stream config in StreamPartitionAssignor Boyang, I see. That makes sense. We should at least update the docs to reduc

Re: [VOTE] 2.0.0 RC2

2018-07-10 Thread Ted Yu
+1 Ran thru test suite. Checked signatures. On Tue, Jul 10, 2018 at 10:17 AM Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers, > > > This is the third candidate for release of Apache Kafka 2.0.0. > > > This is a major version release of Apache Kafka. It includes 40 n

[jira] [Resolved] (KAFKA-7141) kafka-consumer-group doesn't describe existing group

2018-07-10 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian resolved KAFKA-7141. Resolution: Not A Problem > kafka-consumer-group doesn't describe existing group >

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

2018-07-10 Thread Anna Povzner
Sorry, I hit "send" before finishing. Continuing... 2) Hiding most of the consumer handling log truncation logic with minimal exposure in KafkaConsumer API. I was proposing this path. Before answering your specific questions… I want to answer to your comment “In general, maybe we should discus

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

2018-07-10 Thread John Roesler
I had some opportunity to reflect on the default for close time today... Note that the current "close time" is equal to the retention time, and therefore "close" today shares the default retention of 24h. It would definitely break any application that today specifies a retention time to set close

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

2018-07-10 Thread Dong Lin
Hey Anna, Thanks for the comment. To answer your question, it seems that we can cover all case in this KIP. As stated in "Consumer Handling" section, KIP-101 based approach will be used to derive the truncation offset from the 2-tuple (offset, leaderEpoch). This approach is best effort and it is i

[jira] [Created] (KAFKA-7148) Kafka load log very slow after goes down with outOfMemoryError

2018-07-10 Thread wang (JIRA)
wang created KAFKA-7148: --- Summary: Kafka load log very slow after goes down with outOfMemoryError Key: KAFKA-7148 URL: https://issues.apache.org/jira/browse/KAFKA-7148 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-7149) Reduce assignment data size to improve kafka streams scalability

2018-07-10 Thread Ashish Surana (JIRA)
Ashish Surana created KAFKA-7149: Summary: Reduce assignment data size to improve kafka streams scalability Key: KAFKA-7149 URL: https://issues.apache.org/jira/browse/KAFKA-7149 Project: Kafka

Re: [VOTE] 2.0.0 RC2

2018-07-10 Thread Brett Rann
+1 (non binding) rolling upgrade of tiny shared staging multitenacy (200+ consumer groups) cluster from 1.1 to 2.0.0-rc1 to 2.0.0-rc2. cluster looks healthy after upgrade. Lack of burrow lag suggests consumers are still happy, and incoming messages remains the same. Will monitor. On Wed, Jul 11,

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

2018-07-10 Thread Guozhang Wang
That is a good point.. I cannot think of a better option than documentation and warning, and also given that we'd probably better not reusing the function name `until` for close time. Guozhang On Tue, Jul 10, 2018 at 3:31 PM, John Roesler wrote: > I had some opportunity to reflect on the def

[jira] [Resolved] (KAFKA-5991) Change Consumer per partition lag metrics to put topic-partition-id in tags instead of metric name

2018-07-10 Thread Kevin Lu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Lu resolved KAFKA-5991. - Resolution: Fixed > Change Consumer per partition lag metrics to put topic-partition-id in tags > instea

[jira] [Reopened] (KAFKA-7141) kafka-consumer-group doesn't describe existing group

2018-07-10 Thread huxihx (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxihx reopened KAFKA-7141: --- Assignee: huxihx > kafka-consumer-group doesn't describe existing group > -

Kafka Streams new release

2018-07-10 Thread Ayushi Sharma
When is the next Kafka Streams release (Kafka Streams 2.0.0) which was tentatively on 26June?

Re: Requesting Permission To Create KIP And Assign JIRAs

2018-07-10 Thread Kevin Lu
Hi, I received access to assign JIRAs now, but I still cannot create KIPs. When I hover over the "Create KIP" button, it says I do not have permission to create content. When I click on the button, it goes a 404 Not Found page. Can someone add me? Thanks! Regards, Kevin On Tue, Jun 26, 2018 a

Re: Kafka Streams new release

2018-07-10 Thread Ofir Manor
>From the release plan: " While the target release date is fixed at ~2w after code freeze, RCs will roll out as needed until the release vote passes" https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=80448820 You can follow the voting threads on this mailing list - the current t