Re: [DISCUSS] KIP-263: Allow broker to skip sanity check of inactive segments on broker startup

Hey Jay, Thanks for your comments! Yeah recovery is different from the sanity check. They are correlated in the sense that there may still be corrupted index files even after clean broker shutdown. And in that case if we delay the sanity check then we may delay the log recovery. The main goal of

[jira] [Resolved] (KAFKA-6613) The controller shouldn't stop partition reassignment after an exception is being thrown

[ https://issues.apache.org/jira/browse/KAFKA-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chandra kasiraju resolved KAFKA-6613. - Resolution: Fixed Fix Version/s: (was: 0.11.0.2) 1.0.0 > The

[jira] [Created] (KAFKA-6690) Priorities for Source Topics

Bala Prassanna I created KAFKA-6690: --- Summary: Priorities for Source Topics Key: KAFKA-6690 URL: https://issues.apache.org/jira/browse/KAFKA-6690 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-6691) Requesting High Watermark and Truncation documentation be added to Docs

Justin Manchester created KAFKA-6691: Summary: Requesting High Watermark and Truncation documentation be added to Docs Key: KAFKA-6691 URL: https://issues.apache.org/jira/browse/KAFKA-6691 Project

Re: Seeking Feedback on Kafka Connect Issues

Hi Ewen, Thanks for the thoughtful response. I’m happy to take some time to write up a KIP and do some implementation work here. I did KIP-210 previously, so I’ve been through the process before. We also have some organizational interest for improving Kafka Connect. Our concern internally is tha

Build failed in Jenkins: kafka-1.0-jdk7 #175

See Changes: [rajinisivaram] MINOR: Updated SASL Authentication Sequence Docs (#4724) -- [...truncated 373.45 KB...] kafka.log.BrokerCompressionTest > testBrokerSideCompress

[jira] [Created] (KAFKA-6692) Kafka Streams internal topics should be prefixed with an underscore

Yeva Byzek created KAFKA-6692: - Summary: Kafka Streams internal topics should be prefixed with an underscore Key: KAFKA-6692 URL: https://issues.apache.org/jira/browse/KAFKA-6692 Project: Kafka

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

Hi Ted, The additional hash map is very small, possibly a few KB. Each request type ("produce", "fetch", etc.) will have such a map which have a few entries depending on the client API versions the broker will encounter. So if broker encounters two client versions for "produce", there will be two

Build failed in Jenkins: kafka-1.1-jdk7 #93

See Changes: [rajinisivaram] MINOR: Updated SASL Authentication Sequence Docs (#4724) -- [...truncated 1.65 MB...] org.apache.kafka.streams.integration.TableTableJoinIntegrati

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

Tom, on the implications you are referring to. For me they seem the same for the __consumer_offsets and the __transaction_state topic. So I am wondering if we can rely on the same solutions for them, like providing a *.replication.factor config option. Best regards, Steven Op ma 19 mrt. 2

Build failed in Jenkins: kafka-trunk-jdk7 #3275

See Changes: [rajinisivaram] Trogdor's ProducerBench does not fail if topics exists (#4673) [rajinisivaram] MINOR: Updated SASL Authentication Sequence Docs (#4724) -- [.

[jira] [Created] (KAFKA-6693) Add Consumer-only benchmark workload to Trogdor

Anna Povzner created KAFKA-6693: --- Summary: Add Consumer-only benchmark workload to Trogdor Key: KAFKA-6693 URL: https://issues.apache.org/jira/browse/KAFKA-6693 Project: Kafka Issue Type: Impro

Build failed in Jenkins: kafka-trunk-jdk8 #2493

See Changes: [rajinisivaram] Trogdor's ProducerBench does not fail if topics exists (#4673) [rajinisivaram] MINOR: Updated SASL Authentication Sequence Docs (#4724) -- [.

[jira] [Created] (KAFKA-6694) The Trogdor Coordinator should support filtering task responses

Colin P. McCabe created KAFKA-6694: -- Summary: The Trogdor Coordinator should support filtering task responses Key: KAFKA-6694 URL: https://issues.apache.org/jira/browse/KAFKA-6694 Project: Kafka

[jira] [Created] (KAFKA-6695) Add a system test for dynamic broker config update

Rajini Sivaram created KAFKA-6695: - Summary: Add a system test for dynamic broker config update Key: KAFKA-6695 URL: https://issues.apache.org/jira/browse/KAFKA-6695 Project: Kafka Issue Type

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

Thanks for the response. Assuming number of client versions is limited in a cluster, memory consumption is not a concern. Cheers On Tue, Mar 20, 2018 at 10:47 AM, Allen Wang wrote: > Hi Ted, > > The additional hash map is very small, possibly a few KB. Each request type > ("produce", "fetch",

[jira] [Created] (KAFKA-6696) Trogdor should support destroying tasks

Colin P. McCabe created KAFKA-6696: -- Summary: Trogdor should support destroying tasks Key: KAFKA-6696 URL: https://issues.apache.org/jira/browse/KAFKA-6696 Project: Kafka Issue Type: Improve

[VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi, I would like to start the vote for KIP-268: https://cwiki.apache.org/confluence/display/KAFKA/KIP-268%3A+Simplify+Kafka+Streams+Rebalance+Metadata+Upgrade PR https://github.com/apache/kafka/pull/4636 contains the fixes to upgrade from metadata version 1 to 2. Some tests are still missing but

Re: [VOTE] KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils

Thanks everyone. I am closing this vote. The KIP is accepted with 3 binding votes (Damian, Guozhang, Matthias) and 2 non-binding votes (Ted, Bill) Thanks for your comments and votes! -John On Mon, Mar 12, 2018 at 4:08 PM, Damian Guy wrote: > +1 > On Fri, 9 Mar 2018 at 21:30, Matthias J. Sax

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Hi Matthias, Thanks for setting up the upgrade path. +1 (non-binding) On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax wrote: > Hi, > > I would like to start the vote for KIP-268: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 268%3A+Simplify+Kafka+Streams+Rebalance+Metadata+Upgra

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

+1 On Tue, Mar 20, 2018 at 5:56 PM, Richard Yu wrote: > Hi Matthias, > Thanks for setting up the upgrade path. > > +1 (non-binding) > > On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax > wrote: > > > Hi, > > > > I would like to start the vote for KIP-268: > > > > https://cwiki.apache.org/conflu

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

+1 Thanks, Bill On Tue, Mar 20, 2018 at 9:23 PM, Ted Yu wrote: > +1 > > On Tue, Mar 20, 2018 at 5:56 PM, Richard Yu > wrote: > > > Hi Matthias, > > Thanks for setting up the upgrade path. > > > > +1 (non-binding) > > > > On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax > > wrote: > > > > > Hi

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Thanks you of your offer ,agree with you! On 2018/03/21 00:56:11, Richard Yu wrote: > Hi Matthias, > Thanks for setting up the upgrade path. > > +1 (non-binding) > > On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax > wrote: > > > Hi, > > > > I would like to start the vote for KIP-268: >

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

+1 On 2018/03/21 01:23:33, Ted Yu wrote: > +1 > > On Tue, Mar 20, 2018 at 5:56 PM, Richard Yu > wrote: > > > Hi Matthias, > > Thanks for setting up the upgrade path. > > > > +1 (non-binding) > > > > On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax > > wrote: > > > > > Hi, > > > > > > I would

Kafka version and JDK version

Hi If I remember correctly, Kafka 2.0 is targeted this summer as it'll drop support for java 7 and dropping a java version is supposed to imply a major version bump in Kafka. Now that Java has a very quick release cycle for JDK (version 10 today), my question is: how fast will Kafka versioning go

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

I love this. Not much to add - it is an elegant solution, clean implementation and it addresses a real need, especially during upgrades. On Tue, Mar 20, 2018 at 2:49 PM, Ted Yu wrote: > Thanks for the response. > > Assuming number of client versions is limited in a cluster, memory > consumption

[jira] [Created] (KAFKA-6697) JBOD configured broker should not die if log directory is invalid

Dong Lin created KAFKA-6697: --- Summary: JBOD configured broker should not die if log directory is invalid Key: KAFKA-6697 URL: https://issues.apache.org/jira/browse/KAFKA-6697 Project: Kafka Issue

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Sorry, I see that the VOTE started already, but I have a late question on this KIP. In the "version probing" protocol: > Detailed upgrade protocol from metadata version X to Y (with X >= 1.2): > On startup/rolling-bounce, an instance does not know what version the leader > understands and (optim

Re: Kafka version and JDK version

Hi Stephane, I don't see why we would increment Kafka versions as quick as Java versions. The way I think it should work is that we support LTS versions for a long time and only support the most recent non LTS version. The latter is to ensure that we catch any issues with newer Java releases quick

Build failed in Jenkins: kafka-trunk-jdk7 #3276

See Changes: [ismael] MINOR: Fix flaky TestUtils functions (#4743) -- [...truncated 417.88 KB...] kafka.server.epoch.LeaderEpochFileCacheTest > shouldUpdateOffsetBetweenE

Re: Kafka version and JDK version

Sounds good ! Thanks for the detailed explanation :) On Wed., 21 Mar. 2018, 11:40 am Ismael Juma, wrote: > Hi Stephane, > > I don't see why we would increment Kafka versions as quick as Java > versions. The way I think it should work is that we support LTS versions > for a long time and only sup

Build failed in Jenkins: kafka-1.0-jdk7 #176

See Changes: [ismael] MINOR: Fix flaky TestUtils functions (#4743) -- [...truncated 373.20 KB...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[16] STARTED kaf