Build failed in Jenkins: kafka-trunk-jdk8 #3525

See -- [...truncated 545 B...] > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10 Fetching upstream changes from https://github.com/apache/kafka.git > git -

Jenkins build is back to normal : kafka-trunk-jdk8 #3524

See

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Hi Colin / Jason,  Reassignment should really be doing a batches.  I am not too worried about reassignment znode getting larger.  In a real production environment,  too many concurrent reassignment and too frequent submission of reassignments seemed to cause latency spikes of kafka cluster.  S

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Thanks, Ron. Updating the KIP. will add answers here as well 1) If the cold storage technology can be cross-region, is there a possibility for a disaster recovery Kafka cluster to share the messages in cold storage? My guess is the answer is no, and messages replicated to the D/R cluster have

Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

Thanks for the review Matthias! My 2-cent on the rebalance delay is that it is a rather fixed trade-off between task availability and resource shuffling. If we eventually trigger rebalance after rolling bounce, certain consumer setup is still faced with global shuffles, for example member.id r

Re: [VOTE] KIP-445: In-memory session store

Thanks for the KIP Sophie! +1 (binding) -Matthias On 4/8/19 5:26 PM, Sophie Blee-Goldman wrote: > Hello all, > > There has been a positive reception so I'd like to call for a vote on > KIP-445, augmenting our session store options with an in-memory version. > This would round out our store API

[VOTE] KIP-445: In-memory session store

Hello all, There has been a positive reception so I'd like to call for a vote on KIP-445, augmenting our session store options with an in-memory version. This would round out our store API to offer in-memory and persistent versions of all three types of stores. KIP: https://cwiki.apache.org/confl

Build failed in Jenkins: kafka-trunk-jdk8 #3523

See -- [...truncated 545 B...] > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10 Fetching upstream changes from https://github.com/apache/kafka.git > git -

Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

Thank for the KIP, Boyang and Guozhang! I made an initial pass and have some questions/comments. One high level comment: it seems that the KIP "mixes" plain consumer and Kafka Streams use case a little bit (at least in the presentation). It might be helpful to separate both cases clearly, or mayb

Build failed in Jenkins: kafka-trunk-jdk8 #3522

See -- [...truncated 545 B...] > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10 Fetching upstream changes from https://github.com/apache/kafka.git > git -

Build failed in Jenkins: kafka-trunk-jdk8 #3521

See -- [...truncated 545 B...] > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10 Fetching upstream changes from https://github.com/apache/kafka.git > git -

[jira] [Created] (KAFKA-8200) TopologyTestDriver should offer an iterable signature of readOutput

Michael Drogalis created KAFKA-8200: --- Summary: TopologyTestDriver should offer an iterable signature of readOutput Key: KAFKA-8200 URL: https://issues.apache.org/jira/browse/KAFKA-8200 Project: Kafk

[jira] [Resolved] (KAFKA-8013) Avoid buffer underflow when reading a Struct from a partially correct buffer

[ https://issues.apache.org/jira/browse/KAFKA-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8013. Resolution: Fixed > Avoid buffer underflow when reading a Struct from a partially correct b

[jira] [Resolved] (KAFKA-8181) Streams docs on serialization include Avro header, but no content

[ https://issues.apache.org/jira/browse/KAFKA-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8181. Resolution: Fixed Fix Version/s: 2.3.0 > Streams docs on serialization include Avro

[jira] [Resolved] (KAFKA-8169) Wrong topic on streams quick start documentation

[ https://issues.apache.org/jira/browse/KAFKA-8169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8169. Resolution: Fixed Fix Version/s: 2.3.0 [~tcheutchoua], I added you to the list of co

Re: [DISCUSS] KIP-445: In-memory Session Store

Hi Sophie, Thanks for the KIP, it's a +1 from me. -Bill On Tue, Apr 2, 2019 at 1:09 AM Guozhang Wang wrote: > Hello Sophie, > > Thanks for the KIP, I've made a pass on it and it lgtm. > > > Guozhang > > On Fri, Mar 29, 2019 at 5:32 PM Sophie Blee-Goldman > wrote: > > > Hello all, > > > > This

[VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

Hi, I’d like to begin the voting thread for KIP-419. This is a minor KIP to add a new stopped() method to the SourceTask interface in Kafka Connect. Its purpose is to give the task a safe opportunity to clean up its resources, in the knowledge that this is the final call to the task. KIP: http

[jira] [Resolved] (KAFKA-5784) Add a sensor for dropped records in window stores

[ https://issues.apache.org/jira/browse/KAFKA-5784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-5784. -- Resolution: Fixed Fix Version/s: 2.1.0 > Add a sensor for dropped records in window stor

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Yeah, I think adding this information to LeaderAndIsr makes sense. It would be better to track "reassigningReplicas" than "originalReplicas", I think. Tracking "originalReplicas" is going to involve sending a lot more data, since most replicas in the system are not reassigning at any given poi

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

> not efficient copy in out of local disk I'm skeptical of this actually. HDD throughput is much higher than network throughput, so if you are fetching from a remote store, writing to disk in between is unlikely to impact end-to-end latency. After all, Kafka topics are always written through disk,

[jira] [Resolved] (KAFKA-8157) Missing "key.serializer" exception when setting "segment index bytes"

[ https://issues.apache.org/jira/browse/KAFKA-8157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8157. -- Resolution: Fixed Assignee: Guozhang Wang Fix Version/s: 2.2.1

Re: [DISCUSS] KIP-435: Incremental Partition Reassignment

Hey George, For the URP during a reassignment, if the "original_replicas" is kept for > the current pending reassignment. I think it will be very easy to compare > that with the topic/partition's ISR. If all "original_replicas" are in > ISR, then URP should be 0 for that topic/partition. Yeah,

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

I'm going to call this vote. +1: Me, Ryanne, Guozhang, Eno, David, Viktor, Stephane, Gwen, Jun Binding total is +5 with no -1 or +0. Thanks all for the discussion and feedback. -Jason On Thu, Apr 4, 2019 at 4:55 PM Jun Rao wrote: > Hi, Jason, > > Thanks for the updated KIP. +1 from me. > > J

[VOTE] KIP-360: Improve handling of unknown producer when using EOS

Hi All, I'd like to start a vote on KIP-360: https://cwiki.apache.org/confluence/display/KAFKA/KIP-360%3A+Improve+handling+of+unknown+producer . +1 from me (duh) Thanks, Jason

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Harsha. A couple of questions. I think I know the answers, but it would be good to see them explicitly documented. 1) If the cold storage technology can be cross-region, is there a possibility for a disaster recovery Kafka cluster to share the messages in cold storage? My guess is the answer

[jira] [Created] (KAFKA-8199) ClassCastException when trying to groupBy after suppress

Bill Bejeck created KAFKA-8199: -- Summary: ClassCastException when trying to groupBy after suppress Key: KAFKA-8199 URL: https://issues.apache.org/jira/browse/KAFKA-8199 Project: Kafka Issue Type

Re: [DISCUSS] KIP-252: Extend ACLs to allow filtering based on ip ranges and subnets

Hi Colin, quick summary up front: I totally agree, and always have! I think we misunderstood each other a little :) I was never really opposed the idea of restricting which ACL features can be used, I was just opposed to doing it specifically for this change, while not introducing real versioning.

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Jun, Yaodong, 21. The proposed approach sounds very hacky. User principals can contain arbitrary characters. So we can't simply split `user1/clients/clientA` into tokens using '/' as delimiter. Internally, we sanitize names before storing in ZK, but the names provided by the user are actual pr

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

Hi Colin, I am not sure the API proposed in the KIP fits with the type of updates we support. The old API with Map fits better and we need to find a way to do something similar while still retaining the old one. Each request should specify a collection of updates for each ConfigResource with resu

Re: Kafka Jenkins not showing recent builds in history

I've opened an INFRA ticket about this [1]. Apparently, it is a known issue that is solved by a restart of Jenkins. One will be scheduled in the near future, I'll update the list once the restart has happened. Best regards, Sönke [1] https://issues.apache.org/jira/browse/INFRA-18161 On Wed, Apr