Re: RE: RE: [DISCUSS] KIP-280: Enhanced log compaction

Hi Guozhang, As much as I would like to move on from this topic, I've now tried to implement it into the pull request, and could not find a viable way to store a variable size byte array into the current concept of the log cleaner (with long the current approach just always considers it to be

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Hi, Matthias, Thanks for the review! I have updated the KIP, please look again. Thanks. --- Vito On Thu, Apr 19, 2018 at 9:58 PM, Matthias J. Sax wrote: > Thanks for clarification! That makes sense to me. > > Can you update the KIP to make those suggestions explicit? > > > -Matthias > > On

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Hi, John, Thanks for the comment! KIP typically discuss the public interfaces, non-user-facing detail is not necessary. I think your opinion is correct. But in this KIP, when a state store exception throw, the user will see the full exception stacktrace in the log, including the internal excepti

Re: [VOTE] Kafka 2.0.0 in June 2018

+1 (non-binding). Thx Ismael On Thu, Apr 19, 2018 at 10:55 PM, Matt Farmer wrote: > +1 (non-binding). TY! > > On Thu, Apr 19, 2018 at 11:56 AM, tao xiao wrote: > > > +1 non-binding. thx Ismael > > > > On Thu, 19 Apr 2018 at 23:14 Vahid S Hashemian < > vahidhashem...@us.ibm.com> > > wrote: > > >

Re: [VOTE] Kafka 2.0.0 in June 2018

no-binding。 +1 from my iphone！ On 04/24/2018 18:19, Sandor Murakozi wrote: +1 (non-binding). Thx Ismael On Thu, Apr 19, 2018 at 10:55 PM, Matt Farmer wrote: > +1 (non-binding). TY! > > On Thu, Apr 19, 2018 at 11:56 AM, tao xiao wrote: > > > +1 non-binding. thx Ismael > > > > On Thu, 19 Apr

Re: [VOTE] Kafka 2.0.0 in June 2018

+1 (non-binding) Thanks Ismael On Tue, Apr 24, 2018 at 12:20 PM, zhenya Sun wrote: > no-binding。 +1 > > > > > > from my iphone！ > On 04/24/2018 18:19, Sandor Murakozi wrote: > +1 (non-binding). > Thx Ismael > > On Thu, Apr 19, 2018 at 10:55 PM, Matt Farmer wrote: > > > +1 (non-binding). TY! >

[jira] [Created] (KAFKA-6822) Kafka Consumer 0.10.2.1 can not normally read data from Kafka 1.0.0

Phil Mikhailov created KAFKA-6822: - Summary: Kafka Consumer 0.10.2.1 can not normally read data from Kafka 1.0.0 Key: KAFKA-6822 URL: https://issues.apache.org/jira/browse/KAFKA-6822 Project: Kafka

Re: [VOTE] Kafka 2.0.0 in June 2018

+1 (non-binding). Whilst 1.0 -> 2.0 seems like visibly a "big" jump in version numbers to have so soon after 1.0, it makes sense with semver requirements and the deprecations contained. Thanks Tom Crayford Heroku Kafka On Tue, Apr 24, 2018 at 11:20 AM, zhenya Sun wrote: > no-binding。 +1 > > >

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

Hi Jason, Colin, Thank you for your comment, and I'm sorry for late reply. > we refactored all of the tools so that we could use a common set of base options, > it might be a little annoying to have to continue supporting both variations. I feel KIP-14 is good idea (I'm sorry, I didn't know ab

Fwd: Apache Kafka(Website Activity Tracking)

Hi, I am Prajakta Dumbre Software Engineer having 2+ experience in IT industry. I have already done Apache Kafka and spark streaming for web crawling. Now I have to develop Real-time recommendation System. I am trying to use Apache Kafka for Website activity tracking Bu am not able to understand ho

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Hi, Guozhang, Thanks for the comment! Hi, Bill, I'll try to make some update to make the KIP better. Thanks for the comment! --- Vito On Sat, Apr 21, 2018 at 5:40 AM, Bill Bejeck wrote: > Hi Vito, > > Thanks for the KIP, overall it's a +1 from me. > > At this point, the only thing I woul

Re: [VOTE] Kafka 2.0.0 in June 2018

+1 Thanks Ismael. On Tue, Apr 24, 2018 at 5:01 AM, Thomas Crayford wrote: > +1 (non-binding). Whilst 1.0 -> 2.0 seems like visibly a "big" jump in > version numbers to have so soon after 1.0, it makes sense with semver > requirements and the deprecations contained. > > Thanks > > Tom Crayford >

Re: [VOTE] Kafka 2.0.0 in June 2018

+1 On 4/24/18 4:21 PM, Jason Gustafson wrote: > +1 Thanks Ismael. > > On Tue, Apr 24, 2018 at 5:01 AM, Thomas Crayford > wrote: > >> +1 (non-binding). Whilst 1.0 -> 2.0 seems like visibly a "big" jump in >> version numbers to have so soon after 1.0, it makes sense with semver >> requirements an

Re: [VOTE] Kafka 2.0.0 in June 2018

+1 (non-binding) On Tue, Apr 24, 2018 at 5:29 PM, Matthias J. Sax wrote: > +1 > > On 4/24/18 4:21 PM, Jason Gustafson wrote: > > +1 Thanks Ismael. > > > > On Tue, Apr 24, 2018 at 5:01 AM, Thomas Crayford < > tcrayf...@salesforce.com> > > wrote: > > > >> +1 (non-binding). Whilst 1.0 -> 2.0 seems

Re: [VOTE] Kafka 2.0.0 in June 2018

+1. Thanks Ismael! Guozhang On Tue, Apr 24, 2018 at 8:33 AM, Stevo Slavić wrote: > +1 (non-binding) > > On Tue, Apr 24, 2018 at 5:29 PM, Matthias J. Sax > wrote: > > > +1 > > > > On 4/24/18 4:21 PM, Jason Gustafson wrote: > > > +1 Thanks Ismael. > > > > > > On Tue, Apr 24, 2018 at 5:01 AM, Th

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote: > Hi Jason, Colin, > > Thank you for your comment, and I'm sorry for late reply. > > > we refactored all of the tools so that we could use a common set of > base options, > > it might be a little annoying to have to continue supporting both >

Re: [DISCUSS] KIP-285: Connect Rest Extension Plugin

Randall, I'm trying to put together a PR for this KIP with the understanding that that all public interfaces should live in the api module and in the process I noticed the following 1. WorkerConfig is available in runtime. If we are exposing it to the Extension, do we just move it to the A

Re: [DISCUSS] KIP-285: Connect Rest Extension Plugin

Thanks for updating the KIP, Magesh. You've resolved all of my concerns, though I have one more: we should specify the package names for all new interfaces/classes. I'm looking forward to more feedback from others. Best regards, Randall On Fri, Apr 20, 2018 at 12:17 AM, Magesh Nandakumar wrote

[jira] [Created] (KAFKA-6823) Transient failure in DynamicBrokerReconfigurationTest.testThreadPoolResize

Anna Povzner created KAFKA-6823: --- Summary: Transient failure in DynamicBrokerReconfigurationTest.testThreadPoolResize Key: KAFKA-6823 URL: https://issues.apache.org/jira/browse/KAFKA-6823 Project: Kafka

[jira] [Created] (KAFKA-6824) Transient failure in DynamicBrokerReconfigurationTest.testAddRemoveSslListener

Anna Povzner created KAFKA-6824: --- Summary: Transient failure in DynamicBrokerReconfigurationTest.testAddRemoveSslListener Key: KAFKA-6824 URL: https://issues.apache.org/jira/browse/KAFKA-6824 Project: K

[jira] [Resolved] (KAFKA-6677) Remove EOS producer config max.in.flight.request.per.connection=1

[ https://issues.apache.org/jira/browse/KAFKA-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-6677. Resolution: Fixed Fix Version/s: 1.2.0 > Remove EOS producer config max.in.flight.req

[jira] [Resolved] (KAFKA-6705) producer.send() should not block due to metadata not available

[ https://issues.apache.org/jira/browse/KAFKA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-6705. - Resolution: Won't Fix > producer.send() should not block due to metadata not available > -

Re: RE: RE: [DISCUSS] KIP-280: Enhanced log compaction

Not sure if I fully understand your question, but here's my understanding: In LogCleaner we call: "val records = MemoryRecords.readableRecords(readBuffer)" Which returns a MemoryRecords object, and then call filterInto with a given customized RecordFilter that instantiates "checkBatchRetention(ba

Re: [DISCUSS] KIP-285: Connect Rest Extension Plugin

Magesh, I think our last emails cross in mid-stream. We definitely want to put the new public interfaces/classes in the API module, and implementation in the runtime module. Yes, this will affect the design, since for example we don't want to expose runtime types to the API, and we want to prevent

Build failed in Jenkins: kafka-trunk-jdk7 #3367

See Changes: [mjsax] KAFKA-6677: Fixed StreamsConfig producer's max-in-flight allowed when -- Started by an SCM change [EnvInject] - Loading node environment variables. Bu

Build failed in Jenkins: kafka-trunk-jdk8 #2577

See Changes: [mjsax] KAFKA-6677: Fixed StreamsConfig producer's max-in-flight allowed when -- [...truncated 416.21 KB...] kafka.admin.ResetConsumerGroupOffsetTest > testR

Re: [DISCUSS] KIP-280: Enhanced log compaction

Hi Guozhang, I mean here: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/log/OffsetMap.scala It seems that this class was designed with a great focus on preventing memory starvation, and this class is also very central to the changes required for this KIP. So far this h

Build failed in Jenkins: kafka-trunk-jdk10 #44

See Changes: [mjsax] KAFKA-6677: Fixed StreamsConfig producer's max-in-flight allowed when -- [...truncated 1.48 MB...] kafka.controller.PartitionStateMachineTest > testI

Build failed in Jenkins: kafka-trunk-jdk8 #2578

See Changes: [jason] MINOR: Disabled flaky [wangguoz] KAFKA-6376: Document skipped records metrics changes (#4922) -- [...truncated 417.74 KB...] kafka.utils.json.JsonVa

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

+1 on adding the git commit id to the output. We often encounter environments which are testing off of trunk or have modifications on top of an existing release. -Jason On Tue, Apr 24, 2018 at 10:06 AM, Colin McCabe wrote: > On Tue, Apr 24, 2018, at 05:36, Sasaki Toru wrote: > > Hi Jason, Colin

Jenkins build is back to normal : kafka-trunk-jdk7 #3368

See

Jenkins build is back to normal : kafka-trunk-jdk10 #45

See

Re: [VOTE] Kafka 2.0.0 in June 2018

Thanks to everyone who voted. The vote passed with the following votes: * 6 binding +1s: Rajini, Gwen, Jason, Matthias, Guozhang, myself * 16 non-binding +1s: Ted, Edoardo, Jakub, Bill, Rahul, Mickael, Stephane, Manikumar, Jorge, Vahid, Tao, Matt, Sandor, Zhenya, Viktor, Thomas * No +0, -0 or -1 v

[jira] [Resolved] (KAFKA-6796) Surprising UNKNOWN_TOPIC error for produce/fetch requests to non-replicas

[ https://issues.apache.org/jira/browse/KAFKA-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-6796. Resolution: Fixed Fix Version/s: 1.2.0 > Surprising UNKNOWN_TOPIC error for produce/fetch req

Re: [DISCUSS] KIP-219 - Improve Quota Communication

Hi Jon, Not sure about this approach. It's worth mentioning this in the vote thread as well so that the people who voted originally have a chance to comment. Also, we should really get input from developers of Kafka clients (librdkafka, kafka-python, etc.) for this KIP. Ismael On Thu, Apr 5, 201

Re: [DISCUSS] KIP-219 - Improve Quota Communication

I have to develop Real-time recommendation System. I am trying to use Apache Kafka for Website activity tracking Bu am not able to understand how can I track website activity. Please tell me the way how can I track website activity. Right now I am using Apache Kafka, apache spark, and python. pleas