Re: [DISCUSS] KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriver

Hi, If you want to see or test the my current idea of the implementation of this KIP, you can check it out in my repo: https://github.com/jukkakarvanen/kafka/compare/trunk...jukkakarvanen:KAFKA-8233_InputOutputTopics After my test with KPI-451 I do not see need for add methods for Iterables, bu

[jira] [Created] (KAFKA-8288) KStream.through consumer does not use custom TimestampExtractor

Maarten created KAFKA-8288: -- Summary: KStream.through consumer does not use custom TimestampExtractor Key: KAFKA-8288 URL: https://issues.apache.org/jira/browse/KAFKA-8288 Project: Kafka Issue Type

[jira] [Created] (KAFKA-8289) KTable, Long> can't be suppressed

Xiaolin Jia created KAFKA-8289: -- Summary: KTable, Long> can't be suppressed Key: KAFKA-8289 URL: https://issues.apache.org/jira/browse/KAFKA-8289 Project: Kafka Issue Type: Bug Compon

Re: [DISCUSS] KIP-451: Make TopologyTestDriver output iterable

Hi As discussed, if the preferred option is to consume the records always I will change both methods in KIP-451 accordingly and also switch them to return a List. This would be a bit redundant with Jukkas proposal in KIP-456 so the question is if KIP-451 should be scraped in favor of KIP-456 which

Build failed in Jenkins: kafka-trunk-jdk8 #3581

See Changes: [github] KAFKA-8237; Untangle TopicDeleteManager and add test cases (#6588) [jason] MINOR: Remove implicit return statement (#6629) -- [...truncated 4.77 MB.

Re: [VOTE] KIP-417: Allow JmxTool to connect to a secured RMI port

+1 (binding) On Fri, Feb 1, 2019, 8:20 AM Harsha Chintalapani wrote: > +1 (binding). > > Thanks, > Harsha > On Jan 31, 2019, 8:08 PM -0800, Manikumar , > wrote: > > Hi, > > > > +1 (binding). Thanks for the KIP. > > > > On Mon, Jan 28, 2019 at 5:37 PM Fangbin Sun > wrote: > > > > > Hi, All: > >

Re: [DISCUSS] 2.2.1 Bug Fix Release

Thanks Vahid! On Wed, Apr 24, 2019 at 10:44 PM Vahid Hashemian wrote: > Hi all, > > I'd like to volunteer for the release manager of the 2.2.1 bug fix release. > Kafka 2.2.0 was released on March 22, 2019. > > At this point, there are 29 resolved JIRA issues scheduled for inclusion in > 2.2.1: >

Re: [VOTE] KIP-417: Allow JmxTool to connect to a secured RMI port

+1 (non-binding). +2 (binding), +1 (non-binding) until now. Thanks, Dongjin On Thu, Apr 25, 2019 at 6:16 AM Gwen Shapira wrote: > +1 (binding) > > On Fri, Feb 1, 2019, 8:20 AM Harsha Chintalapani wrote: > > > +1 (binding). > > > > Thanks, > > Harsha > > On Jan 31, 2019, 8:08 PM -0800, Manikum

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

Hi Folks, This thread sunk a bit but I'd like to bump it hoping to get some feedback and/or votes. Thanks, Viktor On Thu, Mar 28, 2019 at 8:47 PM Viktor Somogyi-Vass wrote: > Sorry, the end of the message cut off. > > So I tried to be consistent with the convention in LogManager, hence the > h

Re: [VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

I'd like to encourage some more votes on KIP-419. It's a pretty small KIP to make it easier to handle resource clean up in Kafka Connect SourceTasks. Currently only +2 non-binding. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-419%3A+Safely+notify+Kafka+Connect+SourceTask+is+stoppe

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

On Wed, Apr 24, 2019, at 21:04, Jose Armando Garcia Sancio wrote: > Thanks for the reply. Comments below. > > On Wed, Apr 24, 2019 at 6:07 PM Colin McCabe wrote: > > > Hi Jose, > > > > Thanks for the KIP, looks valuable. > > > > If I use a PreferredLeaderElection RPC to specifically request an u

Re: [ANNOUNCE] New Kafka PMC member: Matthias J. Sax

Congrats Matthias! On Tue, Apr 23, 2019 at 4:24 AM Becket Qin wrote: > Congrats, Matthias! > > On Sat, Apr 20, 2019 at 10:28 AM Matthias J. Sax > wrote: > > > Thank you all! > > > > -Matthias > > > > > > On 4/19/19 3:58 PM, Lei Chen wrote: > > > Congratulations Matthias! Well deserved! > > > >

Re: [ANNOUNCE] New Kafka PMC member: Sriharsh Chintalapan

Congrats Harsha! On Tue, Apr 23, 2019 at 6:13 AM Becket Qin wrote: > Congrats, Harsh! > > On Tue, Apr 23, 2019 at 5:41 AM Colin McCabe wrote: > > > Congratulations, Harsh! > > > > cheers, > > Colin > > > > > > On Thu, Apr 18, 2019, at 11:46, Jun Rao wrote: > > > Hi, Everyone, > > > > > > Srihar

[jira] [Created] (KAFKA-8290) Streams Not Closing Fenced Producer On Task Close

Bill Bejeck created KAFKA-8290: -- Summary: Streams Not Closing Fenced Producer On Task Close Key: KAFKA-8290 URL: https://issues.apache.org/jira/browse/KAFKA-8290 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

Hi Viktor, This looks good. Just one question I had is whether we may as well cover the log dir fetchers as well. Thanks, Jason On Thu, Apr 25, 2019 at 7:46 AM Viktor Somogyi-Vass wrote: > Hi Folks, > > This thread sunk a bit but I'd like to bump it hoping to get some feedback > and/or votes.

[jira] [Created] (KAFKA-8291) System test consumer_test.py failed on trunk

Boyang Chen created KAFKA-8291: -- Summary: System test consumer_test.py failed on trunk Key: KAFKA-8291 URL: https://issues.apache.org/jira/browse/KAFKA-8291 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

On Thu, Apr 25, 2019 at 8:22 AM Colin McCabe wrote: > > On Wed, Apr 24, 2019, at 21:04, Jose Armando Garcia Sancio wrote: > > Thanks for the reply. Comments below. > > > > On Wed, Apr 24, 2019 at 6:07 PM Colin McCabe wrote: > > > What's the rationale for using an int8 rather than just having a bo

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

Hi Jose, This looks useful. One comment I had is whether we can improve the leader election tool. Needing to provide a json file is a bit annoying. Could we have a way to specify partitions directly through the command line? Often when we need to enable unclean leader election, it is just one or a

[jira] [Resolved] (KAFKA-8288) KStream.through consumer does not use custom TimestampExtractor

[ https://issues.apache.org/jira/browse/KAFKA-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maarten resolved KAFKA-8288. Resolution: Invalid > KStream.through consumer does not use custom TimestampExtractor > ---

Jenkins build is back to normal : kafka-trunk-jdk8 #3582

See

Jenkins build is back to normal : kafka-2.0-jdk8 #256

See

[jira] [Created] (KAFKA-8292) Add support for --version parameter to command line tools

Sönke Liebau created KAFKA-8292: --- Summary: Add support for --version parameter to command line tools Key: KAFKA-8292 URL: https://issues.apache.org/jira/browse/KAFKA-8292 Project: Kafka Issue T

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Hi Magesh, Agreed that we should avoid `dlq.admin`. I also don't have a strong opinion between `connector.` and `.override`, but I have a slight inclination toward `.override` since `connector.` feels a little redundant given that the whole configuration is for the connector and the use of "overri

[jira] [Resolved] (KAFKA-8291) System test consumer_test.py failed on trunk

[ https://issues.apache.org/jira/browse/KAFKA-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8291. -- Resolution: Fixed Fix Version/s: 2.3.0 > System test consumer_test.py failed on trunk >

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

On Thu, Apr 25, 2019 at 11:47 AM Jason Gustafson wrote: > > Hi Jose, > > This looks useful. One comment I had is whether we can improve the leader > election tool. Needing to provide a json file is a bit annoying. Could we > have a way to specify partitions directly through the command line? Often

Build failed in Jenkins: kafka-2.2-jdk8 #93

See Changes: [jason] MINOR: Do not log retriable offset commit exceptions as errors (#5904) -- [...truncated 2.70 MB...] kafka.controller.PartitionLeaderElectionAlgorithmsTes

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

Hi Magesh, Expanding the type we use to convey cluster metadata from just a Kafka cluster ID string to its own class seems like a good idea for the sake of forwards compatibility, but I'm still not sure what the gains of including the cluster group ID would be--it's a simple map lookup away in the

Jenkins build is back to normal : kafka-2.1-jdk8 #173

See

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

HI Chrise, Overall it looks good to me. Just one last comment - I was wondering if ConnectClusterDetail should be an interface just like ConnectClusterState. Thanks, Magesh On Thu, Apr 25, 2019 at 3:54 PM Chris Egerton wrote: > Hi Magesh, > > Expanding the type we use to convey cluster metadat

Build failed in Jenkins: kafka-trunk-jdk11 #465

See Changes: [wangguoz] KAFKA-8291 : System test fix (#6637) -- [...truncated 2.38 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldNotAllowNullProducerRe

[jira] [Resolved] (KAFKA-8287) JVM global map to fence duplicate client id

[ https://issues.apache.org/jira/browse/KAFKA-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-8287. Resolution: Invalid We don't want to do it for now because there are existing use cases where `cli

[DISCUSS] KIP-463: Auto-configure serdes passed alongside TopologyBuilder

Hi all, Due to issues that was discovered during the first attempt to implement a solution for the KAFKA-3729 ( https://issues.apache.org/jira/browse/KAFKA-3729), a KIP was thought to be necessary. There are a couple of alternatives by which we can proceed, so it would be good if we could discern

Jenkins build is back to normal : kafka-trunk-jdk11 #466

See

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

Hi Magesh, Sounds good; I've updated the KIP to make ConnectClusterDetails an interface. If we want to leave the door open to expand it in the future it definitely makes sense to treat it similarly to how we're treating the ConnectClusterState interface now. Thanks, Chris On Thu, Apr 25, 2019 a

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hey all, there is a minor change to the stream side logic for static membership. Originally we chose to piggy-back on user to supply a unique `client.id` config that

Re:Re: [VOTE] KIP-417: Allow JmxTool to connect to a secured RMI port

Hi All, This KIP passes with three +1 (binding) votes. Thanks to Manikumar for guiding me through my first KIP! Cheers, Fangbin At 2019-04-25 21:28:16, "Dongjin Lee" wrote: >+1 (non-binding). > >+2 (binding), +1 (non-binding) until now. > >Thanks, >Dongjin > >On Thu, Apr 25, 2019 at 6:16 A

Re: [DISCUSS] KIP-440: Extend Connect Converter to support headers

Hi Randall, Konstantine, I've updated the KIP to reflect the details we discussed here. Let me know if it looks good and we can go to the voting phase. Thanks! On 2019/04/22 21:07:31, Randall Hauch wrote: > I think it would be helpful to clarify this in the KIP, just so that > readers are awa

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

Thanks a lot, Chris. The KIP looks good to me. On Thu, Apr 25, 2019 at 7:35 PM Chris Egerton wrote: > Hi Magesh, > > Sounds good; I've updated the KIP to make ConnectClusterDetails an > interface. If we want to leave the door open to expand it in the future it > definitely makes sense to treat i

Re: [DISCUSS] KIP-451: Make TopologyTestDriver output iterable

Thanks for the discussion! After skipping over KIP-456, I was also wondering what the overlap of both KIPs are, and if KIP-456 might actually subsume KIP-451. If we all agree on this, I would recommend to discard KIP-451 in favor of KIP-456. I'll also follow up on KIP-456 DISCUSS thread to get s

[VOTE] KIP-462 : Use local thread id for KStreams

Hey friends, I would like to start the vote for KIP-462: https://cwiki.apache.org/confluence/display/KAFKA/KIP-462%3A+Use+local+thread+id+for+KStreams if you have questions, feel free to raise in the discussion thread, not this one. Thank you! Boyang

[DISCUSS] KIP-462 : Use local thread id for KStreams

Hey friends, I would like to start discussion for a very small KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-462%3A+Use+local+thread+id+for+KStreams it is trying to avoid sharing thread-id increment between multiple stream instances configured in one JVM. This is an important fix fo

Re: [DISCUSS] KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriver

Thanks for the KIP! I was just comparing this KIP with KIP-451 (even if I did not yet make a sorrow read over this KIP), and I agree that there is a big overlap. It seems that KIP-456 might subsume KIP-451. Let's wait on Patrick's input to see how to proceed. -Matthias On 4/25/19 12:03 AM, Juk

Re: [DISCUSS] KIP-451: Make TopologyTestDriver output iterable

Hi It's fine by me, I'll consider doing the PR a good exercise ;-) I'll discard it for the moment, if there should be any hold-up for KIP-456 to make it into the 2.3 release then we could still revive this one because it is mostly done already. best regards and thanks to Jukka for his great work! P