Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi John, 1. Main reason was that it seemed easier change compared to having multiple tags assigned to each host. --- Answering your question what use-case I have in mind: Lets say we have two Kubernetes clusters running the same Kafka Streams application. And each Kubernetes cluster is spanne

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #433

See Changes: [github] KAFKA-12248; Add BrokerHeartbeat/BrokerRegistration RPCs for KIP-500 (#9994) -- [...truncated 3.58 MB...] TransactionsTest > testAbortT

Jenkins build is back to normal : Kafka » kafka-trunk-jdk11 #465

See

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #483

See Changes: [github] KAFKA-12248; Add BrokerHeartbeat/BrokerRegistration RPCs for KIP-500 (#9994) -- [...truncated 7.05 MB...] LogCleanerTest > testClient

Re: [DISCUSS] KIP-707: The future of KafkaFuture

I've previously discounted the possibility of an "Admin2" client, but seeing the recent discussions on the thread for KIP-706, I wonder whether this current proposal in KIP-707 would benefit from a bit more discussion... I think there are broadly two approaches to evolving the Admin client API to u

Re: Re: [DISCUSS] KIP-706: Add method "Producer#produce" to return CompletionStage instead of Future

> > Furthermore, we should align with the longer term plan for the AdminClient > that returns KafkaFuture today. > I've sketched a few ideas about that in KIP-707 and its discussion thread, if people want to share their thoughts. On Sun, Jan 31, 2021 at 5:51 AM Chia-Ping Tsai wrote: > Fair eno

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #484

See Changes: [github] MINOR: AbstractConfigTest.testClassConfigs should reset the class loa… (#10010) [github] KAFKA-12246: Remove redundant suppression in KafkaAdminClient (#9989) --

Jenkins build is back to normal : Kafka » kafka-trunk-jdk8 #434

See

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #466

See Changes: [github] MINOR: AbstractConfigTest.testClassConfigs should reset the class loa… (#10010) [github] KAFKA-12246: Remove redundant suppression in KafkaAdminClient (#9989) --

[jira] [Created] (KAFKA-12268) System tests broken because consumer returns early without records

Rajini Sivaram created KAFKA-12268: -- Summary: System tests broken because consumer returns early without records Key: KAFKA-12268 URL: https://issues.apache.org/jira/browse/KAFKA-12268 Project: Kafk

[jira] [Created] (KAFKA-12269) Replace MetadataResponse.PartitionMetadata with auto-generated protocol

dengziming created KAFKA-12269: -- Summary: Replace MetadataResponse.PartitionMetadata with auto-generated protocol Key: KAFKA-12269 URL: https://issues.apache.org/jira/browse/KAFKA-12269 Project: Kafka

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

Hi all, I have updated the interfaces. I managed to shrink the required number of entities. Basically I store the event type with the event, therefore we can cover all topic related events (create, delete, change) with one event type. I think if on-one has objections then I'll start a vote soon.

[jira] [Resolved] (KAFKA-9881) Verify whether RocksDBMetrics Get Measurements from RocksDB in a Unit Test instead of an Intergration Test

[ https://issues.apache.org/jira/browse/KAFKA-9881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-9881. -- Resolution: Fixed > Verify whether RocksDBMetrics Get Measurements from RocksDB in a Unit Test

[jira] [Resolved] (KAFKA-10366) TimeWindowedDeserializer doesn't allow users to set a custom window size

[ https://issues.apache.org/jira/browse/KAFKA-10366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leah Thomas resolved KAFKA-10366. - Fix Version/s: 2.8.0 Resolution: Fixed > TimeWindowedDeserializer doesn't allow users to

[jira] [Resolved] (KAFKA-9649) Remove/Warn on use of TimeWindowedSerde with no specified window size

[ https://issues.apache.org/jira/browse/KAFKA-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leah Thomas resolved KAFKA-9649. Fix Version/s: 2.8.0 Resolution: Fixed > Remove/Warn on use of TimeWindowedSerde with no spe

[jira] [Resolved] (KAFKA-10700) Support mutual TLS authentication for SASL_SSL listeners

[ https://issues.apache.org/jira/browse/KAFKA-10700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-10700. Reviewer: David Jacot Resolution: Fixed > Support mutual TLS authentication for SASL

[jira] [Reopened] (KAFKA-10091) Improve task idling

[ https://issues.apache.org/jira/browse/KAFKA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reopened KAFKA-10091: -- > Improve task idling > --- > > Key: KAFKA-10091 >

[jira] [Created] (KAFKA-12270) Kafka Connect may fail a task when racing to create topic

Randall Hauch created KAFKA-12270: - Summary: Kafka Connect may fail a task when racing to create topic Key: KAFKA-12270 URL: https://issues.apache.org/jira/browse/KAFKA-12270 Project: Kafka I

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi Levani, 1. Thanks for the details. I figured it must be something like this two-dimensional definition of "rack". It does seem like, if we make the config take a list of tags, we can define the semantics to be that the system will make a best effort to distribute the standbys over each rack d

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #485

See Changes: [github] KAFKA-10700 - Support mutual TLS authentication for SASL_SSL listeners (KIP-684) (#10007) -- [...truncated 3.56 MB...] PlaintextEndTo

[jira] [Created] (KAFKA-12271) Expose consistent Raft metadata to components

David Arthur created KAFKA-12271: Summary: Expose consistent Raft metadata to components Key: KAFKA-12271 URL: https://issues.apache.org/jira/browse/KAFKA-12271 Project: Kafka Issue Type: Imp

[Discuss] KAFKA-12254: MirrorMaker 2.0 creates destination topic with default configs

Hello All, Ref Link : https://issues.apache.org/jira/browse/KAFKA-12254 I'd like to seek feedback and draw attention to the design of MirrorMaker 2.0 creating a destination topic with default configs, as noted in the bug the limitation is that `computeAndCreateTopicPartitions` creates topics with

[jira] [Created] (KAFKA-12272) Kafka Streams metric commit-latency-max and commit-latency-avg is always 0

Damien Gasparina created KAFKA-12272: Summary: Kafka Streams metric commit-latency-max and commit-latency-avg is always 0 Key: KAFKA-12272 URL: https://issues.apache.org/jira/browse/KAFKA-12272 Pr

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #467

See Changes: [github] KAFKA-10700 - Support mutual TLS authentication for SASL_SSL listeners (KIP-684) (#10007) -- [...truncated 7.17 MB...] KafkaZkClientT

Re: [DISCUSS] KIP-618: Atomic commit of source connector records and offsets

Hi Jason, Fine by me; I wanted to be conservative with the return type but the case you've outlined sounds enticing enough that adding a little flexibility to the API seems warranted. I've added your suggestion to the proposed admin API expansions; let me know what you think. Cheers, Chris On M

[jira] [Created] (KAFKA-12273) InterBrokerSendThread#pollOnce throws FatalExitError even though it is shutdown correctly

Chia-Ping Tsai created KAFKA-12273: -- Summary: InterBrokerSendThread#pollOnce throws FatalExitError even though it is shutdown correctly Key: KAFKA-12273 URL: https://issues.apache.org/jira/browse/KAFKA-12273

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi John. Thanks a lot for this detailed analysis! Yes, that is what I had in mind as well. I also like that idea of having “task.assignment.awareness” configuration to tell which instance tags can be used for rack awareness. I may borrow it for this KIP if you don’t mind :) Thanks again John f

[jira] [Created] (KAFKA-12274) Transactional operation fails when broker is replaced using the same broker ID.

Michał Łukowicz created KAFKA-12274: --- Summary: Transactional operation fails when broker is replaced using the same broker ID. Key: KAFKA-12274 URL: https://issues.apache.org/jira/browse/KAFKA-12274

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #486

See Changes: [github] MINOR: adding docs for failed stream thread metric (#9974) -- [...truncated 3.53 MB...] ZooKeeperClientTest > testGetChildrenExistingZ

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #468

See Changes: [github] MINOR: adding docs for failed stream thread metric (#9974) -- [...truncated 7.01 MB...] ControllerChannelManagerTest > testStopReplic

[jira] [Created] (KAFKA-12275) KIP-500: Remove controllerOnly restriction from the DecommissionBroker API

Alok Nikhil created KAFKA-12275: --- Summary: KIP-500: Remove controllerOnly restriction from the DecommissionBroker API Key: KAFKA-12275 URL: https://issues.apache.org/jira/browse/KAFKA-12275 Project: Kaf

Jenkins build is back to normal : Kafka » kafka-2.6-jdk8 #80

See

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #487

See Changes: [github] KAFKA-12209: Add the timeline data structures for the KIP-631 controller (#9901) [github] KAFKA-10413: Allow for even distribution of lost/new tasks when multiple Connect wor

[jira] [Created] (KAFKA-12276) Add KIP-500 controller code

Colin McCabe created KAFKA-12276: Summary: Add KIP-500 controller code Key: KAFKA-12276 URL: https://issues.apache.org/jira/browse/KAFKA-12276 Project: Kafka Issue Type: Improvement

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #469

See Changes: [github] KAFKA-12209: Add the timeline data structures for the KIP-631 controller (#9901) [github] KAFKA-10413: Allow for even distribution of lost/new tasks when multiple Connect wor

[jira] [Created] (KAFKA-12277) Replace MetadataResponse.TopicMetadata with auto-generated protocol

dengziming created KAFKA-12277: -- Summary: Replace MetadataResponse.TopicMetadata with auto-generated protocol Key: KAFKA-12277 URL: https://issues.apache.org/jira/browse/KAFKA-12277 Project: Kafka

[jira] [Created] (KAFKA-12278) Keep api versions consistent with api scope

Jason Gustafson created KAFKA-12278: --- Summary: Keep api versions consistent with api scope Key: KAFKA-12278 URL: https://issues.apache.org/jira/browse/KAFKA-12278 Project: Kafka Issue Type:

Build failed in Jenkins: Kafka » kafka-2.6-jdk8 #81

See Changes: [A. Sophie Blee-Goldman] KAFKA-10362: When resuming Streams active task with EOS, the checkpoint file is deleted (#9247) -- [...truncated 2.81 MB..

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #437

See Changes: [github] KAFKA-12209: Add the timeline data structures for the KIP-631 controller (#9901) [github] KAFKA-10413: Allow for even distribution of lost/new tasks when multiple Connect work

[jira] [Created] (KAFKA-12279) Kafka 2.7 stream app issue

prabhu biradar created KAFKA-12279: -- Summary: Kafka 2.7 stream app issue Key: KAFKA-12279 URL: https://issues.apache.org/jira/browse/KAFKA-12279 Project: Kafka Issue Type: Bug Re

[jira] [Created] (KAFKA-12280) Consider internal topic configuration when created through `CreateTopic`

Jason Gustafson created KAFKA-12280: --- Summary: Consider internal topic configuration when created through `CreateTopic` Key: KAFKA-12280 URL: https://issues.apache.org/jira/browse/KAFKA-12280 Projec

Re: [apache/kafka] KAFKA-10413 Allow even distribution of lost/new tasks when more than one worker

Hi Konstantine, Thanks for reviewing the PR [https://github.com/apache/kafka/pull/9319] and merging the change. Please help assigning this JIRA to me and so that i can update the fix versions. https://issues.apache.org/jira/browse/KAFKA-10413 Thanks Ramesh On Mon, Jan 25, 2021 at 8:03 PM Rame

Re: [apache/kafka] KAFKA-10413 Allow even distribution of lost/new tasks when more than one worker

Hi Ramesh, Do you have the JIRA account? I cannot find your name. You need to have a JIRA account first, and then the committer will grant your access. Thanks. Luke On Wed, Feb 3, 2021 at 1:03 PM Ramesh Krishnan wrote: > Hi Konstantine, > > Thanks for reviewing the PR [https://github.com/apache

Re: [apache/kafka] KAFKA-10413 Allow even distribution of lost/new tasks when more than one worker

Hi Luke, Please find my Jira ID . ramkrish1489 Thanks Ramens On Wed, 3 Feb 2021 at 11:21 AM, Luke Chen wrote: > Hi Ramesh, > Do you have the JIRA account? I cannot find your name. > You need to have a JIRA account first, and then the committer will grant > your access. > > Thanks. > Luke > >

Re: [apache/kafka] KAFKA-10413 Allow even distribution of lost/new tasks when more than one worker

Hi Committers, please help Ramens add JIRA account: *ramkrish1489* into contribution list. Thanks. Luke On Wed, Feb 3, 2021 at 2:10 PM Ramesh Krishnan wrote: > Hi Luke, > > Please find my Jira ID . > > ramkrish1489 > > Thanks > Ramens > > On Wed, 3 Feb 2021 at 11:21 AM, Luke Chen wrote: > > >

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Hi Boyang, sounds good to me. -Jason On Mon, Feb 1, 2021 at 9:32 AM Boyang Chen wrote: > Hey Jason and Chia-Ping, > > looking at the discussion thread for KIP-706, we are still in discussion > with the final API version, so I made the decision to decouple the two KIPs > so that this one could m

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Thanks Jason! On Tue, Feb 2, 2021 at 11:10 PM Jason Gustafson wrote: > Hi Boyang, sounds good to me. > > -Jason > > On Mon, Feb 1, 2021 at 9:32 AM Boyang Chen > wrote: > > > Hey Jason and Chia-Ping, > > > > looking at the discussion thread for KIP-706, we are still in discussion > > with the fi

Re: [VOTE] KIP-691: Transactional Producer Exception Handling

Thanks everyone for the inputs and discussions on this KIP. I'm +1 (binding) here as well, alongside with two other committers' approvals (Guozhang and Jason), the KIP is approved. Feel free to post any follow-up questions on the discussion or voting thread if you see necessary. Best, Boyang On