Re: [VOTE] KIP-1052: Enable warmup in producer performance test

Hi Matt, +1 (non binding) from me. Thanks! Just a suggestion: I think that the following output line does not add much value and could be removed. "Warmup first 10 records. Steady-state results will print after the complete-test summary." On Wed, Aug 14, 2024 at 8:06 PM Welch, Matt wrote: >

[jira] [Created] (KAFKA-17362) Flaky test: kafka.api.AuthorizerIntegrationTest."testDeleteRecordsWithWildCardAuth(String).quorum=kraft"

Apoorv Mittal created KAFKA-17362: - Summary: Flaky test: kafka.api.AuthorizerIntegrationTest."testDeleteRecordsWithWildCardAuth(String).quorum=kraft" Key: KAFKA-17362 URL: https://issues.apache.org/jira/browse/KAF

[jira] [Created] (KAFKA-17363) Flaky test: kafka.api.PlaintextConsumerCommitTest.testAutoCommitOnRebalance(String, String).quorum=kraft+kip848.groupProtocol=consumer

Apoorv Mittal created KAFKA-17363: - Summary: Flaky test: kafka.api.PlaintextConsumerCommitTest.testAutoCommitOnRebalance(String, String).quorum=kraft+kip848.groupProtocol=consumer Key: KAFKA-17363 URL: https://is

[jira] [Created] (KAFKA-17364) Flaky test: org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleQueryStandbyStateDuringRebalance()

Apoorv Mittal created KAFKA-17364: - Summary: Flaky test: org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleQueryStandbyStateDuringRebalance() Key: KAFKA-17364 URL: https://issues.apach

[jira] [Created] (KAFKA-17365) Flaky test: org.apache.kafka.tools.reassign.ReassignPartitionsCommandTest.testProduceAndConsumeWithReassignmentInProgress

Apoorv Mittal created KAFKA-17365: - Summary: Flaky test: org.apache.kafka.tools.reassign.ReassignPartitionsCommandTest.testProduceAndConsumeWithReassignmentInProgress Key: KAFKA-17365 URL: https://issues.apache.or

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Thanks for the KIP and the cleanup! +1 (binding). Lucas On Sun, Aug 18, 2024 at 7:37 PM Chia-Ping Tsai wrote: > > +1 (binding) > > Kuan Po Tseng 於 2024年8月19日 週一 上午12:41寫道： > > > Hello everyone, > > > > After reviewing the feedback received ( > > https://lists.apache.org/thread/6wvbfkjldm8gnbf9

[jira] [Created] (KAFKA-17366) Flaky test: org.apache.kafka.tools.reassign.ReassignPartitionsCommandTest.testAlterReassignmentThrottle

Apoorv Mittal created KAFKA-17366: - Summary: Flaky test: org.apache.kafka.tools.reassign.ReassignPartitionsCommandTest.testAlterReassignmentThrottle Key: KAFKA-17366 URL: https://issues.apache.org/jira/browse/KAFK

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

Hi TengYao, I get your point. I think smaller features definitely go too quickly through stages to even acknowledge the change. However, I would still think it's necessary to have these 2 levels separated when it comes to bigger feature initiatives. The biggest use case I see right now is to signa

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Hi, Thanks for the KIP! +1 (binding) Best, Bruno On 8/19/24 11:39 AM, Lucas Brutschy wrote: Thanks for the KIP and the cleanup! +1 (binding). Lucas On Sun, Aug 18, 2024 at 7:37 PM Chia-Ping Tsai wrote: +1 (binding) Kuan Po Tseng 於 2024年8月19日 週一 上午12:41寫道： Hello everyone, After revi

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Thanks for the KIP and nice insight. +1 (binding) Best regards. TengYao Bruno Cadonna 於 2024年8月19日 週一 下午7:40寫道： > Hi, > > Thanks for the KIP! > > +1 (binding) > > Best, > Bruno > > On 8/19/24 11:39 AM, Lucas Brutschy wrote: > > Thanks for the KIP and the cleanup! > > > > +1 (binding). > > > >

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Thanks for the KIP. +1 (binding) Best regards, Jiunn Yang

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Hi TengYao Chi and Jiunn Yang, Thank you for voting on the KIP! For your information, your votes are not binding if you are not committers for the Apache Kafka project. Please see https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Voting You can vote with "+1" or "+1 (non-bindin

Re: [DISCUSS] KIP-1074: Make the replication of internal topics configurable

Thanks Patrik for updating the KIP. One last feedback, should we update `TOPICS_EXCLUDE_DEFAULT` also to reflect the same pattern instead of `.*[\\-\\.]internal`? Best, Omnia > On 16 Aug 2024, at 13:31, Patrik Marton wrote: > > Hi All, > > I updated the KIP with the new proposal, and change

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Hi Bruno, Thanks for the clarification and for pointing that out! Sorry for the mix-up—I'll make sure to get it right next time. Appreciate the reminder! That said, I'll correct it here: I’m +1. Best, TengYao Bruno Cadonna 於 2024年8月19日 週一 下午8:16寫道： > Hi TengYao Chi and Jiunn Yang, > > Thank

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

Hi Josep Thanks for the explanation. I see your point. It makes sense to keep these levels distinct for larger initiatives like KIP-853. I agree with your perspective. Best regards, TengYao Josep Prat 於 2024年8月19日 週一 下午6:36寫道： > Hi TengYao, > > I get your point. I think smaller features defini

Re: [VOTE] KIP-1077: Deprecate `ForeachProcessor` and move to internal package

Hi Bruno, Thanks for your reminder. Sorry for the wrong voting. I'm +1 (non-binding). Best regards, Jiunn Yang

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Hi Nick, NT4: As discussed, we will still require locking in the new protocol to avoid concurrent read/write access on the checkpoint file, at least as long as KIP-1035 hasn't landed. However, as you correctly pointed out, the assignor will have to accept offsets for overlapping sets of dormant ta

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

Hi TengYao and Chia-Ping, I updated the KIP page with examples. Best, On Mon, Aug 19, 2024 at 2:39 PM TengYao Chi wrote: > Hi Josep > > Thanks for the explanation. I see your point. > It makes sense to keep these levels distinct for larger initiatives like > KIP-853. I agree with your perspect

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Hi Guozhang, Thanks for reviewing the KIP, your feedback is extremely valuable. I think your analysis is quite right - we care about cases a) and b) and I generally agree - we want the protocol to be simple and debuggable. Situation a) should be relatively rare since in the common case all strea

[jira] [Resolved] (KAFKA-17330) Introduce share-coordinator module and add resources

[ https://issues.apache.org/jira/browse/KAFKA-17330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushant Mahajan resolved KAFKA-17330. - Resolution: Fixed > Introduce share-coordinator module and add resources > -

[jira] [Created] (KAFKA-17367) Add share coordinator implementation

Sushant Mahajan created KAFKA-17367: --- Summary: Add share coordinator implementation Key: KAFKA-17367 URL: https://issues.apache.org/jira/browse/KAFKA-17367 Project: Kafka Issue Type: Sub-ta

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Hi Lucas, >From the current description in section "Topology updates", my understanding is that a) coordinator will remember a topology ID as the group topology ID, which has to be initialized and agreed by everyone in the current generation; b) when forming a new generation, if some members has a

[jira] [Resolved] (KAFKA-12989) MockClient should respect the request matcher passed to prepareUnsupportedVersionResponse

[ https://issues.apache.org/jira/browse/KAFKA-12989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12989. Fix Version/s: 4.0.0 Resolution: Fixed > MockClient should respect the request matc

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

Hi everyone, I have updated the proposed solution of the KIP. Instead of extending the DescribeQuorum API, it now suggests updating the DescribeCluster API to optionally return fenced brokers. This should address the comments from Colin and Jose. Please let me know if you have any questions. Reg

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Hi Guozhang, thanks for clarifying. I think I understand better what you meant now, However, my question remains - wouldn't that effectively make a "rolling bounce" like an offline upgrade, if the application effectively halts processing during the roll? I agree that could be simpler, but it would

Re: [DISCUSS] KIP-891: Running multiple versions of Connector plugins

Hi Snehashis, Sorry for the late reply. > Heterogeneous dependencies in a multi cluster deployment is highly discouraged Right, this remains unchanged in this KIP. > Let's add the version information for both connector and tasks in the connector status itself > once we add these two additions t

Re: [VOTE]: KIP-1006: Remove SecurityManager Support

Hi Luke, Thanks for the vote! I have updated the compatibility section to clarify your questions, let me know if anything else is unclear. For anyone interested, the PR for this is almost ready to merge but isn't blocked by this vote, so please take a look! https://github.com/apache/kafka/pull/1

[jira] [Created] (KAFKA-17368) Add delivery count to kafka-console-share-consumer.sh

Andrew Schofield created KAFKA-17368: Summary: Add delivery count to kafka-console-share-consumer.sh Key: KAFKA-17368 URL: https://issues.apache.org/jira/browse/KAFKA-17368 Project: Kafka

[jira] [Resolved] (KAFKA-15203) Remove dependency on Reflections

[ https://issues.apache.org/jira/browse/KAFKA-15203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15203. - Fix Version/s: 4.0.0 (was: 5.0.0) Assignee: (was: Ganesh Sada

[jira] [Resolved] (KAFKA-17357) Replace the deprecated `retry.maxRetries`/`retry.maxFailures` by `develocity.testRetry.maxRetries`/`develocity.testRetry.maxFailures`/

[ https://issues.apache.org/jira/browse/KAFKA-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17357. Fix Version/s: 4.0.0 Resolution: Fixed > Replace the deprecated `retry.maxRetries`/

[jira] [Created] (KAFKA-17369) Remove org.reflections from logging and licenses

Greg Harris created KAFKA-17369: --- Summary: Remove org.reflections from logging and licenses Key: KAFKA-17369 URL: https://issues.apache.org/jira/browse/KAFKA-17369 Project: Kafka Issue Type: Ta

ExpiringCredentialRefreshingLogin causing admin close to block forever

Hi all, I am looking for a bit of guidance on an issue I am seeing in a Structured Streaming (Spark Streaming) query being blocked due to a thread in ExpiringCredentialRefreshingLogin not being able to shutdown. The reason why the streaming query is blocked from making progress is because the kaf

[jira] [Created] (KAFKA-17370) Move LeaderAndIsr from core module to metadata module

Chia-Ping Tsai created KAFKA-17370: -- Summary: Move LeaderAndIsr from core module to metadata module Key: KAFKA-17370 URL: https://issues.apache.org/jira/browse/KAFKA-17370 Project: Kafka Iss

[jira] [Created] (KAFKA-17371) Flaky test in DefaultTaskExecutorTest.shouldUnassignTaskWhenRequired

Ao Li created KAFKA-17371: - Summary: Flaky test in DefaultTaskExecutorTest.shouldUnassignTaskWhenRequired Key: KAFKA-17371 URL: https://issues.apache.org/jira/browse/KAFKA-17371 Project: Kafka Issue

[jira] [Created] (KAFKA-17372) Move `ThrottledChannelExpirationTest` and `ThrottledChannel` to server module

Chia-Ping Tsai created KAFKA-17372: -- Summary: Move `ThrottledChannelExpirationTest` and `ThrottledChannel` to server module Key: KAFKA-17372 URL: https://issues.apache.org/jira/browse/KAFKA-17372 Pro

[jira] [Resolved] (KAFKA-17355) Update TestFeatureVersion 2 to always be based on MetadataVersion.latestTesting

[ https://issues.apache.org/jira/browse/KAFKA-17355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17355. Fix Version/s: 4.0.0 Resolution: Fixed > Update TestFeatureVersion 2 to always be b

[jira] [Created] (KAFKA-17373) add print.epoch to kafka-console-share-consumer.sh/kafka-console-consumer.sh

Chia-Ping Tsai created KAFKA-17373: -- Summary: add print.epoch to kafka-console-share-consumer.sh/kafka-console-consumer.sh Key: KAFKA-17373 URL: https://issues.apache.org/jira/browse/KAFKA-17373 Proj

[jira] [Resolved] (KAFKA-17368) Add delivery count to kafka-console-share-consumer.sh

[ https://issues.apache.org/jira/browse/KAFKA-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17368. Fix Version/s: 4.0.0 Resolution: Fixed > Add delivery count to kafka-console-share-

[jira] [Created] (KAFKA-17374) add bootstrap.controller to kafka-reassign-partitions.sh

Chia-Ping Tsai created KAFKA-17374: -- Summary: add bootstrap.controller to kafka-reassign-partitions.sh Key: KAFKA-17374 URL: https://issues.apache.org/jira/browse/KAFKA-17374 Project: Kafka

[jira] [Resolved] (KAFKA-17180) Upgrade gradle from 8.8 to 8.10

[ https://issues.apache.org/jira/browse/KAFKA-17180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17180. Fix Version/s: 4.0.0 Resolution: Fixed > Upgrade gradle from 8.8 to 8.10 >

Re: [DISCUSS] KIP-1076: Metrics for client applications a KIP-714 extension

Hi Lucas, thanks for the comments. > LB1 Yes, I'll update the KIP signature to `? extends Metric` >LB2 If `enable.metrics.push` is disabled, the metrics passed in are ignored. Thinking about this now, I'm considering updating the return type of `registerAdditionalMetrics` to a `boolean` from `voi

[jira] [Resolved] (KAFKA-17369) Remove org.reflections from logging and licenses

[ https://issues.apache.org/jira/browse/KAFKA-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-17369. - Fix Version/s: 4.0.0 Resolution: Fixed > Remove org.reflections from logging and licenses

[jira] [Resolved] (KAFKA-12824) Remove Deprecated method KStream#branch

[ https://issues.apache.org/jira/browse/KAFKA-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12824. Resolution: Fixed > Remove Deprecated method KStream#branch >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3213

See

[jira] [Resolved] (KAFKA-17181) Fix possible partial write in writing snapshot (ProducerStateManager)

[ https://issues.apache.org/jira/browse/KAFKA-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17181. Fix Version/s: 4.0.0 Resolution: Fixed > Fix possible partial write in writing snap

[jira] [Resolved] (KAFKA-16900) kafka-producer-perf-test reports error when using transaction.

[ https://issues.apache.org/jira/browse/KAFKA-16900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16900. Fix Version/s: 4.0.0 Resolution: Fixed > kafka-producer-perf-test reports error whe

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Thanks for the replies Lucas. Just to help me better understand here: today, do all kinds of modifications to the processing logic, not only topological changes, but also say logical changes (e.g. changing the threshold value of a filter, or changes that do not impact the generated list of tasks as

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #38

See