Re: [DISCUSS] KIP-512: make Record Headers available in onAcknowledgement

Hi Andrew, > AS5: there should not be a default implementation with the current signature Thanks for pointing that out. But if the existing `onAcknowledgement(RecordMetadata, Exception)` method doesn't have a default implementation, users who want to use the new `onAcknowledgement(RecordMetadata,

RE: [DISCUSS] KIP-1052: Enable warmup in producer performance test

Hi again Kafka devs, I think we're about ready to vote on KIP-1052, but just want to make sure there's no additional concerns with the KIP. Apologies if it's poor form, but do the devs who commented previously have any additional input? @Divij, @Luke, @Federico I will call a vote next week if

[jira] [Resolved] (KAFKA-16905) Thread block in describe topics API in Admin Client

[ https://issues.apache.org/jira/browse/KAFKA-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16905. -- Resolution: Fixed > Thread block in describe topics API in Admin Client >

[jira] [Resolved] (KAFKA-15132) Implement disable & re-enablement for Tiered Storage

[ https://issues.apache.org/jira/browse/KAFKA-15132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15132. -- Resolution: Fixed > Implement disable & re-enablement for Tiered Storage > ---

[jira] [Resolved] (KAFKA-16667) KRaftMigrationDriver gets stuck after successive failovers

[ https://issues.apache.org/jira/browse/KAFKA-16667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16667. -- Resolution: Fixed Fixed in 3.9 > KRaftMigrationDriver gets stuck after successive failovers >

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

Thanks for the KIP Gantigmaa! I agree with the motivation but it is not clear to me that this should be solved in the KRaft layer. The KRaft leader only keeps track, in-memory, of observers that have fetched. It is possible, after a kraft leader change, for this state to get lost. If the shutdown

[jira] [Reopened] (KAFKA-16130) Write a junit test for ZK migration rollback

[ https://issues.apache.org/jira/browse/KAFKA-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe reopened KAFKA-16130: -- > Write a junit test for ZK migration rollback > > >

[jira] [Resolved] (KAFKA-14939) Only expose ZkMigrationState metric if metadata.version supports it

[ https://issues.apache.org/jira/browse/KAFKA-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14939. -- Resolution: Won't Fix > Only expose ZkMigrationState metric if metadata.version supports it >

[jira] [Resolved] (KAFKA-14304) ZooKeeper to KRaft Migration

[ https://issues.apache.org/jira/browse/KAFKA-14304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14304. -- Fix Version/s: 3.9.0 Resolution: Fixed > ZooKeeper to KRaft Migration > ---

[jira] [Resolved] (KAFKA-16130) Test migration rollback

[ https://issues.apache.org/jira/browse/KAFKA-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16130. -- Fix Version/s: (was: 3.9.0) Resolution: Fixed > Test migration rollback > -

[jira] [Resolved] (KAFKA-14833) ZK migration cannot start if a ZK controller was not previously elected

[ https://issues.apache.org/jira/browse/KAFKA-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14833. -- Resolution: Fixed > ZK migration cannot start if a ZK controller was not previously elected >

[jira] [Resolved] (KAFKA-14846) Fix overly large record batches in ZkMigrationClient

[ https://issues.apache.org/jira/browse/KAFKA-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14846. -- Resolution: Fixed > Fix overly large record batches in ZkMigrationClient > ---

[jira] [Resolved] (KAFKA-15196) Additional ZK migration metrics

[ https://issues.apache.org/jira/browse/KAFKA-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15196. -- Resolution: Fixed > Additional ZK migration metrics > --- > >

[jira] [Resolved] (KAFKA-14323) KRaft broker time based snapshots

[ https://issues.apache.org/jira/browse/KAFKA-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14323. -- Fix Version/s: 3.4.0 (was: 3.9.0) Resolution: Duplicate This was

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

On Fri, Aug 9, 2024, at 14:46, Colin McCabe wrote: > Hi Gantigmaa, > > I agree with you that we need a way to list fenced brokers. > > However, I don't think this KIP is the right way to do that. This KIP, > if I understand correctly, is about listing all Raft observers. The > broker is one examp

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

Hi Gantigmaa, I agree with you that we need a way to list fenced brokers. However, I don't think this KIP is the right way to do that. This KIP, if I understand correctly, is about listing all Raft observers. The broker is one example of a Raft observer, but there may be others. For example, th

Re: New release branch 3.9

Updates: KIP-1007: I have removed this PR from the Apache Kafka 3.9 release plan. Since the KIP was discarded, I did not add it in the "postponed to subsequent release" section. KIP-1025: Marked 3.9 status as Done KIP-1023: Postponed to subsequent release KIP-1005: Marked 3.9 status as Done

Re: New release branch 3.9

Hi Chia-Ping Tsai, Thank you! And thanks for your own contributions, especially the many code reviews (not only of my PRs, but everyone's). best, Colin On Thu, Aug 8, 2024, at 19:14, Chia-Ping Tsai wrote: >> - We have completed all the feature work for KIP-853: KRaft controller >> membership c

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3186

See

[jira] [Resolved] (KAFKA-16256) Update ConsumerConfig to validate use of group.remote.assignor and partition.assignment.strategy based on group.protocol

[ https://issues.apache.org/jira/browse/KAFKA-16256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-16256. --- Resolution: Duplicate > Update ConsumerConfig to validate use of group.remote.assignor and > partit

[jira] [Resolved] (KAFKA-17208) replica_scale_test.py fails for new consumer

[ https://issues.apache.org/jira/browse/KAFKA-17208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-17208. --- Resolution: Cannot Reproduce This seems to have been fixed as it is no longer seen after three test

[jira] [Resolved] (KAFKA-17183) New consumer system tests pass for subset of tests, but fail if running all tests

[ https://issues.apache.org/jira/browse/KAFKA-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-17183. --- Resolution: Cannot Reproduce This seems to have been fixed as it is no longer seen after three test

[jira] [Resolved] (KAFKA-17240) Try to complete delyed ops for all purgatories even though one of delayed op throws exception

[ https://issues.apache.org/jira/browse/KAFKA-17240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17240. Fix Version/s: 4.0.0 Resolution: Fixed > Try to complete delyed ops for all purgato

[jira] [Resolved] (KAFKA-16328) Remove Deprecated config from StreamsConfig

[ https://issues.apache.org/jira/browse/KAFKA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16328. Resolution: Fixed > Remove Deprecated config from StreamsConfig >

[jira] [Created] (KAFKA-17307) Remove junit-platform.properties from test JARs

Cosimo Damiano Prete created KAFKA-17307: Summary: Remove junit-platform.properties from test JARs Key: KAFKA-17307 URL: https://issues.apache.org/jira/browse/KAFKA-17307 Project: Kafka

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3185

See

[jira] [Created] (KAFKA-17306) Soften the validation when replaying tombstones

Dongnuo Lyu created KAFKA-17306: --- Summary: Soften the validation when replaying tombstones Key: KAFKA-17306 URL: https://issues.apache.org/jira/browse/KAFKA-17306 Project: Kafka Issue Type: Sub

[jira] [Created] (KAFKA-17305) Quorum controller should check the finalized kraft.version during broker registration

José Armando García Sancio created KAFKA-17305: -- Summary: Quorum controller should check the finalized kraft.version during broker registration Key: KAFKA-17305 URL: https://issues.apache.org/jira/bro

[jira] [Created] (KAFKA-17304) QuorumController crashes when kraft write a control record

José Armando García Sancio created KAFKA-17304: -- Summary: QuorumController crashes when kraft write a control record Key: KAFKA-17304 URL: https://issues.apache.org/jira/browse/KAFKA-17304

[jira] [Resolved] (KAFKA-17127) Add check for whether the requested partition exists or not in Share Fetch request

[ https://issues.apache.org/jira/browse/KAFKA-17127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chirag Wadhwa resolved KAFKA-17127. --- Resolution: Fixed > Add check for whether the requested partition exists or not in Share Fet

[jira] [Resolved] (KAFKA-17126) hande piggybacked acknowledgements asynchronously in Share Fetch and properly combine the results of fetch and acknowledge

[ https://issues.apache.org/jira/browse/KAFKA-17126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chirag Wadhwa resolved KAFKA-17126. --- Resolution: Fixed > hande piggybacked acknowledgements asynchronously in Share Fetch and pro

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #27

See

[jira] [Resolved] (KAFKA-12830) Remove Deprecated constructor in TimeWindowedDeserializer and TimeWindowedSerde

[ https://issues.apache.org/jira/browse/KAFKA-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat resolved KAFKA-12830. Resolution: Fixed > Remove Deprecated constructor in TimeWindowedDeserializer and > TimeWindowedS

[jira] [Resolved] (KAFKA-12832) Remove Deprecated methods under RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter

[ https://issues.apache.org/jira/browse/KAFKA-12832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat resolved KAFKA-12832. Resolution: Fixed > Remove Deprecated methods under > RocksDBGenericOptionsToDbOptionsColumnFamil

Re: [DISCUSS] KIP-1074: Make the replication of internal topics configurable

Hi All, Thanks again for your feedback! Regarding the boolean solution, yes this would technically let the customer accidentally replicate "__.*" topics, but only if they modify the TopicFilter to allow their replication, as by default topics matching this pattern are excluded. But in this case w

Re: kafka-native image and -javaagent

Hi Neil, I haven't personally tried. But since kafka-native contains not really a Java application, but a GraalVM-compiled native application, it doesn't support Java agents. Best, Ivan On Thu, Aug 8, 2024, at 22:36, Neil Buesing wrote: > I am attempt to set javaagents for the kafka-native im

[jira] [Resolved] (KAFKA-17228) Static member using new protocol should always replace the one using the old protocol

[ https://issues.apache.org/jira/browse/KAFKA-17228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-17228. - Fix Version/s: 4.0.0 Reviewer: David Jacot Resolution: Fixed > Static member usi

[jira] [Resolved] (KAFKA-17267) New group coordinator can return REQUEST_TIMED_OUT for OFFSET_FETCHes

[ https://issues.apache.org/jira/browse/KAFKA-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-17267. - Fix Version/s: 4.0.0 Reviewer: David Jacot Assignee: Sean Quah Resolutio

Re: [DISCUSS] KIP-512: make Record Headers available in onAcknowledgement

Hi Rich, Thanks for the updates. I’m generally happy with the KIP now with one outstanding comment. AS5: I think there’s a small error in the interface changes in the KIP. It seems to me that ProducerInterceptor should be changed to add the new `onAcknowledgement(RecordMetadata, Exception, Headers

[jira] [Resolved] (KAFKA-17217) Clients : Optimise batching of requests per node in ShareConsumeRequestManager

[ https://issues.apache.org/jira/browse/KAFKA-17217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-17217. -- Resolution: Fixed > Clients : Optimise batching of requests per node in ShareConsumeRe