Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

> because assignors are sticky. I forgot about that spec again :( David Jacot 於 2024年12月20日 週五 下午3:41寫道： > Hi Chia-Ping, > > DJ08: In my opinion, changing the format will be rare so it is > acceptable if rebalances are triggered in this case on > upgrade/downgrade. It is also what will happe

[jira] [Resolved] (KAFKA-18262) Remove DefaultPartitioner and UniformStickyPartitioner

[ https://issues.apache.org/jira/browse/KAFKA-18262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18262. Resolution: Fixed trunk: [https://github.com/apache/kafka/commit/753a00348057e43002dd205e

Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

Hi Chia-Ping, DJ08: In my opinion, changing the format will be rare so it is acceptable if rebalances are triggered in this case on upgrade/downgrade. It is also what will happen if a cluster is downgraded from 4.1 (with this KIP) to 4.0. The rebalance won't change anything if the topology of the

[jira] [Resolved] (KAFKA-18296) Remove deprecated KafkaBasedLog constructor

[ https://issues.apache.org/jira/browse/KAFKA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18296. Fix Version/s: 4.0.0 Resolution: Fixed trunk: [https://github.com/apache/kafka/com

[jira] [Resolved] (KAFKA-12829) Remove Deprecated methods can classes of old Processor API

[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12829. - Resolution: Fixed > Remove Deprecated methods can classes of old Processor API > ---

[jira] [Resolved] (KAFKA-12822) Remove Deprecated APIs of Kafka Streams in 4.0

[ https://issues.apache.org/jira/browse/KAFKA-12822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12822. - Resolution: Fixed > Remove Deprecated APIs of Kafka Streams in 4.0 > ---

[jira] [Resolved] (KAFKA-18292) Remove deprecated methods of UpdateFeaturesOptions

[ https://issues.apache.org/jira/browse/KAFKA-18292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18292. Resolution: Fixed trunk: https://github.com/apache/kafka/commit/bd27e34f2d61735e4be144e07

Jenkins build is still unstable: Kafka » Kafka PowerPC Daily » test-powerpc #153

See

Re: [VOTE] KIP-1034: Dead letter queue in Kafka Streams

I think I did miss the train to vote originally, but I am +1 (binding) on the updated KIP. -Matthias On 12/18/24 12:47 AM, Bruno Cadonna wrote: Hi, We came to a conclusion in the discussion thread and Sébastien updated the KIP. IMO, we can recast our votes. +1 (binding) Thanks for the

Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

ummm, it does not work for downgrade as the old coordinator has no idea about new format :( On 2024/12/20 00:57:27 Chia-Ping Tsai wrote: > hi David > > > DJ08: > > That's a good question. If the "hash" lacks version control, it could trigger > a series of unnecessary rebalances. However, add

Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

ummm, it does not work for downgrade as the old coordinator has no idea about new format :( On 2024/12/20 00:57:27 Chia-Ping Tsai wrote: > hi David > > > DJ08: > > That's a good question. If the "hash" lacks version control, it could trigger > a series of unnecessary rebalances. However, add

Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

hi David > DJ08: That's a good question. If the "hash" lacks version control, it could trigger a series of unnecessary rebalances. However, adding additional information ("magic") to the hash does not help the upgraded coordinator determine the "version." This means that the upgraded coordina

[jira] [Resolved] (KAFKA-18306) TransactionsTest testFailureToFenceEpoch is flaky

[ https://issues.apache.org/jira/browse/KAFKA-18306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-18306. Fix Version/s: 4.0.0 Resolution: Fixed > TransactionsTest testFailureToFenceEpoch i

[jira] [Resolved] (KAFKA-18092) TransactionsTest.testBumpTransactionalEpochWithTV2Enabled is flaky

[ https://issues.apache.org/jira/browse/KAFKA-18092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-18092. Fix Version/s: 4.0.0 Resolution: Fixed > TransactionsTest.testBumpTransactionalEpoc

[jira] [Resolved] (KAFKA-18035) TransactionsTest testBumpTransactionalEpochWithTV2Disabled failed on trunk

[ https://issues.apache.org/jira/browse/KAFKA-18035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-18035. Fix Version/s: 4.0.0 Resolution: Fixed > TransactionsTest testBumpTransactionalEpoc

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #129

See

[jira] [Resolved] (KAFKA-18207) Create TransactionCoordinatorRecordSerde for handling transaction records

[ https://issues.apache.org/jira/browse/KAFKA-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-18207. Fix Version/s: 4.1.0 Resolution: Fixed > Create TransactionCoordinatorRecordSerde f

[jira] [Created] (KAFKA-18330) Update documentation to remove deploying limitations

José Armando García Sancio created KAFKA-18330: -- Summary: Update documentation to remove deploying limitations Key: KAFKA-18330 URL: https://issues.apache.org/jira/browse/KAFKA-18330 Proje

[jira] [Resolved] (KAFKA-14779) Add ACL Authorizer integration test for authorized OffsetCommits with an unknown topic

[ https://issues.apache.org/jira/browse/KAFKA-14779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14779. - Resolution: Won't Do > Add ACL Authorizer integration test for authorized OffsetCommits with an

[jira] [Resolved] (KAFKA-18169) Mark KIP-848's API & Records as stable

[ https://issues.apache.org/jira/browse/KAFKA-18169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-18169. - Resolution: Fixed Done by https://github.com/apache/kafka/pull/18235. > Mark KIP-848's API & Re

[jira] [Created] (KAFKA-18329) Delete old group coordinator

David Jacot created KAFKA-18329: --- Summary: Delete old group coordinator Key: KAFKA-18329 URL: https://issues.apache.org/jira/browse/KAFKA-18329 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-14852) Propagate Topic Ids to the Group Coordinator for Offset Fetch

[ https://issues.apache.org/jira/browse/KAFKA-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14852. - Resolution: Won't Do > Propagate Topic Ids to the Group Coordinator for Offset Fetch > -

[jira] [Created] (KAFKA-18328) Support for Topic Ids in Offset Fetch/Commit APIs (KIP-848)

David Jacot created KAFKA-18328: --- Summary: Support for Topic Ids in Offset Fetch/Commit APIs (KIP-848) Key: KAFKA-18328 URL: https://issues.apache.org/jira/browse/KAFKA-18328 Project: Kafka Is

[jira] [Created] (KAFKA-18327) Client Side Assignors (KIP-848)

David Jacot created KAFKA-18327: --- Summary: Client Side Assignors (KIP-848) Key: KAFKA-18327 URL: https://issues.apache.org/jira/browse/KAFKA-18327 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-15268) Consider replacing Subscription Metadata by a hash

[ https://issues.apache.org/jira/browse/KAFKA-15268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15268. - Resolution: Duplicate > Consider replacing Subscription Metadata by a hash > ---

[jira] [Resolved] (KAFKA-14940) Refactor ApiMessageAndVersion/Record

[ https://issues.apache.org/jira/browse/KAFKA-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14940. - Resolution: Duplicate > Refactor ApiMessageAndVersion/Record > -

[jira] [Resolved] (KAFKA-15225) Define constants for record types

[ https://issues.apache.org/jira/browse/KAFKA-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15225. - Resolution: Duplicate > Define constants for record types > - >

[jira] [Resolved] (KAFKA-18254) Unit Tests for Modified Code

[ https://issues.apache.org/jira/browse/KAFKA-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swikar Patel resolved KAFKA-18254. -- Resolution: Done > Unit Tests for Modified Code > > >

RE: [DISCUSS] KIP-1121: Compression acceleration in Kafka

Hi Frédérik, Thanks for the suggestion to use the existing ServiceLoader feature in Java. It could serve as suitable mechanism for selecting the best available compression codec. Thanks -Original Message- From: Frédérik Rouleau Sent: Wednesday, December 18, 2024 12:24 AM To: dev@kafk

[jira] [Resolved] (KAFKA-18255) Integration Tests (Kafka Specific)

[ https://issues.apache.org/jira/browse/KAFKA-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swikar Patel resolved KAFKA-18255. -- Resolution: Done > Integration Tests (Kafka Specific) > -- > >

[jira] [Resolved] (KAFKA-18252) Implement Changes for Specific API

[ https://issues.apache.org/jira/browse/KAFKA-18252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swikar Patel resolved KAFKA-18252. -- Resolution: Done > Implement Changes for Specific API > --- >

[jira] [Resolved] (KAFKA-18250) Research Alternative APIs/Solutions

[ https://issues.apache.org/jira/browse/KAFKA-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swikar Patel resolved KAFKA-18250. -- Resolution: Done > Research Alternative APIs/Solutions > --- >

Pull Request: Upgrade RocksDB to 9.7.3

Hi Bruno, Matthias, I've created a pull request to upgrade RocksDB to 9.7.3: https://github.com/apache/kafka/pull/18275 Jira issue: KAFKA-15443 Any feedback and any changes you want to me to do please let me know. Thanks, Swikar

Re: [PR] MINOR: update download links [kafka-site]

mjsax merged PR #655: URL: https://github.com/apache/kafka-site/pull/655 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.or

[jira] [Created] (KAFKA-18326) Cached stores may return deleted values

Almog Gavra created KAFKA-18326: --- Summary: Cached stores may return deleted values Key: KAFKA-18326 URL: https://issues.apache.org/jira/browse/KAFKA-18326 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-18325) Add TargetAssignmentBuilder

Lucas Brutschy created KAFKA-18325: -- Summary: Add TargetAssignmentBuilder Key: KAFKA-18325 URL: https://issues.apache.org/jira/browse/KAFKA-18325 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-18324) Add CurrentAssignmentBuilder

Lucas Brutschy created KAFKA-18324: -- Summary: Add CurrentAssignmentBuilder Key: KAFKA-18324 URL: https://issues.apache.org/jira/browse/KAFKA-18324 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-18323) Add StreamsGroup class

Lucas Brutschy created KAFKA-18323: -- Summary: Add StreamsGroup class Key: KAFKA-18323 URL: https://issues.apache.org/jira/browse/KAFKA-18323 Project: Kafka Issue Type: Sub-task R

[jira] [Created] (KAFKA-18322) Add StickyTaskAssignor

Lucas Brutschy created KAFKA-18322: -- Summary: Add StickyTaskAssignor Key: KAFKA-18322 URL: https://issues.apache.org/jira/browse/KAFKA-18322 Project: Kafka Issue Type: Sub-task R

[jira] [Created] (KAFKA-18321) Add StreamsGroupMember, MemberState and Assignment classes

Lucas Brutschy created KAFKA-18321: -- Summary: Add StreamsGroupMember, MemberState and Assignment classes Key: KAFKA-18321 URL: https://issues.apache.org/jira/browse/KAFKA-18321 Project: Kafka

[jira] [Created] (KAFKA-18320) Ensure that assignors are at the right place

David Jacot created KAFKA-18320: --- Summary: Ensure that assignors are at the right place Key: KAFKA-18320 URL: https://issues.apache.org/jira/browse/KAFKA-18320 Project: Kafka Issue Type: Sub-ta

[jira] [Created] (KAFKA-18319) Add task assignor interfaces

Lucas Brutschy created KAFKA-18319: -- Summary: Add task assignor interfaces Key: KAFKA-18319 URL: https://issues.apache.org/jira/browse/KAFKA-18319 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

Hi, I'm following this discussion thread. I may have more comments later, but for now - with KIP-1071 being accepted, could we just include the updates to KIP-1071 in this KIP? They'd be mostly identical, except that we don't have to use tagged versions, because no version of the records were rele

Re: [DISCUSS] KIP-1101: Trigger rebalance on rack topology changes

Hi PoAn and Chia-Ping, Thanks for your responses. DJ02: Sorry, I was not clear. I was wondering whether we could compute the hash without having to convert to bytes before. Guava has a nice interface for this allowing to incrementally add primitive types to the hash. We can discuss this in the PR

[jira] [Created] (KAFKA-18318) Add more logs to the migration path

Dongnuo Lyu created KAFKA-18318: --- Summary: Add more logs to the migration path Key: KAFKA-18318 URL: https://issues.apache.org/jira/browse/KAFKA-18318 Project: Kafka Issue Type: Sub-task

KafkaProducer client (Java) does not start (starts when there is a delay though)

Hi All, I already have an app implemented in Java that creates a kafka producer and consumer to connect to kafka. It is working fine. I have even run it with PLAIN and SHA256 authentication. Worked perfectly fine. Now, I have configured it to use oauthbearer. Suddenly , kafka producer does not co

[jira] [Created] (KAFKA-18317) Remove zookeeper.connect from RemoteLogManagerTest

Mickael Maison created KAFKA-18317: -- Summary: Remove zookeeper.connect from RemoteLogManagerTest Key: KAFKA-18317 URL: https://issues.apache.org/jira/browse/KAFKA-18317 Project: Kafka Issue

Re: Invalid token error while setting up OAuth2 SASL authentication on Kafka

This issue is resolved. The problem: my JWKS URL does not have public issue. To solve this, we used a file based URL to read the JWKS JSON. We are trying to talk to the Oauth2 server admin to see if the JWKS URL can be made public. In the meantime, is the above file based approach sustainable in t

[jira] [Created] (KAFKA-18316) Fix to Kraft or remove tests associate with Zk Broker config in ConnectionQuotasTest

黃竣陽 created KAFKA-18316: --- Summary: Fix to Kraft or remove tests associate with Zk Broker config in ConnectionQuotasTest Key: KAFKA-18316 URL: https://issues.apache.org/jira/browse/KAFKA-18316 Project: Kafka

[jira] [Created] (KAFKA-18315) Fix to Kraft or remove tests associate with Zk Broker config in DynamicBrokerConfigTest, ReplicaManagerTest, DescribeTopicPartitionsRequestHandlerTest, KafkaConfigTest

黃竣陽 created KAFKA-18315: --- Summary: Fix to Kraft or remove tests associate with Zk Broker config in DynamicBrokerConfigTest, ReplicaManagerTest, DescribeTopicPartitionsRequestHandlerTest, KafkaConfigTest Key: KAFKA-18315 UR

[jira] [Created] (KAFKA-18314) Fix to Kraft or remove tests associate with Zk Broker config in KafkaApisTest

黃竣陽 created KAFKA-18314: --- Summary: Fix to Kraft or remove tests associate with Zk Broker config in KafkaApisTest Key: KAFKA-18314 URL: https://issues.apache.org/jira/browse/KAFKA-18314 Project: Kafka

[jira] [Created] (KAFKA-18313) Fix to Kraft or remove tests associate with Zk Broker config in SocketServerTest and ReplicaFetcherThreadTest

黃竣陽 created KAFKA-18313: --- Summary: Fix to Kraft or remove tests associate with Zk Broker config in SocketServerTest and ReplicaFetcherThreadTest Key: KAFKA-18313 URL: https://issues.apache.org/jira/browse/KAFKA-18313

[jira] [Created] (KAFKA-18312) Add "entityType": "topicName" to "SubscribedTopicNames" in ShareGroupHeartbeatRequest.json

Chirag Wadhwa created KAFKA-18312: - Summary: Add "entityType": "topicName" to "SubscribedTopicNames" in ShareGroupHeartbeatRequest.json Key: KAFKA-18312 URL: https://issues.apache.org/jira/browse/KAFKA-18312

[jira] [Resolved] (KAFKA-18309) Add ConsumerGroupRegularExpression record to dump-log tool

[ https://issues.apache.org/jira/browse/KAFKA-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-18309. - Resolution: Not A Problem > Add ConsumerGroupRegularExpression record to dump-log tool > ---

[jira] [Created] (KAFKA-18311) Port tooling to configure topologies to broker

Lucas Brutschy created KAFKA-18311: -- Summary: Port tooling to configure topologies to broker Key: KAFKA-18311 URL: https://issues.apache.org/jira/browse/KAFKA-18311 Project: Kafka Issue Type

Re: [DISCUSS] KIP-1034: Dead letter queue in Kafka Streams

Hi, I have updated the KIP with the correct names! Thanks, Bruno, for the clarification regarding Result#from(XxxExceptionHandlerResponse). We can also make it private, as there is no reason for a user to access it. The @deprecated is no longer needed. regards Sébastien _

Re: [DISCUSS] KIP-1034: Dead letter queue in Kafka Streams

Hi, Putting Result#from(XxxExceptionHandlerResponse) into the interface and deprecating it was kind of my proposal. My thought was to avoid importing internal classes into the public interface and to have everything that is needed to understand the default implementation of the new methods in