Re: [DISCUSS] KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins

Hi all, I'm working on enhancing the KTable foreign key join API to allow extracting foreign keys from both key and value. However, I've encountered a Java type erasure issue that affects our API design choices. Current Situation: - We want to allow foreign key extraction from both key and value

[jira] [Created] (KAFKA-17986) Fix ConsumerRebootstrapTest and ProducerRebootstrapTest

Colin McCabe created KAFKA-17986: Summary: Fix ConsumerRebootstrapTest and ProducerRebootstrapTest Key: KAFKA-17986 URL: https://issues.apache.org/jira/browse/KAFKA-17986 Project: Kafka Issue

[jira] [Resolved] (KAFKA-14077) KRaft should support recovery from failed disk

[ https://issues.apache.org/jira/browse/KAFKA-14077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14077. -- Fix Version/s: 3.9.0 Resolution: Fixed KIP-853 shipped in Apache Kafka 3.9 > KRaft sho

[jira] [Resolved] (KAFKA-15749) KRaft support in KafkaMetricReporterClusterIdTest

[ https://issues.apache.org/jira/browse/KAFKA-15749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15749. -- Resolution: Not A Problem > KRaft support in KafkaMetricReporterClusterIdTest > --

[jira] [Resolved] (KAFKA-13827) Data Loss on moving to KRAFT mode from v3.1

[ https://issues.apache.org/jira/browse/KAFKA-13827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13827. -- Fix Version/s: 3.0.0 Resolution: Won't Fix This related to the kraft support in 2.8, wh

Jenkins build is back to stable : Kafka » Kafka PowerPC Daily » test-powerpc #115

See

Re: [VOTE] KIP-1103: Additional metrics for cooperative consumption

Hi Apoorv, Thanks for the KIP. I'm pleased to see how this augments the metrics in KIP-932. One tiny nit: the "group name" should be "group ID". +1 (non-binding) Thanks, Andrew From: Chia-Ping Tsai Sent: 08 November 2024 14:21 To: dev@kafka.apache.org

[jira] [Resolved] (KAFKA-14538) Implement KRaft metadata transactions

[ https://issues.apache.org/jira/browse/KAFKA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14538. -- Fix Version/s: 3.7.0 Resolution: Fixed > Implement KRaft metadata transactions > --

[jira] [Resolved] (KAFKA-14182) KRaft and ACL + GSSAPI

[ https://issues.apache.org/jira/browse/KAFKA-14182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14182. -- Resolution: Fixed This was fixed a long time ago, probably in 3.2 > KRaft and ACL + GSSAPI >

[jira] [Resolved] (KAFKA-14057) Support dynamic reconfiguration in KRaft remote controllers

[ https://issues.apache.org/jira/browse/KAFKA-14057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14057. -- Fix Version/s: 3.5.0 Resolution: Fixed This was fixed a long time ago, starting in 3.5

[jira] [Created] (KAFKA-17984) Move MetadataSupport to server module

黃竣陽 created KAFKA-17984: --- Summary: Move MetadataSupport to server module Key: KAFKA-17984 URL: https://issues.apache.org/jira/browse/KAFKA-17984 Project: Kafka Issue Type: Improvement Repor

[jira] [Created] (KAFKA-17983) Configurable JMX Port for Metrics Collection for Kafka MirrorMaker2

Basapuram Kumar created KAFKA-17983: --- Summary: Configurable JMX Port for Metrics Collection for Kafka MirrorMaker2 Key: KAFKA-17983 URL: https://issues.apache.org/jira/browse/KAFKA-17983 Project: Ka

Assistance Needed: High Memory Usage with Large Number of Topics in Kafka

Hi, I’m experiencing significant storage and memory challenges with our Kafka server. Currently, each Kafka topic occupies around 20MB of memory at runtime. When scaling to 1,000 topics, this setup consumes nearly 20GB of memory, which is becoming unsustainable as our data grows. Could you please

[jira] [Resolved] (KAFKA-17787) Remove --zookeeper option and logic from ConfigCommand

[ https://issues.apache.org/jira/browse/KAFKA-17787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-17787. Fix Version/s: 4.0.0 Resolution: Fixed > Remove --zookeeper option and logic from C

Re: [VOTE] KIP-1091: Improved Kafka Streams operator metrics

Hi All, The vote is now closed. KIP-1091 has been accepted with 3 binding votes (Lucas, Matthias, and Sophie) and 1 non-binding vote (Apoorv). Thanks to everyone for participating. -Bill On Wed, Nov 6, 2024 at 11:40 PM Sophie Blee-Goldman wrote: > +1 (binding) > > thanks Bill > > On Wed, Nov

[jira] [Created] (KAFKA-17985) Set default value for share.auto.offset.reset in ShareRoundTripWorker.

Shivsundar R created KAFKA-17985: Summary: Set default value for share.auto.offset.reset in ShareRoundTripWorker. Key: KAFKA-17985 URL: https://issues.apache.org/jira/browse/KAFKA-17985 Project: Kafka

[jira] [Created] (KAFKA-17990) Fix flaky ShareConsumerTest#testShareAutoOffsetResetDefaultValue

Chia-Ping Tsai created KAFKA-17990: -- Summary: Fix flaky ShareConsumerTest#testShareAutoOffsetResetDefaultValue Key: KAFKA-17990 URL: https://issues.apache.org/jira/browse/KAFKA-17990 Project: Kafka

[jira] [Resolved] (KAFKA-17975) Remove ControllerQuorumVotersFutureManager

[ https://issues.apache.org/jira/browse/KAFKA-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17975. Fix Version/s: 4.0.0 Resolution: Fixed > Remove ControllerQuorumVotersFutureManager

[jira] [Created] (KAFKA-17988) Fix flaky ReconfigurableQuorumIntegrationTest.testRemoveAndAddSameController

Chia-Ping Tsai created KAFKA-17988: -- Summary: Fix flaky ReconfigurableQuorumIntegrationTest.testRemoveAndAddSameController Key: KAFKA-17988 URL: https://issues.apache.org/jira/browse/KAFKA-17988 Proj

[jira] [Created] (KAFKA-17989) Fix flaky ShareConsumerTest#testAcknowledgeCommitCallbackCallsShareConsumerWakeup

Chia-Ping Tsai created KAFKA-17989: -- Summary: Fix flaky ShareConsumerTest#testAcknowledgeCommitCallbackCallsShareConsumerWakeup Key: KAFKA-17989 URL: https://issues.apache.org/jira/browse/KAFKA-17989

[jira] [Resolved] (KAFKA-16370) offline rollback procedure from kraft mode to zookeeper mode.

[ https://issues.apache.org/jira/browse/KAFKA-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16370. -- Resolution: Won't Fix I don't think there are any plans to implement offline rollback currentl

Re: [VOTE] KIP-866 ZooKeeper to KRaft Migration

Hi all, I was doing some JIRA cleanup and noticed someone commenting about the language in KIP-866 referencing an offline KRaft -> ZK conversion tool. Since we never ended up implementing this, I have removed it from the KIP. best, Colin On Wed, Nov 30, 2022, at 12:27, David Arthur wrote: > H

[jira] [Resolved] (KAFKA-13502) Support configuring BROKER_LOGGER on controller-only KRaft nodes

[ https://issues.apache.org/jira/browse/KAFKA-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13502. -- Fix Version/s: 3.7.0 Assignee: Colin McCabe Resolution: Fixed KIP-919 made it

[jira] [Resolved] (KAFKA-15430) Kafla create replca partition on controller node

[ https://issues.apache.org/jira/browse/KAFKA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15430. -- Resolution: Invalid Replicas aren't created "on the controller." However, the controller does

[jira] [Resolved] (KAFKA-14290) Fix bugs that could block KRaft controlled shutdown indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14290. -- Resolution: Fixed > Fix bugs that could block KRaft controlled shutdown indefinitely > ---

[jira] [Resolved] (KAFKA-9837) New RPC for notifying controller of failed replica

[ https://issues.apache.org/jira/browse/KAFKA-9837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-9837. - Fix Version/s: 3.7.0 Resolution: Fixed > New RPC for notifying controller of failed replic

[jira] [Resolved] (KAFKA-12886) Enable request forwarding by default

[ https://issues.apache.org/jira/browse/KAFKA-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12886. -- Resolution: Won't Fix At this point we won't be changing the behavior of ZK mode. Closing as "

[jira] [Resolved] (KAFKA-12733) KRaft: always bump leader epoch when changing the ISR for a controlled shutdown

[ https://issues.apache.org/jira/browse/KAFKA-12733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12733. -- Resolution: Won't Fix > KRaft: always bump leader epoch when changing the ISR for a controlled

[jira] [Resolved] (KAFKA-13083) In KRaft mode, ISR is set incorrectly when creating a manual partition assignment with fenced broker(s)

[ https://issues.apache.org/jira/browse/KAFKA-13083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13083. -- Fix Version/s: 3.0.0 Resolution: Fixed > In KRaft mode, ISR is set incorrectly when cre

[jira] [Resolved] (KAFKA-12806) KRaft: Confusing leadership status exposed in metrics for controller without quorum

[ https://issues.apache.org/jira/browse/KAFKA-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12806. -- Fix Version/s: 3.0.0 Resolution: Fixed I think this was one of many metrics bugs we fix

[jira] [Resolved] (KAFKA-15861) In Kraft mode, "ssl.keystore.key" private keys are accesible to all the controllers and brokers

[ https://issues.apache.org/jira/browse/KAFKA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15861. -- Resolution: Won't Fix If you want to distributed keys separately, consider using a keystore fi

[jira] [Resolved] (KAFKA-14312) Kraft + ProducerStateManager: produce requests to new partitions with a non-zero sequence number should be rejected

[ https://issues.apache.org/jira/browse/KAFKA-14312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14312. -- Resolution: Won't Fix Based on the discussion, this behavior isn't unique to KRaft, and isn't

[jira] [Resolved] (KAFKA-14313) Kraft: immediately producing to new topics occasionally returns NOT_LEADER_FOR_PARTITION

[ https://issues.apache.org/jira/browse/KAFKA-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14313. -- Resolution: Won't Fix This isn't a bug. It results from metadata propagation delays. The clien

[jira] [Resolved] (KAFKA-13835) Fix two bugs related to dynamic broker configs in KRaft

[ https://issues.apache.org/jira/browse/KAFKA-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13835. -- Fix Version/s: 3.2.0 Resolution: Fixed This was merged a long time ago, in Kafka 3.2.

[jira] [Resolved] (KAFKA-13245) KIP 771: Change behavior of how controller metrics are exposed when using KRaft.

[ https://issues.apache.org/jira/browse/KAFKA-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13245. -- Resolution: Won't Fix We decided to have standby KRaft controllers expose their metrics normal

[jira] [Resolved] (KAFKA-13515) Fix KRaft config validation issues

[ https://issues.apache.org/jira/browse/KAFKA-13515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13515. -- Fix Version/s: 3.2.0 Assignee: Colin McCabe Resolution: Fixed > Fix KRaft conf

[jira] [Resolved] (KAFKA-13841) Fix a case where we were unable to place on fenced brokers In KRaft mode

[ https://issues.apache.org/jira/browse/KAFKA-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13841. -- Fix Version/s: 3.3.0 Assignee: Colin McCabe Resolution: Fixed > Fix a case whe

[jira] [Resolved] (KAFKA-14327) Unify KRaft snapshot generation between broker and controller

[ https://issues.apache.org/jira/browse/KAFKA-14327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14327. -- Fix Version/s: 3.4.0 Resolution: Fixed > Unify KRaft snapshot generation between broker

[jira] [Created] (KAFKA-17987) Remove assorted ZK-related files

Colin McCabe created KAFKA-17987: Summary: Remove assorted ZK-related files Key: KAFKA-17987 URL: https://issues.apache.org/jira/browse/KAFKA-17987 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16657. -- Resolution: Won't Fix ZK mode is deprecated and will be removed in the upcoming 4.0 release. C

[jira] [Resolved] (KAFKA-14129) KRaft must check manual assignments for createTopics are contiguous

[ https://issues.apache.org/jira/browse/KAFKA-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14129. -- Fix Version/s: 3.3.0 Resolution: Fixed This was implemented in Kafka 3.3 > KRaft must

[jira] [Resolved] (KAFKA-15509) KRaft Controller doesn't consider brokers that are shutting down during topic creating

[ https://issues.apache.org/jira/browse/KAFKA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15509. -- Resolution: Duplicate > KRaft Controller doesn't consider brokers that are shutting down durin

[jira] [Resolved] (KAFKA-13142) KRaft brokers do not validate dynamic configs before forwarding them to controller

[ https://issues.apache.org/jira/browse/KAFKA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13142. -- Fix Version/s: 3.2.0 Resolution: Fixed This was fixed in Kafka 3.2 > KRaft brokers do