Re: [VOTE] 3.9.0 RC3

Hi Colin, Thanks for the explanation. And I agree with Mickael that if there are such limitations, we should clearly document it. So far, in the v3.9 document , I can't see anything about it. Also, it seems metrics are still not added, we should docu

[jira] [Created] (KAFKA-17870) Disallow creating too many partitions before using ReplicaPlacer

Colin McCabe created KAFKA-17870: Summary: Disallow creating too many partitions before using ReplicaPlacer Key: KAFKA-17870 URL: https://issues.apache.org/jira/browse/KAFKA-17870 Project: Kafka

[jira] [Created] (KAFKA-17869) Tests for clients ensuring KIP-1076 doesn't affect existing metrics

Bill Bejeck created KAFKA-17869: --- Summary: Tests for clients ensuring KIP-1076 doesn't affect existing metrics Key: KAFKA-17869 URL: https://issues.apache.org/jira/browse/KAFKA-17869 Project: Kafka

Re: [VOTE] 3.9.0 RC3

On Wed, Oct 23, 2024, at 23:22, Justine Olshan wrote: > Hey Colin, > > Thanks for running the release. > > I checked the keys and scanned the docs > I built from source, used kraft quickstart, ran a transactions workload, > played around with a few other things including running some of the feature

Re: [VOTE] 3.9.0 RC2

Hi Anton, The implicit defaults have been documented for a long time. Take a look at this documentation from 2019 in clients/src/main/resources/common/message/README.md : > Deserializing Messages > -- > Message objects may be deserialized using the Message#read method. This

Re: doc clarification about meesage format

Hey Xiang, Thanks for your questions! This is getting to the limit of my knowledge, but I'll answer as best I can. The partitionLeaderEpoch is only set once during the batch lifetime (during Produce), and is not mutated any other time. This includes when data is fetched by other replicas and by c

Re: [VOTE] 3.9.0 RC3

Hi Luke, Unfortunately, we don't have a way to upgrade from static quorums to dynamic quorums yet. We are hoping to include this in the next release. We're trakcing it under https://issues.apache.org/jira/browse/KAFKA-16538 . best, Colin On Thu, Oct 24, 2024, at 05:25, Luke Chen wrote: > Hi C

Re: [VOTE] 3.9.0 RC3

Hi Colin, I found I cannot upgrade to v3.9.0 and then successfully add a new controller or remove a controller. I've opened KAFKA-17866 . Please take a look. Thanks. Luke On Thu, Oct 24, 2024 at 2:27 PM TengYao Chi wrote: > Hi Colin, > > Thank

[jira] [Resolved] (KAFKA-8779) Fix flaky tests introduced by dynamic log levels

[ https://issues.apache.org/jira/browse/KAFKA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-8779. --- Fix Version/s: 4.0.0 Resolution: Fixed > Fix flaky tests introduced by dynamic log lev

[jira] [Resolved] (KAFKA-16728) Add dynamic group configuration for heartbeat interval and session timeout

[ https://issues.apache.org/jira/browse/KAFKA-16728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-16728. -- Resolution: Duplicate > Add dynamic group configuration for heartbeat interval and ses

[jira] [Resolved] (KAFKA-17846) ClientTelemetryReporter does not log trace-level message

[ https://issues.apache.org/jira/browse/KAFKA-17846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17846. Fix Version/s: 4.0.0 Resolution: Fixed > ClientTelemetryReporter does not log trace

[jira] [Created] (KAFKA-17868) Do not ignore --feature flag in kafka-storage.sh

Colin McCabe created KAFKA-17868: Summary: Do not ignore --feature flag in kafka-storage.sh Key: KAFKA-17868 URL: https://issues.apache.org/jira/browse/KAFKA-17868 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-17866) kraft.version/dirID is not updated after upgrading to MV 3.9

[ https://issues.apache.org/jira/browse/KAFKA-17866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-17866. -- Resolution: Duplicate > kraft.version/dirID is not updated after upgrading to MV 3.9 > ---

[jira] [Resolved] (KAFKA-17574) KafkaClusterTestKit produces error logs if shut down from a shutdown hook

[ https://issues.apache.org/jira/browse/KAFKA-17574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17574. Fix Version/s: 4.0.0 Assignee: Stig Rohde Døssing Resolution: Fixed > Kafk

Re: [VOTE] 3.9.0 RC3

Hi Justine, I looked at this again, and it seems like we somehow dropped the code to handle --feature. I filed a blocker JIRA to fix: Do not ignore --feature flag in kafka-storage.sh I will also add some more junit tests in this area. Sadly, I guess that means we'll need an RC4. thanks, Colin

[jira] [Created] (KAFKA-17866) kraft.version is not updated after upgrading to MV 3.9

Luke Chen created KAFKA-17866: - Summary: kraft.version is not updated after upgrading to MV 3.9 Key: KAFKA-17866 URL: https://issues.apache.org/jira/browse/KAFKA-17866 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Thanks Lianet for valuable comments 107) & 108) We have updated group exception names, which will prevent error in current applications which are handing producer/consumer exceptions ``` ProducerRetriableTransactionException -> RetriableException ProducerRefreshRetriableTransactionException -> Re

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Thanks Artem for valuable comments I have incorporated them into the updated KIP. On Sat, Oct 19, 2024 at 3:31 AM Artem Livshits wrote: > Hi Kaushik, > > Thank you for the KIP! I think it'll make writing transactional > application easier and less error prone. I have a couple comments: > > A

Re: [DISCUSS] KIP-1098: Reverse Checkpointing in MirrorMaker

Hi all, Just a bump/minor update here: As I've started working on a POC of the proposed solution, I've realised that the hard requirement related to the ReplicationPolicy implementation can be eliminated, updated the KIP accordingly. Daniel Dániel Urbán ezt írta (időpont: 2024. okt. 21., H, 16:18

Re: doc clarification about meesage format

Thanks Greg! It helps so much. This KIP seems familiar somehow, I should go through it again. Really appreciated! Replied Message From Greg Harris Date 10/25/2024 00:40 To dev@kafka.apache.org Cc us...@kafka.apache.org Subject Re: doc clarification about meesage format Hey Xiang, Tha

[jira] [Created] (KAFKA-17871) Source task source offset reads can block herder task cancellation

Greg Harris created KAFKA-17871: --- Summary: Source task source offset reads can block herder task cancellation Key: KAFKA-17871 URL: https://issues.apache.org/jira/browse/KAFKA-17871 Project: Kafka

Re: Cluster ID starting with hyphen, docs not exactly in sync

Hi! Thank you! The problematic command is possibly in use in bitnami kafka ( I do not know )? The problem came when we used it & the problem was analyzed only with it. Bitnami kafka is not used anymore, but Apache Kafka is used directly instead, nowadays. So, the argument can be given in a better

[jira] [Created] (KAFKA-17872) Kafka stream job does not commit offset when packets are getting ignored intentionally

Manish Kumar Sinha created KAFKA-17872: -- Summary: Kafka stream job does not commit offset when packets are getting ignored intentionally Key: KAFKA-17872 URL: https://issues.apache.org/jira/browse/KAFKA-1787

[jira] [Resolved] (KAFKA-17854) Improve tests for ReadOnlyWindowStoreStub#fetch and #backwardFetch

[ https://issues.apache.org/jira/browse/KAFKA-17854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17854. Fix Version/s: 4.0.0 Resolution: Fixed > Improve tests for ReadOnlyWindowStoreStub#

Re: [VOTE] 3.9.0 RC3

Hi Colin, This is a major caveat. We've advertised KRaft as production ready and anyone who has already migrated is now stuck with a static quorum. The whole point of 3.9.0 was to reach "feature parity" with ZooKeeper, this is now true for only a subset of users (users that have not already migrat

Build failed in Jenkins: Kafka » kafka-2.7-jdk8 #196

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on builds32 (ubuntu) in workspace

SLF4J error when running Kafka in IDEA

Hi community, I am trying to run/debug Kafka in IDEA, it works fine except the following SLF4J errors. I kind of know how to fix this, but it seems messy to me that a workaround is required when the code is just downloaded and built. I am just wondering whether this is due to some misconfigured se

Re: [VOTE] 3.9.0 RC3

Thanks Colin. Let me know if you need any help with reviews here. Justine On Thu, Oct 24, 2024 at 10:45 AM Colin McCabe wrote: > Hi Justine, > > I looked at this again, and it seems like we somehow dropped the code to > handle --feature. I filed a blocker JIRA to fix: Do not ignore --feature >

Running Kafka from IDE

Hello everyone, Is there a way I could run Kafka from within IntelliJ, in debug mode, and let producers and consumers connect to it? (I want to run producers and consumers as separate programs on my computer and let them connect to this Kafka which is running under debugger, so that I can put bre

[jira] [Created] (KAFKA-17864) Ensure all fields in the protocol have a description

Mickael Maison created KAFKA-17864: -- Summary: Ensure all fields in the protocol have a description Key: KAFKA-17864 URL: https://issues.apache.org/jira/browse/KAFKA-17864 Project: Kafka Issu

Re: Running Kafka from IDE

Hi, You can also import the Kafka project as a Gradle project in IntelliJ, and then create an Application Run Configuration to run/debug it. In your run configuration, set the module classpath to kafka.core.main, the main class to kafka.Kafka, and the program arguments to one of the server.propert

Re: Running Kafka from IDE

Hi Pradyumna, It's possible. Here are some points: 1. kafka.Kafka is the class you want to run. 2. As the parameter, pass the path to your server.properties, i.e. the config file. 3. Point to the logger config as the VM properties: -Dlog4j.configuration=file:/path/to/log4j.properties. 4. You nee

[jira] [Created] (KAFKA-17865) Protocol documentation field names are sometimes ambiguous

Andrew Schofield created KAFKA-17865: Summary: Protocol documentation field names are sometimes ambiguous Key: KAFKA-17865 URL: https://issues.apache.org/jira/browse/KAFKA-17865 Project: Kafka

[jira] [Resolved] (KAFKA-17860) Remove log4j-appender module

[ https://issues.apache.org/jira/browse/KAFKA-17860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17860. Fix Version/s: 4.0.0 Resolution: Fixed > Remove log4j-appender module > ---

Re: Running Kafka from IDE

I've found one approach that works. Posting it here in case someone else needs it. If there is a simpler method, do let me know. Use "./gradlew clean releaseTarGz -x test" to build a tarball with my changes Take the generated tarball in core/build/distributions, expand it and run it. Set KAFKA_

[jira] [Created] (KAFKA-17863) Align share consumer poll result with record batch boundaries

Andrew Schofield created KAFKA-17863: Summary: Align share consumer poll result with record batch boundaries Key: KAFKA-17863 URL: https://issues.apache.org/jira/browse/KAFKA-17863 Project: Kafka