[PR] Add blog entry for Kafka 3.9 [kafka-site]

cmccabe opened a new pull request, #641: URL: https://github.com/apache/kafka-site/pull/641 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Hello, sorry for the late reply, took another pass after the latest updates. Most of the proposed groupings and handling seem good and very helpful indeed from the producer perspective. I just have some more comments related to the changes in the exception hierarchy, that are not scoped to the prod

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Hey Lianet, Thanks for raising these issues. I think that the parent classes (AuthorizationException and InvalidMetadataException) should extend our new groupings (InvalidConfig and RefreshRetriable). We can confirm this works as expected. I'm not as familiar with CommitFailedException, but we can

[jira] [Resolved] (KAFKA-17952) Remove Set.of usage in ReplicationControlManagerTest in 3.8

[ https://issues.apache.org/jira/browse/KAFKA-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-17952. Resolution: Fixed > Remove Set.of usage in ReplicationControlManagerTest in 3.8 >

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.8 #106

See

Re: [PR] Update site for 3.9.0 [kafka-site]

cmccabe merged PR #640: URL: https://github.com/apache/kafka-site/pull/640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.

Re: [VOTE] KIP-1091: Improved Kafka Streams operator metrics

+1 (binding) On 11/6/24 3:39 AM, Apoorv Mittal wrote: Hi Bill, Thanks for the KIP. +1 (non-binding) Regards, Apoorv Mittal On Wed, Nov 6, 2024 at 10:55 AM Lucas Brutschy wrote: Hey Bill, thanks for the KIP! +1 (binding) Cheers, Lucas On Wed, Nov 6, 2024 at 2:00 AM Bill Bejeck wrote:

Re: Configuration format for Log4j Core 2.x

Hi, I think our example log4j2 file should be as idiomatic as possible. If YAML is now the recommended format, then it makes sense to adopt it. I would also do this directly in 4.0.0. Like David, one concern when adding extra dependencies is CVEs. YAML is a widely used format and the libraries ar

Re: [DISCUSS] New label for breaking changes in 4.0

Hi Divij, Thanks for raising this! I agree we should have some ways to identify the breaking changes in v4.0.0. As I proposed in another thread, in v4.0.0, instead of tracking with time as before, we should also consider the status of some features because there are some changes only allowed in th

[jira] [Resolved] (KAFKA-17911) Unable to set kafka.metrics.reporter via environment variables in docker image

[ https://issues.apache.org/jira/browse/KAFKA-17911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-17911. Fix Version/s: 4.0.0 (was: 3.9.0) (was: 3.8.1

Re: [VOTE] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Hi all, The KIP is passed with 3 (+1) binding votes. Thanks everyone! Binding vote: Divij, Satish, and Luke. -- Kamal On Wed, Nov 6, 2024 at 11:53 AM Luke Chen wrote: > Had another read of the KIP, LGTM! > +1 from me. > Thanks for the KIP! > > Luke > > On Wed, Nov 6, 2024 at 1:58 PM Satish Du

[jira] [Resolved] (KAFKA-17363) Flaky test: kafka.api.PlaintextConsumerCommitTest.testAutoCommitOnRebalance(String, String).quorum=kraft+kip848.groupProtocol=consumer

[ https://issues.apache.org/jira/browse/KAFKA-17363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-17363. Fix Version/s: (was: 4.0.0) Resolution: Cannot Reproduce > Flaky test: > kafka

Re: [VOTE] KIP-1073: Return fenced brokers in DescribeCluster response

Hi everyone, A quick reminder that this KIP is still awaiting more votes. You can find the implementation of this KIP in this draft PR: https://github.com/apache/kafka/pull/17524. Thanks! Regards, Tina On Fri, Oct 18, 2024 at 4:37 PM Gantigmaa Selenge wrote: > Hi, > > I have raised a draft PR

[jira] [Created] (KAFKA-17949) Add GroupState and ListGroupsOptions.inStates

Andrew Schofield created KAFKA-17949: Summary: Add GroupState and ListGroupsOptions.inStates Key: KAFKA-17949 URL: https://issues.apache.org/jira/browse/KAFKA-17949 Project: Kafka Issue T

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Thanks Manikumar and Andrew. For AM6: Though I was thinking of enhancing OffsetFetcher and adding "seekToLocalBeginning" where OffsetFetcher does use Admin.listOffsets API itself. However I agree that this is just another convenient way and not necessary to support functionality. Skipping it sound

Re: [VOTE] 3.9.0 RC5

Hey All, It seems like I ran into an issue. While trying to do the migration (3.9 ZK -> 3.9 KRaft), at the very last step the whole cluster (except the controller I just restarted) fails with a fatal error where a controller (I think) requests a truncation that is below the HWM. I tried to capture

[jira] [Created] (KAFKA-17951) Rotate share partitions in share fetch

Apoorv Mittal created KAFKA-17951: - Summary: Rotate share partitions in share fetch Key: KAFKA-17951 URL: https://issues.apache.org/jira/browse/KAFKA-17951 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-17950) The leader requested truncation to below the current high watermark

[ https://issues.apache.org/jira/browse/KAFKA-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass resolved KAFKA-17950. - Resolution: Invalid Ok, my mistake. It seems like an incorrect voter list config

Re: [VOTE] 3.9.0 RC5

Ok, sorry for the noise. It seems like my setup was incorrectly configured, one controller had an incorrect voters config and that caused the issue. Retested it and it works now. But given this, I might as well just give a +1. :) On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass < viktor.somo...@

Re: [VOTE] KIP-1074: Allow the replication of user internal topics

Hey all, Just revisiting this KIP from Patrik. We currently have 2 binding and a non-binding vote, so we're really close to getting this minor change done. Anyone who would look at this and share some feedback or a vote would be welcomed. Thanks, Viktor On Thu, Oct 3, 2024 at 9:22 AM Patrik Mart

[VOTE] KIP-1081: Graduation Steps for KIPs

Hi all, The discussion about graduation steps for KIPs has now stabilized and I'd like to start a vote for it. Here you can find the KIP: https://cwiki.apache.org/confluence/x/mYn9Eg Thanks, -- [image: Aiven] *Josep Prat* Open Source Engineering Director, *Aiven* josep.p..

Re: [VOTE] KIP-1091: Improved Kafka Streams operator metrics

Hey Bill, thanks for the KIP! +1 (binding) Cheers, Lucas On Wed, Nov 6, 2024 at 2:00 AM Bill Bejeck wrote: > > Hi All, > > I'd like to call for a vote on KIP-1091 > > (discussion thread > <

[jira] [Resolved] (KAFKA-17896) Create Admin.describeClassicGroups

[ https://issues.apache.org/jira/browse/KAFKA-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-17896. -- Resolution: Fixed > Create Admin.describeClassicGroups > -

[jira] [Created] (KAFKA-17952) Remove Set.of usage in ReplicationControlManagerTest

Shivsundar R created KAFKA-17952: Summary: Remove Set.of usage in ReplicationControlManagerTest Key: KAFKA-17952 URL: https://issues.apache.org/jira/browse/KAFKA-17952 Project: Kafka Issue Ty

[jira] [Created] (KAFKA-17950) The leader requested truncation to below the current high watermark

Viktor Somogyi-Vass created KAFKA-17950: --- Summary: The leader requested truncation to below the current high watermark Key: KAFKA-17950 URL: https://issues.apache.org/jira/browse/KAFKA-17950 Pro

Re: [VOTE] KIP-1091: Improved Kafka Streams operator metrics

Hi Bill, Thanks for the KIP. +1 (non-binding) Regards, Apoorv Mittal On Wed, Nov 6, 2024 at 10:55 AM Lucas Brutschy wrote: > Hey Bill, > > thanks for the KIP! > > +1 (binding) > > Cheers, > Lucas > > On Wed, Nov 6, 2024 at 2:00 AM Bill Bejeck wrote: > > > > Hi All, > > > > I'd like to call f

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

Hi all, I started a VOTE thread for this KIP: https://lists.apache.org/thread/rblzdcwjx3fvm09ob67zzn5qyrhp6w94 Thanks! On Fri, Oct 4, 2024 at 2:35 AM Matthias J. Sax wrote: > I also never like "early access" and "preview" as I think they are too > similar. (And frankly: I personally always fel

Re: [VOTE] KIP-1102: Enable clients to rebootstrap based on timeout or error code

Hi Rajini, Thanks for the KIP. +1 (binding). Thanks, On Tue, Nov 5, 2024 at 2:55 PM Rajini Sivaram wrote: > > Hi everyone, > > I would like to start vote on KIP-1102 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1102%3A+Enable+clients+to+rebootstrap+based+on+timeout+or+error+code).

Re: [DISCUSS] KIP-1103: Additional metrics for cooperative consumption

Hi All, Please let me know if there is any other feedback. Else will start a voting thread tomorrow. Regards, Apoorv Mittal On Tue, Nov 5, 2024 at 12:21 PM Apoorv Mittal wrote: > Thanks for the feedback Andrew. > > AS9: Yes, you are correct it's topic name and not topic id. > > AS10: Thanks, I

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi Manikumar, AS7: In response to AM6, I agree that in general the combination of offsetsForTimes() and seek() will do what is required. The one case that I think is more tricky is seeking to the local earliest/beginning. The Consumer API has Consumer.beginningOffsets and seekToBeginning, which c

[jira] [Created] (KAFKA-17953) Docker image fails when setting env vars

Federico Valeri created KAFKA-17953: --- Summary: Docker image fails when setting env vars Key: KAFKA-17953 URL: https://issues.apache.org/jira/browse/KAFKA-17953 Project: Kafka Issue Type: Bu

Re: [DISCUSS] KIP-655: Add deduplication processor in kafka-streams

Hi Ayoub, I see that there was already a very long discussion thread, I won't start nitpicking names here. I think we are ready to move on to a vote. The agreed upon solutioins sound very reasonable to me, I especially like how you handled the ALOS case. Not deduplicating late records is a defensi

Re: [DISCUSS] KIP-1102: Enable clients to rebootstrap based on timeout or error code

Hi, Rajini, Thanks for the KIP. Looks good to me overall. Just a couple of minor comments. 1. "So if a connection attempt is stuck for over a second, other nodes that are in reconnection backoff state will no longer be in backoff and as a result, rebootstrap will never be attempted." Not sure tha

Jenkins build is back to stable : Kafka » Kafka PowerPC Daily » test-powerpc #110

See

Re: [DISCUSS] KIP-1105: Make remote log manager thread-pool configs dynamic

Hi all, If there are no more comments, then I'll start a voting thread as the change is minor. -- Kamal On Thu, Nov 7, 2024 at 9:00 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Federico, > > Updated the KIP by replacing the `isInitialized` to `isReady` in the KIP. > > >

[VOTE] KIP-1105: Make remote log manager thread-pool configs dynamic

Hi all, I'd like to call for a vote on KIP-1105 https://cwiki.apache.org/confluence/display/KAFKA/KIP-1105%3A+Make+remote+log+manager+thread-pool+configs+dynamic Thanks, Kamal

Re: [VOTE] KIP-1091: Improved Kafka Streams operator metrics

+1 (binding) thanks Bill On Wed, Nov 6, 2024 at 4:44 PM Matthias J. Sax wrote: > +1 (binding) > > On 11/6/24 3:39 AM, Apoorv Mittal wrote: > > Hi Bill, > > Thanks for the KIP. > > > > +1 (non-binding) > > > > Regards, > > Apoorv Mittal > > > > > > On Wed, Nov 6, 2024 at 10:55 AM Lucas Brutschy

Re: [VOTE] KIP-1105: Make remote log manager thread-pool configs dynamic

+1 On Thu, Nov 7, 2024 at 12:39 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi all, > > I'd like to call for a vote on KIP-1105 > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1105%3A+Make+remote+log+manager+thread-pool+configs+dynamic > > Thanks, > Kamal >

Re: [DISCUSS] Require KIPs to include "How to teach this section"

It seems like the "Create KIP" button on the main KIP page actually does not point to this template? I was actually surprised to hear that we had a "Documentation Plan" section already because I have literally never seen it. I took a quick look at this button but as far as I can tell it's pointing

[VOTE] KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins

Hi all, I'd like to call for a vote on KIP-1104 https://cwiki.apache.org/confluence/display/KAFKA/KIP-1104%3A+Allow+Foreign+Key+Extraction+from+Both+Key+and+Value+in+KTable+Joins Thanks, Peter

Re: [DISCUSS] Apache Kafka 4.0.0 release

Thanks David for being the release manager for 4.0! I noticed the release plan does not yet have KIP-996 (Pre-Vote) listed, can I edit the page to add it? Best, Alyssa On Sun, Oct 20, 2024 at 11:36 AM Colin McCabe wrote: > H

[PR] Update site for 3.9.0 [kafka-site]

cmccabe opened a new pull request, #640: URL: https://github.com/apache/kafka-site/pull/640 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[jira] [Resolved] (KAFKA-17635) Lost events on internal repartition topic when excatly_once_v2 is set and producer is fenced

[ https://issues.apache.org/jira/browse/KAFKA-17635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-17635. - Resolution: Fixed > Lost events on internal repartition topic when excatly_once_v2 is set and >

[jira] [Created] (KAFKA-17955) Add 3.9 to the releases tested in kafkatest/version.py and related

Colin McCabe created KAFKA-17955: Summary: Add 3.9 to the releases tested in kafkatest/version.py and related Key: KAFKA-17955 URL: https://issues.apache.org/jira/browse/KAFKA-17955 Project: Kafka

Re: [VOTE] 3.9.0 RC5

Hi Colin, I see that you made 3.9 the default docs version in Kafka-Site. However, as your doc changes were pushed before the 3.8.1 release, now the upgrade.html page doesn't mention 3.8.1 at all. I'll update the site. On Wed, Nov 6, 2024 at 4:48 PM Viktor Somogyi-Vass wrote: > Ok, sorry for the

Re: [DISCUSS] KIP-1105: Make remote log manager thread-pool configs dynamic

Hi Federico, Updated the KIP by replacing the `isInitialized` to `isReady` in the KIP. On Wed, Nov 6, 2024 at 12:47 PM Federico Valeri wrote: > Thanks Kamal, LGTM, but you should replace all instances of > isInitialized to isReady in the rest of the KIP. > > On Wed, Nov 6, 2024 at 5:22 AM Kam

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi Luke, Please add the KIP-1058 and KIP-1105 to the release plan. KIP-1058 is accepted and PRs are ready. KIP-1105 is a minor one, and will open a voting thread soon. Thanks, Kamal On Thu, Nov 7, 2024 at 7:28 AM Luke Chen wrote: > Hi Alyssa, > > I've added KIP-966 into the release plan. > > T

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi Kamal, Just added them in to the release plan. Thanks. Luke On Thu, Nov 7, 2024 at 11:11 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Luke, > > Please add the KIP-1058 and KIP-1105 to the release plan. > KIP-1058 is accepted and PRs are ready. > KIP-1105 is a minor o

Re: [DISCUSS] KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins

Thank you, Lucas and Matthias, for your feedback and thoughts. I'll start the voting session for this KIP shortly. Best regards, Peter On Wed, Nov 6, 2024 at 6:35 PM Lucas Brutschy wrote: > Hey, > > this makes sense to me! Thank you for the KIP. > > I think the change is straightforward, and I

Re: [PR] MINOR: Improve doc for controller membership change [kafka-site]

showuon commented on code in PR #642: URL: https://github.com/apache/kafka-site/pull/642#discussion_r1832138254 ## 39/upgrade.html: ## @@ -41,6 +41,81 @@ Notable changes in 3 See https://cwiki.apache.org/confluence/display/KAFKA/KIP-956+Tiered+Storage+Quotas

[PR] MINOR: Improve doc for controller membership change [kafka-site]

showuon opened a new pull request, #642: URL: https://github.com/apache/kafka-site/pull/642 Porting the change in Kafka repo: https://github.com/apache/kafka/pull/17598 to Kafka-site repo for v3.9.0 doc. 1. Added a notable change note for KIP-853 to help users to find the document.

[jira] [Resolved] (KAFKA-16331) Remove Deprecated EOSv1

[ https://issues.apache.org/jira/browse/KAFKA-16331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16331. - Resolution: Fixed > Remove Deprecated EOSv1 > --- > >

Re: [DISCUSS] KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins

Hey, this makes sense to me! Thank you for the KIP. I think the change is straightforward, and I think we can just vote this through. The only thing we could discuss is whether we want to keep the old overloads. I thought about it, but I think I agree that it's nicer to simplify the API a bit eve

[jira] [Created] (KAFKA-17954) Error getting oldest-iterator-open-since-ms from JMX

Nicholas Telford created KAFKA-17954: Summary: Error getting oldest-iterator-open-since-ms from JMX Key: KAFKA-17954 URL: https://issues.apache.org/jira/browse/KAFKA-17954 Project: Kafka

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #98

See

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi Alyssa, I've added KIP-966 into the release plan. Thanks. Luke On Thu, Nov 7, 2024 at 3:54 AM Alyssa Huang wrote: > Thanks David for being the release manager for 4.0! > I noticed the release plan > > does > not yet have