Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #9

See

Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing

Hi, Last PR that completed KIP-1033 has been merged. Thanks to all participants, for the guidance and the feedbacks! Just letting you know about some decisions and changes made through the different PRs that were not initially mentioned in the KIP: * The sourceRawKey and the sourceRawValue

Re: [DISCUSS] KIP-877: Mechanism for plugins and connectors to register metrics

Hi Tom, Thanks for taking a look at the KIP! 1. I've clarified that point. The tag names automatically added are now mentioned in the Javadoc. In case user-provided tags collide, the metricName() method will throw IllegalArgumentException 2. That's a nice suggestion! I've made that change. 3. Y

[jira] [Created] (KAFKA-17237) [rack-aware assignors] Rebalance is triggered every time a broker isn't reported from a metadata call

Emanuele Sabellico created KAFKA-17237: -- Summary: [rack-aware assignors] Rebalance is triggered every time a broker isn't reported from a metadata call Key: KAFKA-17237 URL: https://issues.apache.org/jira/bro

Re: [DISCUSS] KIP-1062: Introduce Pagination for some requests used by Admin API

Great, I’ll ask again for a vote then on this KIP as it seems like all the concerns have been addressed. > On 30 Jul 2024, at 16:00, David Arthur wrote: > > Omnia, thanks for the updates! > >> Am happy to add section for throttling in this KIP if it is high concern > or open a followup KIP fo

Re: [VOTE] KIP-1062: Introduce Pagination for some requests used by Admin API

Hi everyone it seems like all the concerns have been addressed in the discussion thread. So I would like to pump this back for a vote. Thanks Omnia > On 11 Jul 2024, at 10:41, Omnia Ibrahim wrote: > > Hi, > I would like to open vote on KIP-1062 > https://cwiki.apache.org/confluence/display/K

Re: [PR] Update ZK migration section to indicate GA [kafka-site]

mumrah merged PR #545: URL: https://github.com/apache/kafka-site/pull/545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.o

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Hi Bruno, Thanks for adding the detail on the schemas on records written to __consumer_offsets. I’ve reviewed them in detail and they look good to me. I have one naive question. AS11: I notice that an assignment is essentially a set of partition indices for subtopologies. Since a subtopology can

[jira] [Created] (KAFKA-17238) Move VoterSet from raft.internals to raft

José Armando García Sancio created KAFKA-17238: -- Summary: Move VoterSet from raft.internals to raft Key: KAFKA-17238 URL: https://issues.apache.org/jira/browse/KAFKA-17238 Project: Kafka

Re: New release branch 3.9

Hi, Considering KIP-853 does not look like it's near completion (20 out of 49 tasks done in https://issues.apache.org/jira/browse/KAFKA-14094), I'm not sure I understand why we decided to cut the 3.9 branch this early. By waiting a bit longer we could have reduced the risks in case KIP-853 is furt

[jira] [Created] (KAFKA-17239) Kafka admin client doesn't report node request-latency metrics

Chia-Ping Tsai created KAFKA-17239: -- Summary: Kafka admin client doesn't report node request-latency metrics Key: KAFKA-17239 URL: https://issues.apache.org/jira/browse/KAFKA-17239 Project: Kafka

[jira] [Created] (KAFKA-17240) Try to complete delyed ops for all purgatories even though one of delayed op throws exception

Chia-Ping Tsai created KAFKA-17240: -- Summary: Try to complete delyed ops for all purgatories even though one of delayed op throws exception Key: KAFKA-17240 URL: https://issues.apache.org/jira/browse/KAFKA-17240

[jira] [Created] (KAFKA-17241) KIP-853 follow-ups

Colin McCabe created KAFKA-17241: Summary: KIP-853 follow-ups Key: KAFKA-17241 URL: https://issues.apache.org/jira/browse/KAFKA-17241 Project: Kafka Issue Type: Improvement Report

[jira] [Created] (KAFKA-17242) MirrorCheckpointTask prints spurious timeout for starting checkpoint and offset sync stores

Greg Harris created KAFKA-17242: --- Summary: MirrorCheckpointTask prints spurious timeout for starting checkpoint and offset sync stores Key: KAFKA-17242 URL: https://issues.apache.org/jira/browse/KAFKA-17242

Re: New release branch 3.9

Hi Mickael, KAFKA-14094 is basically a long list of things that we might want to do at some time in the future, that are related to KIP-853. They are certainly not all required for Kafka 3.9. Sorry that this was confusing. In order to avoid confusion about this in the future, I'll move the stu

Re: New release branch 3.9

Hi Chris, I don't see why this shouldn't go to 3.9. Would you call this a bugfix? best, Colin On Wed, Jul 31, 2024, at 13:54, Chris Egerton wrote: > Hi Colin, > > Would it be alright if I cherry-picked > https://github.com/apache/kafka/pull/16678 onto 3.9? This isn't a > regression but it's a l

Re: New release branch 3.9

Hi Colin, Thanks for the clarification. This makes much more sense now. Mickael On Thu, Aug 1, 2024 at 8:43 PM Colin McCabe wrote: > > Hi Chris, > > I don't see why this shouldn't go to 3.9. Would you call this a bugfix? > > best, > Colin > > > On Wed, Jul 31, 2024, at 13:54, Chris Egerton wrot

[jira] [Resolved] (KAFKA-17230) Kafka consumer client doesn't report node request-latency metrics

[ https://issues.apache.org/jira/browse/KAFKA-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-17230. - Fix Version/s: 3.9.0 Resolution: Fixed merged the PR to trunk > Kafka consumer client doesn't re

Re: New release branch 3.9

Hi Colin, Will JIRAs like adding metrics ( https://issues.apache.org/jira/browse/KAFKA-16524) make it in patch version of 3.9? Best, On Thu, Aug 1, 2024 at 8:49 PM Mickael Maison wrote: > Hi Colin, > > Thanks for the clarification. This makes much more sense now. > > Mickael > > On Thu, Aug 1,

Re: New release branch 3.9

Hi Josep, I don't have a strong opinion on whether KAFKA-16524 should land in 3.9.1. Let's discuss that more on the mailing list when it's time for 3.9.1. best, Colin On Thu, Aug 1, 2024, at 12:07, Josep Prat wrote: > Hi Colin, > Will JIRAs like adding metrics ( > https://issues.apache.org/jir

[jira] [Reopened] (KAFKA-17156) Metadata controller to provide the KafkaRaftClient with the broker's supported kraft.version

[ https://issues.apache.org/jira/browse/KAFKA-17156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio reopened KAFKA-17156: > Metadata controller to provide the KafkaRaftClient with the broker's

Re: New release branch 3.9

Hi all, I've moved the optional and follow-on JIRAs for KIP-853 (many of which were "refactors," "optimizations", "tests", etc.) to KAFKA-17241 to hopefully clear up the confusion a bit. I think a lot of these would be good newbie JIRAs (although probably not the "refactor" ones :) As I said,

Re: New release branch 3.9

Hi Colin, Looking at the JIRAs of the new umbrella issue ( https://issues.apache.org/jira/browse/KAFKA-17241) it seems to me that the adding metrics one is the only one that is different. As you mention most of the tasks are refactors, optimizations and tests. However, having proper metrics I thin

Re: [DISCUSS] Graduation steps for Features

Hi all, Sorry, I mis-spoke earlier. For KRaft, we did EA -> Preview -> GA. For some reason I remembered the order differently, but I was wrong. EA was 2.8; preview was 3.0, GA was 3.3. The main thing I remember was that we broke compatibility between KRaft 2.8 and everything else. That release

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hmm, I thought I had added that already. I guess I missed it. Sorry for the confusion, and thanks for the update. best, Colin On Tue, Jul 30, 2024, at 15:06, Jun Rao wrote: > Thanks for updating the KIP, Justine. > > Jun > > On Tue, Jul 30, 2024 at 1:37 PM Justine Olshan > wrote: > >> I added

[jira] [Resolved] (KAFKA-16521) kafka-metadata-quorum describe changes for KIP-853

[ https://issues.apache.org/jira/browse/KAFKA-16521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio resolved KAFKA-16521. Resolution: Fixed > kafka-metadata-quorum describe changes for KIP

Re: New release branch 3.9

Surely "Document how to use KIP-853" ( https://issues.apache.org/jira/browse/KAFKA-17048) should be required for the release, or? Why bother with the KIP and the release if you don't tell the users how to use it? Jakub On Thu, Aug 1, 2024 at 8:40 PM Colin McCabe wrote: > Hi Mickael, > > KAFKA-1

Re: New release branch 3.9

Hi Josep, I don't think that all of the metrics are required to operate a kraft cluster. The KIP documents at different points which metrics should be monitored by the operator. I'll make that clear in the jira and we can implement those in 3.9.1. Thanks On Thu, Aug 1, 2024 at 3:29 PM Josep Pra

Re: New release branch 3.9

Jakub, Yes but that can be done after code freeze. It needs to get done before RC and the announcement email. Thanks, On Thu, Aug 1, 2024 at 3:43 PM Jakub Scholz wrote: > Surely "Document how to use KIP-853" ( > https://issues.apache.org/jira/browse/KAFKA-17048) should be required for > the re

Re: New release branch 3.9

On Wed, Jul 31, 2024 at 12:28 AM Ismael Juma wrote: > I would recommend against large refactorings in trunk until the first RC > for 3.9 - that will reduce cherry-pick friction. Once we have the first RC, > subsequent changes to 3.9 should be limited in scope. > I agree. We should specially dela

[jira] [Created] (KAFKA-17243) MetadataQuorumCommand describe to include CommittedVoters

Alyssa Huang created KAFKA-17243: Summary: MetadataQuorumCommand describe to include CommittedVoters Key: KAFKA-17243 URL: https://issues.apache.org/jira/browse/KAFKA-17243 Project: Kafka Iss

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

+1 (binding). Thanks Mickael! On Thu, 1 Aug 2024 at 05:12, Mickael Maison wrote: > Hi, > > Bumping this thread to get some more votes and/or feedback. > > As I restarted the vote on June 10 after major changes, I'm only > counting votes since then. > So we have 1 binding (Chris) and 1 non-bindi

Re: [DISCUSS] Graduation steps for Features

Interesting point. To me, EA could be the same as "experimental". I don't think we should introduce too many categories. From a naming point of view, the difference between EA and Preview is not super intuitive though. If we would go with "experimental" instead of EA it might be easier to unde

Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing

Thanks Loïc! On 8/1/24 6:19 AM, Loic Greffier wrote: Hi, Last PR that completed KIP-1033 has been merged. Thanks to all participants, for the guidance and the feedbacks! Just letting you know about some decisions and changes made through the different PRs that were not initially mentioned in

Re: [DISCUSS] Graduation steps for Features

> At the same time, even if something is in Preview and we would guarantee backward compatibility, a major release would still allow us to break it? I think so. The way I see it is the following. We have 3 levels of graduation: 1, 2 and 3 (naming is hard and don't want to bike shed on this). Level

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.7 #188

See Changes: -- [...truncated 462844 lines...] [2024-08-02T01:57:33.361Z] [2024-08-02T01:57:33.361Z] > Task :core:test [2024-08-02T01:57:33.361Z] [2024-08-02T01:57:33.361Z]

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #78

See

[jira] [Created] (KAFKA-17244) java.base/java.lang.VirtualThread$VThreadContinuation.onPinned

Jianbin Chen created KAFKA-17244: Summary: java.base/java.lang.VirtualThread$VThreadContinuation.onPinned Key: KAFKA-17244 URL: https://issues.apache.org/jira/browse/KAFKA-17244 Project: Kafka

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3161

See

[jira] [Resolved] (KAFKA-16666) Migrate `TransactionLogMessageFormatter`, GroupMetadataMessageFormatter` and `OffsetsMessageFormatter`to tools module

[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-1. Resolution: Fixed > Migrate `TransactionLogMessageFormatter`, GroupMetadataMessageFormatte

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Hi Divij, Thanks for the review! And, sorry for the late reply. >From the UnifiedLog.scala doc: """ The last stable offset (LSO) is defined as the first offset such that all low