[jira] [Resolved] (KAFKA-17463) Flaky test: kafka.api.PlaintextAdminIntegrationTest."testShareGroups(String).quorum=kraft+kip932"

[ https://issues.apache.org/jira/browse/KAFKA-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17463. Fix Version/s: 4.0.0 Assignee: Apoorv Mittal Resolution: Fixed > Flaky tes

Re: Seeking Beginner-Friendly Contribution Opportunities in Apache Kafka

Hello Grace, thanks for your interest in Apache Kafka! Other than the beginner's tickets marked as newbie, I suggest to also check tickets that may not have that label but are considered Minor/Trivial ( https://issues.apache.org/jira/browse/KAFKA-17936?jql=project%20%3D%20KAFKA%20AND%20status%20in

Re: [DISCUSS] KIP-1102: Enable clients to rebootstrap based on timeout or error code

Hi Apoorv, Thanks for the review. Good idea to prefix with metadata.recovery, I have renamed the config to metadata.recovery.rebootstrap.trigger.ms as you suggested. Regards, Rajini On Mon, Nov 4, 2024 at 2:43 PM Apoorv Mittal wrote: > Hi Rajni, > Thanks for the KIP. It would be a good addit

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi Manikumar, Thanks for the KIP, this new strategy would be helpful in specifying fetch behaviour. AM1: The config `auto.offset.reset` is currently applied as per the enum class OffsetResetStarategy which is part of kafka-clients javadoc

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi Manikumar, Thanks for the KIP! I have a few questions: 1. Why did you choose the "minus-" prefix for the new value format? Can't we just drop the prefix? If we have to keep the prefix, I'd suggest "last-N-Duration", as it somewhat "rhymes" with latest-earliest. 2. Should minutes and seconds be

[jira] [Resolved] (KAFKA-17906) remove redundant sourceSets from build.gradle

[ https://issues.apache.org/jira/browse/KAFKA-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17906. Fix Version/s: 4.0.0 Resolution: Fixed > remove redundant sourceSets from build.gra

Re: Advice on how to comprehend Kafka source code

If you want to understand the architecture, I think you probably want to start with reading a book like "Kafka: The Definitive Guide." If you understand the architecture, you can learn the source code with time. I would suggest always reading the newest source code, since that is what people wi

[jira] [Resolved] (KAFKA-15321) Document consumer group member state machine

[ https://issues.apache.org/jira/browse/KAFKA-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-15321. Resolution: Fixed > Document consumer group member state machine > ---

Re: Advice on how to comprehend Kafka source code

hi Xiang > 1. Which version(s) is the most proper version to go to if I want to get > the core design and implementation of Kafka so that it won’t be too old and > not too new either. The code in the trunk branch is the best choice if you want to understand Kafka's most up-to-date architecture :

Re: [DISCUSS] KIP-1102: Enable clients to rebootstrap based on timeout or error code

Hi Rajni, Thanks for the KIP. It would be a good addition. If I understand it correctly then the client will trigger the re-bootstrap process when the client is unable to obtain metadata in a defined period of time. The proposed configuration "rebootstrap.timeout.ms" seems to me more of a timeout

Re: [DISCUSS] KIP-1098: Reverse Checkpointing in MirrorMaker

Hi Vidor, Thank you for your comments! 1. I think the optimization sounds nice, but would not work well with TopicFilter implementations which can be dynamically updated in the background (without touching the Connector configuration). If we actually dropped the offset sync records for a topic wh

[jira] [Created] (KAFKA-17939) Kafka Streams API exposes org.apache.kafka.common.utils.Bytes which is not part of the public API

Mickael Maison created KAFKA-17939: -- Summary: Kafka Streams API exposes org.apache.kafka.common.utils.Bytes which is not part of the public API Key: KAFKA-17939 URL: https://issues.apache.org/jira/browse/KAFKA-17

Re: [DISCUSS] Require KIPs to include "How to teach this section"

Hi Matthias, Thanks for pointing this out! Maybe what we need is to start enforcing that this section appears (even if it just says "no change"). I think we have people copying previous KIPs (including me...) and in that case just we won't have the section, unless someone complains. best, Coli

Re: [DISCUSS] KIP-1105: Make remote log manager thread-pool configs dynamic

Hi Kamal, these changes make sense to me. Thanks. In this case, I wonder if "isReady" could be a better name, instead of "isInitialized". Wdyt? Could you please add an example of the stack trace that the RLMM can raise during the initialization phase? On Sun, Nov 3, 2024 at 4:50 PM Kamal Chandra

[jira] [Created] (KAFKA-17934) Add duration based offset reset option for consumer clients

Manikumar created KAFKA-17934: - Summary: Add duration based offset reset option for consumer clients Key: KAFKA-17934 URL: https://issues.apache.org/jira/browse/KAFKA-17934 Project: Kafka Issue

Re: Wiki Kafka Kip access

Hi TaiJuWu, Thanks for your interest in Apache Kafka. Your accounts are now all set. Please let me know if something doesn't work for you. Best, On Mon, Nov 4, 2024 at 8:53 AM 吳岱儒 wrote: > Hi folk, > > I would like to have the right access permission to create Kip. > My wiki id and jira id are

[jira] [Created] (KAFKA-17933) Add ShareRoundTripWorker for ShareConsumers.

Shivsundar R created KAFKA-17933: Summary: Add ShareRoundTripWorker for ShareConsumers. Key: KAFKA-17933 URL: https://issues.apache.org/jira/browse/KAFKA-17933 Project: Kafka Issue Type: Sub-

[DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi everyone, I would like to start a discussion on KIP-1106: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1106%3A+Add+duration+based+offset+reset+option+for+consumer+clients This KIP proposes to add an additional auto offset reset strategy for consumer clients. Regards, Manikumar

[jira] [Resolved] (KAFKA-17938) Minor : Remove Set.of usage in 3.9 branch.

[ https://issues.apache.org/jira/browse/KAFKA-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-17938. Resolution: Fixed > Minor : Remove Set.of usage in 3.9 branch. > -

Jenkins build is still unstable: Kafka » Kafka PowerPC Daily » test-powerpc #108

See

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Thanks for the KIP. A somewhat orthogonal question I have is, if we should try to merge this KIP with the existing KIP-842: https://cwiki.apache.org/confluence/display/KAFKA/KIP-842%3A+Add+richer+group+offset+reset+mechanisms While KIP-842 has a different intention, ie, separating the case of

Re: [DISCUSS] KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins

Seems you did link to the wrong KIP. Should be https://cwiki.apache.org/confluence/display/KAFKA/KIP-1104%3A+Allow+Foreign+Key+Extraction+from+Both+Key+and+Value+in+KTable+Joins :) Overall the KIP does make sense and I don't have any comments. Seems to be quite clear. -Matthias On 10/30/24

Re: [DISCUSS] Require KIPs to include "How to teach this section"

Hi all, FYI the Documentation Plan was added to the template in September 2023 around the time ~979 was opened. I was not able to find any contemporary discussion thread, does anyone know of where/when we discussed this change? Not to say that I think the change to the template should be undone, a

Re: [DISCUSS] KIP-1091: Improved Kafka Streams operator metrics

Thanks for the comments, Matthias! I've updated the table to contain the new JMX metric and the correct MBean name. Regards, Bill On Fri, Nov 1, 2024 at 10:27 PM Matthias J. Sax wrote: > Thanks for the KIP Bill. Very useful addition to be able to collect all > metrics broker side. > > The KIP

[jira] [Created] (KAFKA-17941) TransactionStateManager.loadTransactionMetadata method may get stuck in an infinite loop

Vincent Jiang created KAFKA-17941: - Summary: TransactionStateManager.loadTransactionMetadata method may get stuck in an infinite loop Key: KAFKA-17941 URL: https://issues.apache.org/jira/browse/KAFKA-17941

[jira] [Resolved] (KAFKA-17857) Move AbstractResetIntegrationTest and subclasses to tools

[ https://issues.apache.org/jira/browse/KAFKA-17857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17857. Fix Version/s: 4.0.0 Resolution: Fixed > Move AbstractResetIntegrationTest and subc

[DISCUSS]KIP-1107: Adding record-level acks for producers

Hi all, I open a KIP-1107: Adding record-level acks for producers to reduce the limitation associated with reusing KafkaProducer. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1107%3A++

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.9 #97

See

[jira] [Resolved] (KAFKA-17920) GroupType.parse should return Optional to represent the null or empty

[ https://issues.apache.org/jira/browse/KAFKA-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17920. Resolution: Won't Fix > GroupType.parse should return Optional to represent the null or >

[jira] [Resolved] (KAFKA-16775) Fix flaky PlaintextAdminIntegrationTest#testCreateExistingTopicsThrowTopicExistsException

[ https://issues.apache.org/jira/browse/KAFKA-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16775. Resolution: Won't Fix [~perli] thanks for this finding. close it right now > Fix flaky >

Re: [DISCUSS] Require KIPs to include "How to teach this section"

Guess it would be better to use the template when starting a new KIP -- that's why we have it :) On 11/4/24 10:23 AM, Colin McCabe wrote: Hi Matthias, Thanks for pointing this out! Maybe what we need is to start enforcing that this section appears (even if it just says "no change"). I think w

[jira] [Resolved] (KAFKA-16964) Integration tests for adding and removing voters

[ https://issues.apache.org/jira/browse/KAFKA-16964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16964. Fix Version/s: 4.0.0 Resolution: Fixed > Integration tests for adding and removing

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Sorry for the churn on this one. But circling back to the API. Today, the API forces the plugins to have an optimization which can provide the nextSegment with trx Idx directly. This goes against the original invariant of KIP 405, where we wanted the plugins to be quite dumb CRUD APIs. An ideal A

Re: [VOTE] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

+1 (I have participated in the review and I think we have a good solution in the KIP) -- Divij Vaidya On Mon, Nov 4, 2024 at 5:41 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi all, > > Bumping up the voting thread. > > On Mon, Sep 30, 2024 at 3:27 PM Francois Visconte

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Thanks for patiently addressing all the comments. I will add the vote in the other thread. -- Divij Vaidya On Mon, Nov 4, 2024 at 5:40 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi all, > > If the review comments are addressed and KIP looks good, please submit your > v

[jira] [Created] (KAFKA-17938) Minor : Remove Set.of usage in 3.9 branch.

Shivsundar R created KAFKA-17938: Summary: Minor : Remove Set.of usage in 3.9 branch. Key: KAFKA-17938 URL: https://issues.apache.org/jira/browse/KAFKA-17938 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-17935) Add metrics for whether produced messages are compressed

Xiao Yang created KAFKA-17935: - Summary: Add metrics for whether produced messages are compressed Key: KAFKA-17935 URL: https://issues.apache.org/jira/browse/KAFKA-17935 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-17919) enable back the failing testShareGroups test

[ https://issues.apache.org/jira/browse/KAFKA-17919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17919. Fix Version/s: 4.0.0 Resolution: Fixed > enable back the failing testShareGroups te

[jira] [Created] (KAFKA-17936) Refactor SslTransportLayerTest#testDsaKeyPair to avoid skip test

Chia-Ping Tsai created KAFKA-17936: -- Summary: Refactor SslTransportLayerTest#testDsaKeyPair to avoid skip test Key: KAFKA-17936 URL: https://issues.apache.org/jira/browse/KAFKA-17936 Project: Kafka

[jira] [Created] (KAFKA-17937) Use ValueSource to refactor AbstractFetcherThreadTest to eliminate unnecessary skip tests

Chia-Ping Tsai created KAFKA-17937: -- Summary: Use ValueSource to refactor AbstractFetcherThreadTest to eliminate unnecessary skip tests Key: KAFKA-17937 URL: https://issues.apache.org/jira/browse/KAFKA-17937

Re: [DISCUSS] KIP-1102: Enable clients to rebootstrap based on timeout or error code

Hi all, Any other feedback or suggestions? If there are no concerns, I will start the vote tomorrow. Thank you, Rajini On Wed, Oct 30, 2024 at 4:27 PM Rajini Sivaram wrote: > Hi Andrew, > > Thanks for reviewing the KIP. > > AS1: Updated to KIP-559. Thanks for pointing that out. > AS2: Yes, t

[jira] [Created] (KAFKA-17940) Allow starting clients and tools with the docker images

Mickael Maison created KAFKA-17940: -- Summary: Allow starting clients and tools with the docker images Key: KAFKA-17940 URL: https://issues.apache.org/jira/browse/KAFKA-17940 Project: Kafka I