Re: [VOTE] 3.6.0 RC2

Hi Satish, I did the following to verify the release: - Built from source with Java 17 and Scala 2.13 - Ran all unit and integration tests - Spot checked documentation - Ran custom client applications using staging artifacts on a 3-nodes cluster - Tested tiered storage with one of the available RS

[jira] [Created] (KAFKA-15525) Segment uploads stop working following a broker failure

Francois Visconte created KAFKA-15525: - Summary: Segment uploads stop working following a broker failure Key: KAFKA-15525 URL: https://issues.apache.org/jira/browse/KAFKA-15525 Project: Kafka

Re: [VOTE] 3.6.0 RC2

+ 1 (non-binding) Verifications: 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd compression using an open messaging benchmark and found 3.6 to be better than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently 6-7% lower CPU utilization, lesser spikes on P9

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2244

See

Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers

Hi Ron, Yes, that's what I am proposing, yes. Ismael On Sat, Sep 30, 2023 at 2:30 PM Ron Dagostino wrote: > Thanks, Ismael. I think you are proposing a pair of mutually exclusive > args --process.roles and --node.id, right? I agree that is more > user-friendly than the --required-config arg,

Re: [VOTE] 3.6.0 RC2

Hi, To verify the release of release 3.6.0 RC2 I did the following: - Downloaded the source, built and ran the tests. - Validated SCRAM with KRaft including creating credentials with kafka-storage. - Validated Delegation Tokens with KRaft +1 (non-binding) --Proven On Mon, Oct 2,

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi David 01 Same as Java Client's existing behaviour of requesting a full expedited metadata-refresh done asynchronously, for error NOT_LEADER OR FENCED_EPOCH, the KIP proposes to continue doing so. As this would help prevent similar errors on future requests to other partitions affected by leader

Need generate a KIP

wiki ID: pzheng Jira ID: hanyuzheng -- [image: Confluent] Hanyu (Peter) Zheng he/him/his Software Engineer Intern +1 (213) 431-7193 <+1+(213)+431-7193> Follow us: [image: Blog]

Re: Need generate a KIP

Hi Peter, You are set :) Please share if you also need Jira rights. Best, On Mon, Oct 2, 2023 at 5:59 PM Hanyu (Peter) Zheng wrote: > wiki ID: pzheng > Jira ID: hanyuzheng > > -- > > [image: Confluent] > Hanyu (Peter) Zheng he/him/his > Software Engineer Intern > +

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2245

See Changes: -- [...truncated 314964 lines...] Gradle Test Run :streams:test > Gradle Test Executor 81 > TasksTest > shouldAddAndRemovePendingTaskToUpdateInputPartitions(

Re: Need generate a KIP

Thank you, Josep. Yes I also need Jira rights. On Mon, Oct 2, 2023 at 9:05 AM Josep Prat wrote: > Hi Peter, > > You are set :) Please share if you also need Jira rights. > > Best, > > On Mon, Oct 2, 2023 at 5:59 PM Hanyu (Peter) Zheng > wrote: > > > wiki ID: pzheng > > Jira ID: hanyuzheng > >

[jira] [Created] (KAFKA-15526) Simplify LogAppendInfo parameters

David Mao created KAFKA-15526: - Summary: Simplify LogAppendInfo parameters Key: KAFKA-15526 URL: https://issues.apache.org/jira/browse/KAFKA-15526 Project: Kafka Issue Type: Improvement

DISCUSS KIP-984 Add pluggable compression interface to Kafka

https://cwiki.apache.org/confluence/display/KAFKA/KIP-984%3A+Add+pluggable+compression+interface+to+Kafka

Re: Need generate a KIP

Done. Thanks for showing interest in Apache Kafka! Best, On Mon, Oct 2, 2023 at 6:13 PM Hanyu (Peter) Zheng wrote: > Thank you, Josep. Yes I also need Jira rights. > > On Mon, Oct 2, 2023 at 9:05 AM Josep Prat > wrote: > > > Hi Peter, > > > > You are set :) Please share if you also need Jira

[jira] [Created] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

Hanyu Zheng created KAFKA-15527: --- Summary: Add reverseRange and reverseAll query over kv-store in IQv2 Key: KAFKA-15527 URL: https://issues.apache.org/jira/browse/KAFKA-15527 Project: Kafka Is

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Yash, Thanks for the KIP! Frankly this feels like an oversight in 875 and I'm glad we're closing that loop ASAP. Here are my thoughts: 1. (Nit): IMO "start.state", "initial.state", or "target.state" might be better than just "state" for the field name in the connector creation request. 2. W

[jira] [Resolved] (KAFKA-15449) Verify transactional offset commits (KIP-890 part 1)

[ https://issues.apache.org/jira/browse/KAFKA-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15449. Resolution: Fixed > Verify transactional offset commits (KIP-890 part 1) > ---

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #76

See Changes: -- [...truncated 2712284 lines...] at jdk.internal.reflect.GeneratedMethodAccessor487.invoke(Unknown Source) at java.base/jdk.internal.reflect.De

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2246

See Changes: -- [...truncated 420469 lines...] Gradle Test Run :streams:test > Gradle Test Executor 77 > TasksTest > onlyRemovePendingTaskToCloseDirtyShouldRemoveTaskFrom

Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers

Hi Ismeal, Thanks for the comments. I'll change the implementation to use a pair of mutually exclusive args --process.roles and --node.id. Thanks, Hailey On Mon, Oct 2, 2023 at 6:34 AM Ismael Juma wrote: > Hi Ron, > > Yes, that's what I am proposing, yes. > > Ismael > > On Sat, Sep 30, 2023 at

Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers

Hi Kamal, I think the broker.id property has been replaced with the `node.id` property in KRaft. The documentation for `node.id` says it is required ( https://github.com/apache/kafka/blob/72e275f6ea867747e6b4e524c80d5ebd726ac25b/core/src/main/scala/kafka/server/KafkaConfig.scala#L741), and the Qu

Re: [VOTE] 3.6.0 RC2

Hi Satish, Thanks for running the release. I performed the following steps: - Validated all the checksums, signatures, and keys - Built the release from source - Ran all unit tests - Quick start validations - ZK and Kraft - Connect - Kafka Streams - Spot checked j

Re: [VOTE] 3.6.0 RC2

Hey all -- I noticed we still have the system tests as something that will be updated. Did we get a run for this RC? On Mon, Oct 2, 2023 at 1:24 PM Bill Bejeck wrote: > Hi Satish, > > Thanks for running the release. > I performed the following steps: > >- Validated all the checksums, signatu

[DISCUSS] KIP-986: Cross-Cluster Replication

Hi all, I've opened an extremely early draft of the Cross-Cluster Replication feature, and I would like to invite any and all co-authors to expand on it. Find the page here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-986%3A+Cross-Cluster+Replication This is not strictly an invitation t

[jira] [Created] (KAFKA-15528) KIP-986: Cross-Cluster Replication

Greg Harris created KAFKA-15528: --- Summary: KIP-986: Cross-Cluster Replication Key: KAFKA-15528 URL: https://issues.apache.org/jira/browse/KAFKA-15528 Project: Kafka Issue Type: New Feature

Re: Re: [DISCUSS] Cluster Linking / Cross-Cluster Replication - Call for Interest & Co-Authors

Hi everyone, Following up on this thread, I've opened a draft KIP for everyone to contribute to: https://cwiki.apache.org/confluence/display/KAFKA/KIP-986%3A+Cross-Cluster+Replication https://lists.apache.org/thread/4563fqdcwlqwjqlm7mbj8wftx1rpo0yd Please consider yourselves owners of this featur

[REVIEW REQUEST] ReassignPartitionCommand scala -> java

Hello. Thanks everyone for the help. I think we come to the stage when command itself can be reviewed. All tests and dependencies of ReassignPartitionCommand translated from scala to java in previous PR’s. Please, join the review - https://github.com/apache/kafka/pull/13247

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.6 #82

See

Re: [VOTE] 3.6.0 RC2

Hey Satish, I verified KIP-898 functionality and the KAFKA-15473 patch. +1 (non-binding) Thanks! On Mon, Oct 2, 2023 at 1:28 PM Justine Olshan wrote: > > Hey all -- I noticed we still have the system tests as something that will > be updated. Did we get a run for this RC? > > On Mon, Oct 2, 202

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2247

See Changes: -- [...truncated 315427 lines...] Gradle Test Run :streams:test > Gradle Test Executor 84 > TasksTest > onlyRemovePendingTaskToUpdateInputPartitionsShouldRem

Re: [DISCUSS] KIP-714: Client metrics and observability

Hi, I did not pay attention to this KIP in the past; seems it was on-hold for a while. Overall it sounds very useful, and I think we should extend this with a follow up KIP for Kafka Streams. What is unclear to me at this point is the statement: Kafka Streams applications have an applicat

Re: [VOTE] 3.6.0 RC2

I realized Luke shared the results here for RC1 https://drive.google.com/drive/folders/1S2XYd79f6_AeWj9f9qEkliRg7JtL04AC Given we had some runs that looked reasonable, and we made a small change, I'm ok with this. But I wouldn't be upset if we had another set of runs :) As for the validation:

Re: [DISCUSS] KIP-986: Cross-Cluster Replication

Hi Greg, Thanks for this KIP! It is obviously very ambitious, but it's great to have a conversation about it. I'll start with some general points: Do you have a plan in mind for how to proceed with elaborating this KIP? While I like how you're involving the community in elaborating the KIP, I th

Re: [DISCUSS] KIP-986: Cross-Cluster Replication

Hey Tom, Thanks for the high-level questions, as I am certainly approaching this KIP differently than I've seen before. I think that ideally this KIP will expand to include lots of requirements and possible implementations, and that through discussion we can narrow the scope and form a roadmap fo

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2248

See

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

Thanks for updating the KIP. Just re-read it. Overall LGTM. A few nits: single-key lookup with timestamp (upper) bound Not sure if "(upper) bound" is the right phrasing? Personally, I find it a little bit confusing, because the term "bound" kinda indicates a range, but we still do a point loo

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Thanks for the updated KIP. Overall I like it. Victoria raises a very good point, and I personally tend to prefer (I believe so does Victoria, but it's not totally clear from her email) if a range query would not return any tombstones, ie, only two records in Victoria's example. Thus, it seems

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

Thanks for updating the KIP. Not sure if I agree or not with Bruno's idea to split the query types further? In the end, we split them only because there is three different return types: single value, value-iterator, key-value-iterator. What do we gain by splitting out single-ts-range-key? In