Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Nick, Thanks for the updates and sorry for the delay on my side! 1. Making the default implementation for flush() a no-op sounds good to me. 2. I think what was bugging me here is that a third-party state store needs to implement the state store interface. That means they need to impleme

Re: Apache Kafka 3.6.0 release

Hi Satish, this is a small documentation fix about ZK to KRaft migration, that we would like to backport to 3.5 and 3.6 branches. Are you ok with that? https://github.com/apache/kafka/pull/14366 On Wed, Sep 13, 2023 at 3:13 AM Satish Duggana wrote: > > Thanks David for the quick resolution. > >

[jira] [Resolved] (KAFKA-14502) Implement LeaveGroup API

[ https://issues.apache.org/jira/browse/KAFKA-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14502. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement LeaveGroup API > -

[jira] [Created] (KAFKA-15460) Add group type filter to ListGroups API

David Jacot created KAFKA-15460: --- Summary: Add group type filter to ListGroups API Key: KAFKA-15460 URL: https://issues.apache.org/jira/browse/KAFKA-15460 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15461) Add integration test for the ListGroup API

David Jacot created KAFKA-15461: --- Summary: Add integration test for the ListGroup API Key: KAFKA-15461 URL: https://issues.apache.org/jira/browse/KAFKA-15461 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15462) Add group type filter to the admin client

David Jacot created KAFKA-15462: --- Summary: Add group type filter to the admin client Key: KAFKA-15462 URL: https://issues.apache.org/jira/browse/KAFKA-15462 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Bruno, Thanks for getting back to me! 2. The fact that implementations can always track estimated memory usage in the wrapper is a good point. I can remove -1 as an option, and I'll clarify the JavaDoc that 0 is not just for non-transactional stores, which is currently misleading. 6. The prob

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2199

See

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Bruno, Thinking about 3a. in addition to adding the IsolationLevel to QueryStoreParameters and Query, what about also adding a method like "ReadOnlyKeyValueStore view(IsolationLevel isolationLevel)" to ReadOnlyKeyValueStore? This would enable us to easily select/switch between IsolationLevels, ev

[jira] [Created] (KAFKA-15463) StreamsException: Accessing from an unknown node

Yevgeny created KAFKA-15463: --- Summary: StreamsException: Accessing from an unknown node Key: KAFKA-15463 URL: https://issues.apache.org/jira/browse/KAFKA-15463 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Nick, 6. Of course, you are right! My bad! Wiping out the state in the downgrading case is fine. 3a. Focus on the public facing changes for the KIP. We will manage to get the internals right. Regarding state stores that do not support READ_COMMITTED, they should throw an error stating that

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #73

See Changes: -- [...truncated 283096 lines...] > Task :streams:integrationTest Gradle Test Run :streams:integrationTest > Gradle Test Executor 181 > VersionedKeyValueStoreIn

Re: Re: Re: Re: [DISCUSS] KIP-971 Expose replication-offset-lag MirrorMaker2 metric

Elkhan, do you think making yours similar would make sense? On Wed, Sep 6, 2023 at 4:12 AM hudeqi <16120...@bjtu.edu.cn> wrote: > Hey, Viktor. > As far as my implementation is concerned, the default setting is 30s, but > I added it to `MirrorConnectorConfig`, which can be adjusted freely > accord

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2200

See

[jira] [Resolved] (KAFKA-15115) Implement resetPositions functionality in OffsetsRequestManager

[ https://issues.apache.org/jira/browse/KAFKA-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15115. - Fix Version/s: 3.7.0 Resolution: Fixed merged the PR to trunk > Implement resetPositions functio

[jira] [Resolved] (KAFKA-15163) Implement validatePositions functionality for new KafkaConsumer

[ https://issues.apache.org/jira/browse/KAFKA-15163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15163. - Fix Version/s: 3.7.0 Resolution: Fixed This is covered in https://github.com/apache/kafka/pull/14

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

Hi Ron, Thanks for drilling down on this. I think the KIP isn't really clear here, and the metadata caching section you quoted needs clarification. The "hosting broker's latest registration" refers to the previous, not the current registration. The registrations are only compared by the controlle

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Bruno, I've updated the KIP based on our conversation. The only things I've not yet done are: 1. Using transactions under ALOS and EOS. 2. Making IsolationLevel a query-time constraint, rather than linking it to the processing.guarantee. There's a wrinkle that makes this a challenge: Interact

Re: Apache Kafka 3.6.0 release

Hey Satish -- yes, you are correct. KAFKA-15459 only affects 3.6. PR should be finalized soon. Thanks, Justine On Wed, Sep 13, 2023 at 1:41 AM Federico Valeri wrote: > Hi Satish, this is a small documentation fix about ZK to KRaft > migration, that we would like to backport to 3.5 and 3.6 branc

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Addendum: I think we would also face the same problem with the approach John outlined earlier (using the record cache as a transaction buffer and flushing it straight to SST files). This is because the record cache (the ThreadCache class) is not thread-safe, so every commit would invalidate open I

Re: [VOTE] KIP-714: Client metrics and observability

Hey Andrew - Thank you for taking the time to reply to my questions. I'm just adding some notes to this discussion. 1. epoch: It can be helpful to know the delta of the client side and the actual leader epoch. It is helpful to understand why sometimes commit fails/client not making progress. 2.

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2201

See

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

On Tue, Sep 12, 2023, at 17:21, Calvin Liu wrote: > Hi Colin > Thanks for the comments! > Hi Calvin, Thanks again for the KIP. One meta-comment: it's usually better to just do a diff on a message spec file or java file if you're including changes to it in the KIP. This is easier to read than l

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.6 #39

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2202

See

[jira] [Created] (KAFKA-15464) Allow dynamic reloading of certificates with different DN / SANs

Jakub Scholz created KAFKA-15464: Summary: Allow dynamic reloading of certificates with different DN / SANs Key: KAFKA-15464 URL: https://issues.apache.org/jira/browse/KAFKA-15464 Project: Kafka

[DISCUSS] KIP-978: Allow dynamic reloading of certificates with different DN / SANs

Hi all, I would like to start the discussion about the KIP-978: Allow dynamic reloading of certificates with different DN / SANs . It proposes adding an option to disable the current validation of the DN and SANs when dyna

[jira] [Created] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

Ahmed HIBOT created KAFKA-15465: --- Summary: MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation Key: KAFKA-15465 URL: https://issues.apache.org/jira/browse/KAFKA-15465

[jira] [Resolved] (KAFKA-15459) Convert coordinator retriable errors to a known producer response error.

[ https://issues.apache.org/jira/browse/KAFKA-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15459. Resolution: Fixed > Convert coordinator retriable errors to a known producer response erro

[jira] [Created] (KAFKA-15466) Add KIP-919 support to kafka-features.sh, kafka-metadata-quorum.sh, kafka-cluster.sh

Colin McCabe created KAFKA-15466: Summary: Add KIP-919 support to kafka-features.sh, kafka-metadata-quorum.sh, kafka-cluster.sh Key: KAFKA-15466 URL: https://issues.apache.org/jira/browse/KAFKA-15466

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #40

See

Re: [VOTE] KIP-714: Client metrics and observability

Hey Andrew, +1 on the KIP. For many users of Kafka, it may not be fully understood how much of a challenge client monitoring is. With tens of clients in a cluster, it is already difficult to coordinate metrics collection. When there are thousands of clients, and when the cluster operator has no co

Re: Apache Kafka 3.6.0 release

Hi Satish, Since this PR: https://github.com/apache/kafka/pull/14366 only changes the doc, I've backported to 3.6 branch. FYI. Thanks. Luke On Thu, Sep 14, 2023 at 12:15 AM Justine Olshan wrote: > Hey Satish -- yes, you are correct. KAFKA-15459 only affects 3.6. > PR should be finalized soon.

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #41

See

Re: Apache Kafka 3.6.0 release

Thanks Luke for the update. ~Satish. On Thu, 14 Sept 2023 at 07:29, Luke Chen wrote: > > Hi Satish, > > Since this PR: > https://github.com/apache/kafka/pull/14366 only changes the doc, I've > backported to 3.6 branch. FYI. > > Thanks. > Luke > > On Thu, Sep 14, 2023 at 12:15 AM Justine Olshan >

Re: [VOTE] KIP-942: Add Power(ppc64le) support

I don’t have a strong opinion on this. I am fine with running these tests nightly instead of on every merge to trunk. On Wed 13. Sep 2023 at 08:13, Vaibhav Nazare wrote: > Hi Colin, > I do agree with you on running a nightly job. > > Any thoughts on this Mickael and Divij so I can update the KIP

Re: [VOTE] KIP-942: Add Power(ppc64le) support

+1 (binding) On Wed 26 Jul 2023 at 09:04, Vaibhav Nazare wrote: > I'd like to call a vote on KIP-942 >