[jira] [Reopened] (KAFKA-15743) KRaft support in ReplicationQuotasTest

[ https://issues.apache.org/jira/browse/KAFKA-15743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano reopened KAFKA-15743: --- The {{registerBroker}} function in this test file needs to use the directory UUIDs

Re: [VOTE] 3.7.0 RC4

HI, I've downloaded, built from source and then validated JBOD with KRaft works along with migrating a cluster with JBOD from ZK to KRaft works. +1 (nonbinding) from me. --Proven On Tue, Feb 20, 2024 at 2:13 PM Justine Olshan wrote: > Hey folks, > > I've done the following to validate the rel

Re: [VOTE] 3.7.0 RC2

, 2024 at 12:04 PM Proven Provenzano wrote: > > I have a PR https://github.com/apache/kafka/pull/15197 for > https://issues.apache.org/jira/browse/KAFKA-16131 that is building now. > --Proven > > On Mon, Jan 15, 2024 at 5:03 AM Jakub Scholz wrote: > >> *> Hi Jakub,> &g

Re: [VOTE] 3.7.0 RC2

I have a PR https://github.com/apache/kafka/pull/15197 for https://issues.apache.org/jira/browse/KAFKA-16131 that is building now. --Proven On Mon, Jan 15, 2024 at 5:03 AM Jakub Scholz wrote: > *> Hi Jakub,> > Thanks for trying the RC. I think what you found is a > blocker bug because it * > *>

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

> > > In the test plan, should we also have one system test checking that > > "features with a stable MV will never have that MV changed"? > > > > On Wed, Jan 10, 2024 at 8:16 AM Colin McCabe wrote: > > > > > > On Tue, Jan 9, 2024, at 18:56,

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

t; > In the test plan, should we also have one system test checking that > "features with a stable MV will never have that MV changed"? > > On Wed, Jan 10, 2024 at 8:16 AM Colin McCabe wrote: > > > > On Tue, Jan 9, 2024, at 18:56, Proven Provenzano wrote: > > > H

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

gt; >> >> of us who deploy from trunk (or branches dervied from trunk). > >> >> > >> >> With regard to "unstable.metadata.versions.enable": is this going to > be > >> a > >> >> documented configuration, or an internal one

[jira] [Resolved] (KAFKA-15922) Add MetadataVersion for JBOD

[ https://issues.apache.org/jira/browse/KAFKA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15922. --- Resolution: Fixed > Add MetadataVersion for J

[DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

Hey folks, I am starting a discussion thread for managing unstable metadata versions in Apache Kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1014%3A+Managing+Unstable+Metadata+Versions+in+Apache+Kafka This KIP is actually already implemented in 3.7 with PR https://github.com/apache

Re: Apache Kafka 3.7.0 Release

We have potentially one more blocker https://issues.apache.org/jira/browse/KAFKA-16082 which might cause a data loss scenario with JBOD in KRaft. Initial analysis thought this is a problem and further review looks like it isn't but we are continuing to dig into the issue to ensure that it isn't. We

[jira] [Created] (KAFKA-16082) JBOD: Possible dataloss when moving leader partition

Proven Provenzano created KAFKA-16082: - Summary: JBOD: Possible dataloss when moving leader partition Key: KAFKA-16082 URL: https://issues.apache.org/jira/browse/KAFKA-16082 Project: Kafka

[jira] [Resolved] (KAFKA-15904) Downgrade tests are failing with directory.id 

[ https://issues.apache.org/jira/browse/KAFKA-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15904. --- Resolution: Fixed This was merged into trunk a month ago, long before the 3.7

[jira] [Resolved] (KAFKA-15893) Bump MetadataVersion for directory assignments

[ https://issues.apache.org/jira/browse/KAFKA-15893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15893. --- Fix Version/s: 3.7.0 Resolution: Fixed This is finally merged into 3.7

[jira] [Resolved] (KAFKA-15633) Bug: Generated Persistent Directory IDs are overwritten on startup.

[ https://issues.apache.org/jira/browse/KAFKA-15633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15633. --- Resolution: Fixed > Bug: Generated Persistent Directory IDs are overwritten

[jira] [Created] (KAFKA-15633) Bug: Generated Persistent Directory IDs are overwritten on startup.

Proven Provenzano created KAFKA-15633: - Summary: Bug: Generated Persistent Directory IDs are overwritten on startup. Key: KAFKA-15633 URL: https://issues.apache.org/jira/browse/KAFKA-15633

Re: [VOTE] 3.6.0 RC2

Hi, To verify the release of release 3.6.0 RC2 I did the following: - Downloaded the source, built and ran the tests. - Validated SCRAM with KRaft including creating credentials with kafka-storage. - Validated Delegation Tokens with KRaft +1 (non-binding) --Proven On Mon, Oct 2,

[jira] [Reopened] (KAFKA-15411) DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky

[ https://issues.apache.org/jira/browse/KAFKA-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano reopened KAFKA-15411: --- > DelegationTokenEndToEndAuthorizationWithOwnerTest is Fl

[jira] [Resolved] (KAFKA-15411) DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky

[ https://issues.apache.org/jira/browse/KAFKA-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15411. --- Resolution: Cannot Reproduce After trying for a week, I have yet to see the error

[jira] [Resolved] (KAFKA-15422) Update documentation for Delegation Tokens in Kafka with KRaft

[ https://issues.apache.org/jira/browse/KAFKA-15422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15422. --- Resolution: Fixed > Update documentation for Delegation Tokens in Kafka with KR

[jira] [Created] (KAFKA-15422) Update documentation for Delegation Tokens in Kafka with KRaft

Proven Provenzano created KAFKA-15422: - Summary: Update documentation for Delegation Tokens in Kafka with KRaft Key: KAFKA-15422 URL: https://issues.apache.org/jira/browse/KAFKA-15422 Project

[jira] [Created] (KAFKA-15411) DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky

Proven Provenzano created KAFKA-15411: - Summary: DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky Key: KAFKA-15411 URL: https://issues.apache.org/jira/browse/KAFKA-15411 Project: Kafka

[jira] [Created] (KAFKA-15114) StorageTool help specifies user as parameter not name

Proven Provenzano created KAFKA-15114: - Summary: StorageTool help specifies user as parameter not name Key: KAFKA-15114 URL: https://issues.apache.org/jira/browse/KAFKA-15114 Project: Kafka

[jira] [Resolved] (KAFKA-15017) New ClientQuotas are not written to ZK from snapshot

[ https://issues.apache.org/jira/browse/KAFKA-15017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15017. --- Resolution: Fixed > New ClientQuotas are not written to ZK from snaps

[jira] [Resolved] (KAFKA-14970) Dual write mode testing for SCRAM and Quota

[ https://issues.apache.org/jira/browse/KAFKA-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14970. --- Resolution: Fixed Committed and merged into 3.5. > Dual write mode testing

[jira] [Resolved] (KAFKA-14084) Support SCRAM when using KRaft mode

[ https://issues.apache.org/jira/browse/KAFKA-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14084. --- Resolution: Fixed SCRAM support in KRaft is implemented. Test of SCRAM

[jira] [Created] (KAFKA-14970) Dual write mode testing for SCRAM and Quota

Proven Provenzano created KAFKA-14970: - Summary: Dual write mode testing for SCRAM and Quota Key: KAFKA-14970 URL: https://issues.apache.org/jira/browse/KAFKA-14970 Project: Kafka Issue

[jira] [Resolved] (KAFKA-14859) Support SCRAM ZK to KRaft Migration

[ https://issues.apache.org/jira/browse/KAFKA-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14859. --- Resolution: Fixed Merged into trunk and 3.5 > Support SCRAM ZK to KRaft Migrat

[jira] [Resolved] (KAFKA-14948) Broker fails to rejoin cluster when cluster is in dual write mode

[ https://issues.apache.org/jira/browse/KAFKA-14948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14948. --- Resolution: Cannot Reproduce I updated my code to tip of master and cannot

[jira] [Created] (KAFKA-14948) Broker fails to rejoin cluster when cluster is in dual write mode

Proven Provenzano created KAFKA-14948: - Summary: Broker fails to rejoin cluster when cluster is in dual write mode Key: KAFKA-14948 URL: https://issues.apache.org/jira/browse/KAFKA-14948 Project

[jira] [Resolved] (KAFKA-14881) Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

[ https://issues.apache.org/jira/browse/KAFKA-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14881. --- Resolution: Fixed Merged to Trunk and 3.5. > Update UserScramCredentialRecord

[jira] [Created] (KAFKA-14881) Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

Proven Provenzano created KAFKA-14881: - Summary: Update UserScramCredentialRecord for SCRAM ZK to KRaft migration Key: KAFKA-14881 URL: https://issues.apache.org/jira/browse/KAFKA-14881 Project

[jira] [Resolved] (KAFKA-14765) Support SCRAM for brokers at bootstrap

[ https://issues.apache.org/jira/browse/KAFKA-14765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14765. --- Resolution: Fixed [PR #13374|https://github.com/apache/kafka/pull/13374]includes

[jira] [Resolved] (KAFKA-14776) Update SCRAM integration tests to run with KRaft

[ https://issues.apache.org/jira/browse/KAFKA-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14776. --- Resolution: Fixed [PR #13374|https://github.com/apache/kafka/pull/13374]includes

[jira] [Created] (KAFKA-14861) Migration Documentation should state the need for connectivity from controller to brokers when provisioning the KRaft controller quorum

Proven Provenzano created KAFKA-14861: - Summary: Migration Documentation should state the need for connectivity from controller to brokers when provisioning the KRaft controller quorum Key: KAFKA-14861 URL

[jira] [Created] (KAFKA-14860) New controllers created for migration should validate connectivity to brokers before starting migration

Proven Provenzano created KAFKA-14860: - Summary: New controllers created for migration should validate connectivity to brokers before starting migration Key: KAFKA-14860 URL: https://issues.apache.org/jira

[jira] [Created] (KAFKA-14859) Support SCRAM ZK to KRaft Migration

Proven Provenzano created KAFKA-14859: - Summary: Support SCRAM ZK to KRaft Migration Key: KAFKA-14859 URL: https://issues.apache.org/jira/browse/KAFKA-14859 Project: Kafka Issue Type

[jira] [Created] (KAFKA-14776) Update SCRAM system tests to run with KRaft

Proven Provenzano created KAFKA-14776: - Summary: Update SCRAM system tests to run with KRaft Key: KAFKA-14776 URL: https://issues.apache.org/jira/browse/KAFKA-14776 Project: Kafka Issue

[jira] [Created] (KAFKA-14775) Support SCRAM for broker to controller authentication

Proven Provenzano created KAFKA-14775: - Summary: Support SCRAM for broker to controller authentication Key: KAFKA-14775 URL: https://issues.apache.org/jira/browse/KAFKA-14775 Project: Kafka

Re: [VOTE] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for Kafka Brokers

Hi all, I am going to close the vote and start implementing. The KIP is accepted with three binding votes from Colin, Jose, and Manikumar. --Proven On Sat, Feb 25, 2023 at 1:21 AM Manikumar wrote: > +1 (binding) > > Thanks for the KIP. > > On Wed, Feb 22, 2023 at 3:48 AM José Armando García San

[jira] [Created] (KAFKA-14765) Support SCRAM for brokers at bootstrap

Proven Provenzano created KAFKA-14765: - Summary: Support SCRAM for brokers at bootstrap Key: KAFKA-14765 URL: https://issues.apache.org/jira/browse/KAFKA-14765 Project: Kafka Issue Type

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

al Kafka doc link? That is > this one: https://kafka.apache.org/documentation/#security_sasl_scram > > Thank you. > Luke > > > On Wed, Feb 22, 2023 at 6:17 AM José Armando García Sancio > wrote: > > > Hi Proven, > > > > On Tue, Feb 21, 2023 at 1:37 PM Proven Proven

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

Hi Jose, 1. The SCRAM in SCRAM-SHA-256 is required as the mechanism name is SCRAM-SHA-256. I do realize there is a bit of redundancy here. 2. I'll add documentation for all the possible values. They are SCRAM-SHA-256 and SCRAM-SHA-512. 3. I'd like to keep it with a capital letter as it is a

[VOTE] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for Kafka Brokers

Hi, I'd like to start the vote on KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for Kafka Brokers. The KIP is here: KIP-900 The Original discuss

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

ecord is incomplete, then kafka-storage will exit with a failure. --Proven On Mon, Feb 13, 2023 at 4:54 PM José Armando García Sancio wrote: > Comments below. > > On Mon, Feb 13, 2023 at 11:44 AM Proven Provenzano > wrote: > > > > Hi Jose > > > > I want

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

Hi Jose I want to clarify that the file parsing that Argparse4j provides is just a mechanism for taking command line args and putting them in a file. It doesn't actually change what the command line args are for processing the file. So I can add any kafka-storage command line arg into the file inc

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

6:20 PM Colin McCabe wrote: > On Fri, Jan 27, 2023, at 11:25, Proven Provenzano wrote: > > Hi Colin, > > > > So I think we are converging where we put the adding of records in > > kafka-storage, allow for salt and iterations to be optional and now we > are > >

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

-secret],SCRAM-SHA-512=[password=alice-secret]' --entity-type users --entity-name alice Thoughts? --Proven On Tue, Jan 24, 2023 at 12:22 AM Colin McCabe wrote: > On Fri, Jan 20, 2023, at 13:02, Proven Provenzano wrote: > > Hi Colin, > > Thanks for the response. > > >

Re: [DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

r the > plight of people with low limits on the maximum length of their command > lines. One fix for these people could be allowing them to read their > arguments from a file like this: > > $ ./bin/kafka-storage.sh @myfile > $ cat myfile: > ./bin/kafka-storage.sh format \ > --config

[DISCUSS] KIP-900: KRaft kafka-storage.sh API additions to support SCRAM

I have written a KIP describing the API additions needed to kafka-storage to store SCRAM credentials at bootstrap time. Please take a look at https://cwiki.apache.org/confluence/display/KAFKA/KIP-900%3A+KRaft+kafka-storage.sh+API+additions+to+support+SCRAM+for+Kafka+Brokers -- --Proven

Request for permission to contribute to Apache Kafka

Hi, I'd like permission to create KIPs. I am working on SCRAM for KRaft and would like to file a KIPs to support this work. This is to add bootstrapping SCRAM for KRaft by extending KIP-631. The basic SCRAM code without bootstrap is available for comments in PR https://github.com/apache/kafka/pull/

[jira] [Resolved] (KAFKA-14398) Update EndToEndAuthorizerTest.scala to test with ZK and KRAFT quorum servers

[ https://issues.apache.org/jira/browse/KAFKA-14398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14398. --- Resolution: Resolved > Update EndToEndAuthorizerTest.scala to test with ZK

[jira] [Resolved] (KAFKA-14375) Remove use of "authorizer-properties" in EndToEndAuthorizationTest.scala

[ https://issues.apache.org/jira/browse/KAFKA-14375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-14375. --- Resolution: Fixed > Remove use of "authorizer-proper

[jira] [Created] (KAFKA-14398) Update EndToEndAuthorizerTest.scala to test with ZK and KRAFT quorum servers

Proven Provenzano created KAFKA-14398: - Summary: Update EndToEndAuthorizerTest.scala to test with ZK and KRAFT quorum servers Key: KAFKA-14398 URL: https://issues.apache.org/jira/browse/KAFKA-14398

[jira] [Created] (KAFKA-14375) Remove use of "authorizer-properties" n EndToEndAuthorizationTest.scala

Proven Provenzano created KAFKA-14375: - Summary: Remove use of "authorizer-properties" n EndToEndAuthorizationTest.scala Key: KAFKA-14375 URL: https://issues.apache.org/jira/browse/K