Re: [kafka-clients] Re: [VOTE] 3.3.0 RC2

e. I'd like to close out the vote by the end of day today, so please take a look. Thanks! David On Thu, Sep 22, 2022 at 9:06 AM David Arthur wrote: > > Josep, thanks for the note. We will mention the CVEs fixed in this release > in the announcement email. I believe we can also up

Re: [VOTE] 3.3.0 RC2

source release. I noticed a non-blocker issue with the KRaft readme > and submitted a PR: > > https://github.com/apache/kafka/pull/12688 > > Ismael > > On Tue, Sep 20, 2022 at 4:17 PM David Arthur wrote: > > > Hello Kafka users, developers and client-developers, > > &g

[VOTE] KIP-868 Metadata Transactions

Hey folks, I'd like to start a vote on KIP-868. This proposal adds lightweight transactions to the KRaft controller. These transactions will allow us to generate atomic batches of records that exceed the current limits of the metadata layer. Here is the KIP: https://cwiki.apache.org/confluence/dis

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

ts? > > -- > Best, > Ziming > > On Wed, Sep 21, 2022 at 10:56 PM David Arthur wrote: > > > Ziming, thanks for the feedback! Let me know your thoughts on #2 and #3 > > > > 1. Good idea. I consolidated all the details of record visibility into > > that sec

Re: [VOTE] 3.3.0 RC2

vote from Jakub Scholz No -1 votes The vote for Apache Kafka 3.3 passes! Thanks to everyone who voted and helped verify this release! A special thanks to José who has driven the release up to this point. Best, David Arthur On Tue, Sep 27, 2022 at 6:50 AM Mickael Maison wrote: > > +1 (bin

Re: [DISCUSS] KIP-868 Metadata Transactions (new thread)

> David, > > > Thanks for the feedback about #2 and #3, I'm OK with them. > > > I also mentioned the visibility in the MetadataShell in #1, do you have > > any > > > thoughts? > > > > > > -- > > > Best, > > > Ziming > > &g

Re: [VOTE] 3.3.0 RC2

gt; > 4. Observe that the current website does not have this bug. > > > > > > Divij Vaidya > > > > > > > > On Tue, Sep 27, 2022 at 8:35 PM David Arthur > > wrote: > > > >> I re-ran the failing system tests last night and got passin

Re: [VOTE] 3.3.0 RC2

For those interested, here's a PR to fix the HTML rendering issue: https://github.com/apache/kafka-site/pull/446 -David On Wed, Sep 28, 2022 at 9:45 AM David Arthur wrote: > > Thanks Divij, we made a bunch of documentation changes at the last minute for > this release, so w

Re: [VOTE] KIP-868 Metadata Transactions

> > > On Sep 28, 2022, at 10:05 PM, David Jacot > > > wrote: > > > > > > +1 (binding). Thanks for the KIP. I really like the approach! > > > > > > Best, > > > David > > > > > > On Wed, Sep 28, 2022 at 4:20 AM Luke C

[DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Hey folks, I wanted to get the ball rolling on the discussion for the ZooKeeper migration KIP. This KIP details how we plan to do an online migration of metadata from ZooKeeper to KRaft as well as a rolling upgrade of brokers to KRaft mode. The general idea is to keep KRaft and ZooKeeper in sync d

Re: [VOTE] KIP-876: Time based cluster metadata snapshots

er metadata > > > snapshots." > > > > > > KIP: https://cwiki.apache.org/confluence/x/MY3GDQ > > > Discussion thread: > > > https://lists.apache.org/thread/ww67h9d4xvgw1f7jn4zxwydmt8x1mq72 > > > > > > Thanks! > > > -- > > > -José > > > > -- David Arthur

Re: [ANNOUNCE] New committer: Deng Ziming

Fantastic! Congratulations, Ziming! -David On Tue, Oct 11, 2022 at 12:51 PM José Armando García Sancio wrote: > > Congratulations Ziming. Well deserved and I much appreciate your > contributions to the project. > > -- > -José -- David Arthur

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

ire any protocol changes to > > UpdateMetadata? > > > > > > > Yeah, I am also curious why the we need to care whether brokers are ZK or > > KRaft in UpdateMetadataRequest. We don't reveal this to clients, so can we > > just leave this out? > > >

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

here is self-contradictory, as you pointed out. > > >> > > >> I suspect what we should do here is add a new field, > KRaftControllerId, > > >> and populate it with the real controller ID, and leave the old > > controllerId > > >> field as -1. A

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

>> > “ControllerId” field in this request, the controller should > specify a > >> >> > random KRaft broker." > >> >> > "In the UpdateMetadataRequest sent by the KRaft controller to the > ZK > >> >> > brokers, the Controller

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

ting cluster are configured with multiple log dirs, does the migration > imply that the existing brokers need to drop use of that feature? Or is > there some way to upgrade them later? > > Thanks, > > -- > Igor > > On Mon, Oct 17, 2022, at 10:07 PM, David Arthur wrote

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

igible" is reported by ZK controller, and > > the rest of states are reported by KRaft controller --> makes sense to > me. > > One question from it is, when KRaft controller takes over the leadership > > from ZK controller, what will the "MigrationState" value i

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

s in the cluster to check that the endpoints don't conflict. Things that do use watches are dynamic configs, ACLs, and some others (listed in the KIP). On Wed, Oct 26, 2022 at 4:26 PM David Arthur wrote: > Luke and Andrew, thanks for taking a look! > > I think the names of the state

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

dow for incorrect broker registration to > sneak in during KRaft controller failover. > > 26. Then, we could just remove Broker Registration in that section. > > Thanks, > > Jun > > On Wed, Oct 26, 2022 at 2:21 PM David Arthur > wrote: > > > Jun > > > > 2

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

a metadata record? > > 22. If we have the broker fencing logic, do we need to write the broker > registration path in ZK for KRaft brokers at all? > > Thanks, > > Jun > > > On Thu, Oct 27, 2022 at 1:02 PM David Arthur > wrote: > > > Jun, > > > >

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

> > 20/21. Sounds good. > > Could you update the doc with all the changes being discussed? > > Thanks, > > Jun > > On Fri, Oct 28, 2022 at 10:11 AM David Arthur > wrote: > > > Jun, > > > > 20/21. I was also wondering about a "migration" reco

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

es allowing for ZK controller election to take > place". I guess the operator only needs to remove the /controller path? > > Thanks, > > Jun > > On Mon, Oct 31, 2022 at 7:17 AM David Arthur > wrote: > > > Happy Monday, everyone! I've updated the

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

s is a bit intrusive since it exposes unneeded info to the > > clients. Another option is to add that field as part of the Fetch > request. > > We can choose to only set that field in the very first Fetch request > from a > > Quorum follower. > > > > 40. For kaf

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

aft runs in the combined mode, does a controller know whether > an ApiRequest is from a client or another broker? > > 40. Adding a "None" state sounds reasonable. > > Thanks, > > Jun > > On Thu, Nov 3, 2022 at 8:39 AM David Arthur > wrote: > > > Jun, &g

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

; to use a single port for both client and server side communications. > > Thanks, > > Jun > > On Thu, Nov 3, 2022 at 9:59 AM David Arthur > wrote: > > > 20/21, in combined mode we still have a separate listener for the > > controller

[VOTE] KIP-866 ZooKeeper to KRaft Migration

Hello everyone, I'd like to start the vote on KIP-866. https://cwiki.apache.org/confluence/display/KAFKA/KIP-866+ZooKeeper+to+KRaft+Migration Thanks! David Arthur

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

anks, > > Jun > > On Tue, Nov 8, 2022 at 7:18 AM David Arthur wrote: > > > Thanks for the discussion everyone, I'm going to move ahead with the > > vote for this KIP. > > > > -David > > > > On Thu, Nov 3, 2022 at 1:20 PM Jun Rao wrote: > > &

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

uld we make the name more specific to ZK migration like > zookeeper.metadata.migration.enable? > > Thanks, > > Jun > > On Tue, Nov 8, 2022 at 9:47 AM David Arthur wrote: > > > Ah, sorry about that, you're right. Since we won't support ZK > > migr

Re: [VOTE] KIP-866 ZooKeeper to KRaft Migration

act the > KRaft quorum in order to forward requests to there, once we are in migration > mode. This raises a question: rather than changing the broker registration, > can we have those brokers send an RPC to the kraft controller quorum instead? > This would serve to confirm that they ca

Re: [VOTE] KIP-866 ZooKeeper to KRaft Migration

#x27;ve replaced the ZK registration section with a new RPC and brief description. Please take a look. Thanks! David On Wed, Nov 9, 2022 at 5:46 PM Colin McCabe wrote: > > Hi David, > > Thanks for the response. Replies inline. > > On Wed, Nov 9, 2022, at 08:17, David Arthu

Re: [DISCUSS] Apache Kafka 3.4.0 release

Sophie, the vote for KIP-866 is underway, but there is still some discussion happening. I'm hopeful that the vote can close this week, but it may fall into next week. Can we include this KIP in 3.4? Thanks, David On Tue, Nov 15, 2022 at 6:52 AM Rajini Sivaram wrote: > Hi Sophie, > > I was out o

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Jun, Thanks for the comments. Igor, please see 54 below for some additional discussion on the meta.properties 50.1 Yes, that field name sounds fine to me. 50.2 Ok, I'll add something to the KIP under the Controller section. To your other question, NoOpRecords are used as part of our liveness chec

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

broker > registration since the KRaft controller has already generated a > BrokerRegistrationRecord based on BrokerRegistrationRequest? > > Thanks, > > Jun > > On Tue, Nov 29, 2022 at 7:14 AM David Arthur > wrote: > > > Jun, Thanks for the comments. Igor, please see 54 below fo

Re: [VOTE] KIP-866 ZooKeeper to KRaft Migration

s, but I agree the alternative has risks > > as well. > > > > Best, > > Jason > > > > On Mon, Nov 14, 2022 at 12:00 PM Colin McCabe wrote: > > > > > On Fri, Nov 11, 2022, at 08:59, David Arthur wrote: > > > > Thanks, Colin. > > &g

Re: [DISCUSS] Apache Kafka 3.4.0 release

t; > > Hi Sophie, > > > > KIP-881 has three binding votes (David Jacot, Jun and me) and one > > non-binding vote (Maulin). So it is good to go for 3.4.0 if there are no > > objections until the voting time of 72 hours completes on Friday. > > > > Thanks, &g

Re: [DISCUSS] Apache Kafka 3.4.0 release

Hey Sophie, I found a KRaft blocker for 3.4 https://issues.apache.org/jira/browse/KAFKA-14531. The fix is committed to trunk and is quite small. If you agree, I'll merge the fix to the 3.4 branch. Thanks! David On Tue, Dec 20, 2022 at 7:53 AM David Jacot wrote: > Hi Sophie, > > We just found a

[VOTE] 3.4.0 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.4.0. Some of the major features include: * KIP-881: Rack-aware Partition Assignment for Kafka Consumers

[VOTE] 3.4.0 RC2

I do not expect that any of these test failures are blockers for the release. Thanks! David Arthur

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

Igor, thanks for the KIP! 1. For the ZK migration part, I wonder if we could avoid monitoring the directory failure ZNode while in dual-write mode. So far in the migration design, we have avoided reading anything from ZK after the initial metadata migration. We have modified the ZK brokers to use

Re: [VOTE] 3.4.0 RC2

hanks to all who voted and a big thanks to Sophie who did a vast majority of the work to prepare this release. We will continue the release process and get the announcement sent out this week. Cheers, David Arthur On Fri, Feb 3, 2023 at 12:25 PM Federico Valeri wrote: > +1 (non binding) >

[ANNOUNCE] Apache Kafka 3.4.0

, Dalibor Plavcic, Dan Stelljes, Daniel Fonai, David Arthur, David Jacot, David Karlsson, David Mao, dengziming, Derek Troy-West, Divij Vaidya, Edoardo Comar, Elkhan Eminov, Eugene Tolbakov, Federico Valeri, Francesco Nigro, FUNKYE, Greg Harris, Guozhang Wang, Hao Li, Himani Arora, Huilin Shi, Igor

[DISCUSS] Apache Kafka 2.5.0 release

Greetings! I'd like to volunteer to be release manager for the next time-based feature release which will be 2.5. If there are no objections, I'll send out the release plan in the next few days. Thanks, David Arthur

Re: [DISCUSS] Apache Kafka 2.5.0 release

gt; > Thanks for volunteering David! > > Ismael > > > On Fri, Dec 20, 2019, 7:00 PM David Arthur wrote: > > > Greetings! > > > > I'd like to volunteer to be release manager for the next time-based > feature > > release which will be 2.5. If there

Re: [VOTE] KIP-551: Expose disk read and write metrics

; > > > > > > > > > > > > I'd like to start the vote on KIP-551: Expose disk read and > > write > > > > > > > metrics. > > > > > > > > > > > > > > > > KIP: https://cwiki.apache.org/confluence/x/sotSC > > > > > > > > > > > > > > > > Discussion thread: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/cfaac4426455406abe890464a7f4ae23a5c69a39afde66fe6eb3d696%40%3Cdev.kafka.apache.org%3E > > > > > > > > > > > > > > > > cheers, > > > > > > > > Colin > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -Jose > > > > > > > > > > > > > > > > > > > > -- > Sönke Liebau > Partner > Tel. +49 179 7940878 > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > -- David Arthur

Re: [VOTE] KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies

21, 2020 at 9:35 AM David Jacot > wrote: > > > > > > > Hi all, > > > > > > > > I would like to start a vote on KIP-559: Make the Kafka Protocol > > > Friendlier > > > > with L7 Proxies. > > > > > > > > The KIP is here: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-559%3A+Make+the+Kafka+Protocol+Friendlier+with+L7+Proxies > > > > > > > > Thanks, > > > > David > > > > > > > > > > -- David Arthur

Re: [DISCUSS] Apache Kafka 2.5.0 release

to reach out to me directly or in this thread. Thanks! David On Mon, Jan 13, 2020 at 1:35 PM Colin McCabe wrote: > +1. Thanks for volunteering, David. > > best, > Colin > > On Fri, Dec 20, 2019, at 10:59, David Arthur wrote: > > Greetings! > > > > I'd li

Re: [DISCUSS] Apache Kafka 2.5.0 release

> > KIP-447 is partially implemented in 2.5 (ie, broker and > > consumer/producer changes -- the Kafka Streams parts slip) > > > > > > -Matthias > > > > > > On 1/29/20 9:05 AM, David Arthur wrote: > >> Hey everyone, just a quick update on the 2.5 relea

New release branch 2.5

our reviewer whether your PR should go to trunk or to trunk+release so they can merge accordingly. As always, please help us test the release! Thanks! David Arthur

Re: New release branch 2.5

To clarify one point: all bug fixes are welcome on the release branch until the code freeze on Feb 12th. After that, only blocker bugs should be merged to the release branch. I will not be moving JIRAs to 2.6 until after the code freeze. -David On Fri, Jan 31, 2020 at 5:21 PM David Arthur

Re: [DISCUSS] Apache Kafka 2.5.0 release

Just a friendly reminder that this Wednesday, February 12th, is the code freeze for the 2.5.0 release. After this time we will only accept blocker bugs onto the release branch. Thanks! David On Fri, Jan 31, 2020 at 5:13 PM David Arthur wrote: > Thanks! I've updated the list. > >

Re: [DISCUSS] Apache Kafka 2.5.0 release

ion > > tests add significant coverage and we've run the new tests numerous > times, > > and b) the fixes help gate the new feature even more and allow the > feature > > to be completely disabled. > > > > I'd like approve to merge https://github.co

Re: [DISCUSS] Apache Kafka 2.5.0 release

remove > KIP-467 from the list? > > Thanks, > Tu > > On Thu, Feb 20, 2020 at 7:18 AM David Arthur wrote: > > > Randall / Konstantine, > > > > Sorry for the late reply. Thanks for the fix and for the update! I see > this > > change on the 2.5 branc

Re: [DISCUSS] Apache Kafka 2.5.0 release

to merge them in? > https://github.com/apache/kafka/pull/7434 (trunk) > https://github.com/apache/kafka/pull/8127 (2.5 cherry-pick) > > Thanks, > Viktor > > > On Mon, Feb 24, 2020 at 7:22 PM David Arthur wrote: > > > Thanks, Tu. I've moved KIP-467 out of the re

Re: [DISCUSS] Apache Kafka 2.5.0 release

gt; > Randall > > On Wed, Feb 26, 2020 at 11:26 AM David Arthur wrote: > > > Viktor, the change LGTM. I've approved and merged the cherry-pick version > > into 2.5. > > > > Thanks! > > David > > > > On Tue, Feb 25, 2020 at 4:43 AM Vikt

[VOTE] 2.5.0 RC0

rg/25/documentation.html * Protocol: https://kafka.apache.org/25/protocol.html * Links to successful Jenkins builds for the 2.5 branch to follow Thanks, David Arthur

Re: [VOTE] 2.5.0 RC0

: > Thanks David! Note that support for Scala 2.11 was removed, not deprecated. > > On Fri, Feb 28, 2020, 7:47 PM David Arthur wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the first candidate for release of Apache Kafka 2.5.0. >

Re: [VOTE] 2.5.0 RC0

hanks! David On Sat, Feb 29, 2020 at 9:53 AM David Arthur wrote: > Thanks for the clarification, Ismael. Scala 2.11 is indeed removed in 2.5 > > Here is a passing build for the 2.5 branch > https://builds.apache.org/job/kafka-2.5-jdk8/50/ > > Still waiting on a passing build for

Re: Subject: [VOTE] 2.4.1 RC0

> > * Documentation: > > > https://kafka.apache.org/24/documentation.html > > > > > > * Protocol: > > > https://kafka.apache.org/24/protocol.html > > > > > > * Successful Jenkins builds for the 2.4 branch: > > > Unit/integration tests: Links to successful unit/integration test > build to > > > follow > > > System tests: > > > https://jenkins.confluent.io/job/system-test-kafka/job/2.4/152/ > > > > > > > > > Thanks, > > > Bill Bejeck > > > -- David Arthur

[VOTE] 2.5.0 RC1

https://github.com/apache/kafka/releases/tag/2.5.0-rc1 * Documentation: https://kafka.apache.org/25/documentation.html * Protocol: https://kafka.apache.org/25/protocol.html * Links to successful Jenkins builds for the 2.5 branch to follow Thanks, David Arthur

Re: [VOTE] 2.5.0 RC1

c1/javadoc/ > > > > * Tag to be voted upon (off 2.5 branch) is the 2.5.0 tag: > > https://github.com/apache/kafka/releases/tag/2.5.0-rc1 > > > > * Documentation: > > https://kafka.apache.org/25/documentation.html > > > > * Protocol: > > https://kafka.apache.org/25/protocol.html > > > > * Links to successful Jenkins builds for the 2.5 branch to follow > > > > Thanks, > > David Arthur > > > -- -David

Re: [kafka-clients] Re: [VOTE] 2.5.0 RC1

> >quickstart > >3. Verified the signatures > > > > +1 (non-binding) > > > > Thanks for running the release David! > > > > -Bill > > > > On Tue, Mar 10, 2020 at 4:01 PM David Arthur > > wrote: > > > > >

Re: [kafka-clients] Re: [VOTE] 2.5.0 RC1

ake longer to > > > figure out why the tests are not working/running > > > > > > > > > > > > Thanks. > > > > > > > > > > > > On Thu, Mar 12, 2020 at 11:21 AM Bill Bejeck < bbejeck@ gmail. com ( > > > bbej.

[VOTE] 2.5.0 RC2

ent.io/job/system-test-kafka/job/2.5/42/ -- David Arthur

Re: [Vote] KIP-569: Update DescribeConfigsResponse to include additional metadata information

; > The KIP is here: > > > https:/ / cwiki. apache. org/ confluence/ display/ KAFKA/ > > > KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field > > > ( > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field > > > ) > > > > > > > > > > > > Thanks, > > > Shailesh > > > > > > > > > > -- David Arthur

Re: [DISCUSS] KIP-704: Send a hint to broker if it is an unclean leader

Jose, thanks for the KIP! 1) Does recovering from an unclean state bump the leader epoch? 2) The name of "NewIsUnclean" field in AlterIsrRequest is a little strange. >From the description, it sounds like this will be used to by the broker to indicate to the controller that it has recovered from u

Re: [DISCUSS] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

Sounds good on the ordering, and yea I agree we can look at atomic ACL modifications in the future. Thanks! On Wed, Jan 12, 2022 at 3:53 PM Colin McCabe wrote: > Hi David, > > On Wed, Dec 15, 2021, at 07:28, David Arthur wrote: > > 5) Ok, gotcha. So will the StandardAuthoriz

Re: [VOTE] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

+1 binding, thanks Colin! On Mon, Dec 13, 2021 at 7:47 PM Colin McCabe wrote: > Hi all, > > I'd like to start the vote on KIP-801: Implement an Authorizer that stores > metadata in __cluster_metadata > > The KIP is here: https://cwiki.apache.org/confluence/x/h5KqCw > > The original DISCUSS threa

Re: [VOTE] KIP-704: Send a hint to partition leader to recover the partition

+1 binding, thanks José! On Tue, Feb 8, 2022 at 1:12 PM Colin McCabe wrote: > Hi José, > > Thanks for the KIP. +1 (binding). > > best, > Colin > > > On Mon, Feb 7, 2022, at 19:03, Luke Chen wrote: > > I'm +1 (non-binding) > > > > Thanks for the KIP! > > > > Luke > > > > On Tue, Feb 8, 2022 at 7:

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

d `metadat.max.retention.ms`, > > etc. > > I get the impression that we use `metadata` for things that are kinda > > log/disk related and `controller` for things that are not. I am > > thinking that the `metadata` prefix is more consistent with the > > current situation. What do you think Jason? > > > > > - Could we avoid letting BrokerMetadataPublisher escape into the metric > > > name? Letting the classnames leak into the metrics tends to cause > > > compatibility issues over time. > > > > Good point. For Raft we use `kafka.server:type=raft-metrics,name=...`. > > I'll change it to > > `kafka.server:type=broker-metadata-metrics,name=...`. > > > > Thanks, > > -José > > > -- David Arthur

Re: [VOTE] KIP-835: Monitor KRaft Controller Quorum Health

Thanks José, the KIP looks good to me! +1 binding -David On Mon, May 16, 2022 at 11:59 AM José Armando García Sancio wrote: > Hi all, > > I would like to start a vote for KIP-835: > https://cwiki.apache.org/confluence/x/0xShD > > Thanks you, > -José > -- David Arthur

Re: [DISCUSS] KIP-836: Addition of Information in DescribeQuorumResponse about Voter Lag

Niket, thanks for the KIP! Sorry for the late feedback on this, but I just had a quick question. The KIP indicates the two new fields will be set for voters only, however this ReplicaState struct is also used by the Observers in DescribeQuorumResponse. Will we simply fill in -1 for these values, o

Re: [VOTE] KIP-836: Addition of Information in DescribeQuorumResponse about Voter Lag

min Client? E.g. it is missing LastFetchTimestamp and > LastCaughtUpTimestamp. > > For those fields what will the admin client return when the RPC > version doesn't support those features? > > -- > -José > -- David Arthur

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

n 1, 2022 at 7:32 PM José Armando García Sancio > wrote: > > > > Thanks for the updates to the KIP. > > > > I like enumerating invariants. Is it safe to say that if > > `InControlledShutdown` is true then `Fenced` must be false. > -- David Arthur

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

ading the > software if this field has been serialized. > > Right. There is a note about this in the Compatibility section. > > Best, > David > > On Thu, Jun 2, 2022 at 6:04 PM David Arthur wrote: > > > > David, thanks for the KIP! > > > > When an

Re: [VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

gt; > > Cheers, > > David > > > > On Tue, May 31, 2022 at 9:42 PM Colin McCabe wrote: > >> > >> > We should add a comment in AlterPartitionResponse about the new error > >> > code INELIGIBLE_REPLICA. This is very important for error codes so we > >> > can track which ones are returned in which RPC version. I also wonder > >> > >> Here I'm referring to AlterPartitionResponse.json > >> > >> cheers, > >> Colin > >> > >> > > >> > > >> > On Tue, May 31, 2022, at 08:36, David Jacot wrote: > >> >> Hi folks, > >> >> > >> >> I'd like to start a vote for KIP-841: > >> >> https://cwiki.apache.org/confluence/x/phmhD. > >> >> > >> >> Thanks, > >> >> David > -- David Arthur

[DISCUSS] Apache Kafka 3.2.1 release

Hey folks, I'd like to volunteer to be the release manager for a hotfix release of the 3.2 line. This will be the first hotfix release of this line and will be version 3.2.1. We have fixed several bugs since 3.2.0 including an OAuth token refresh bug, a problem with transaction markers, and a prot

Re: [DISCUSS] Apache Kafka 3.2.1 release

for volunteering David. > > > > > > -- > > > -José > > > > > > -- David Arthur

Re: [DISCUSS] Apache Kafka 3.2.1 release

> On Sat, Jul 16, 2022 at 12:26 PM Christopher Shannon < > > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > > > HI, > > > > > > > > > > I think I found a memory leak that was introduced in 3.2.0 in a >

[VOTE] 3.2.1 RC3

-test-kafka/job/3.2/ Thanks! David Arthur

Re: [DISCUSS] KIP-859: Add Metadata Log Processing Error Related Metrics

Thanks for the KIP, Niket! I definitely agree we need to surface metadata processing errors to the operator. I have some questions about the semantics of the new metrics: 1) Does "publisher-error-count" represent the number of errors encountered only when loading the most recent image? Or will thi

Re: [VOTE] 3.2.1 RC3

I'm closing out the vote now. Thanks to everyone who voted. The RC passed with the required number of votes. I'll send out the results thread shortly. Cheers, David Arthur On Wed, Jul 27, 2022 at 11:54 AM Bill Bejeck wrote: > Hi David, > > Thanks for running the rel

[RESULTS] [VOTE] Release Kafka version 3.2.1

: None -1 Votes: None Vote Thread: https://lists.apache.org/thread/kcr2xncr762sqy79rbl83w0hzw85w775 I'll continue with the release process and send out the release announcement over the next few days. Thanks! David Arthur

[ANNOUNCE] Apache Kafka 3.2.1

release! Akhilesh Chaganti, Bruno Cadonna, Christopher L. Shannon, David Arthur, Divij Vaidya, Eugene Tolbakov, Guozhang Wang, Ismael Juma, James Hughes, Jason Gustafson, Kirk True, Lucas Bradstreet, Luke Chen, Nicolas Guyomar, Niket Goel, Okada Haruki, Shawn Wang, Viktor Somogyi-Vass, Walker Carlson

Re: [VOTE] KIP-859: Add Metadata Log Processing Error Related Metrics

Thanks, Niket. +1 binding from me -David On Mon, Aug 1, 2022 at 8:15 PM Niket Goel wrote: > > Hi all, > > I would like to start a vote on KIP-859 which adds some new metrics to KRaft > to allow for better visibility into log processing errors. > > KIP > —ttps://cwiki.apache.org/confluence/dis

Re: [ANNOUNCE] New Kafka PMC Member: A. Sophie Blee-Goldman

Well done! Congrats! -David On Wed, Aug 3, 2022 at 9:48 AM Sagar wrote: > Congratulations Sophie ! > > On Wed, 3 Aug 2022 at 6:37 PM, José Armando García Sancio > wrote: > > > Congratulations Sophie! > > -- > > -José > > > -- -David

Re: ARM/PowerPC builds

Divij, I believe the ARM node is not managed by Apache but rather access to it is donated by some external entity. I opened an INFRA ticket the last time we had issues and the Infra folks reached out to the owner of the node to resolve the issue. On Thu, Aug 4, 2022 at 7:13 AM Matthew Benedict d

Re: [ANNOUNCE] New Kafka PMC Member: Chris Egerton

Congrats, Chris! On Fri, Mar 10, 2023 at 8:27 AM Matthew Benedict de Detrich wrote: > Congrats, well deserved! > > On Thu, 9 Mar 2023, 19:12 Jun Rao, wrote: > > > Hi, Everyone, > > > > Chris Egerton has been a Kafka committer since July 2022. He has been > very > > instrumental to the community

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

;t support combined mode upgrade in this KIP is > helpful. > > Thank you. > Luke > > On Wed, Nov 30, 2022 at 8:47 AM Jun Rao wrote: > > > Hi, David, > > > > Thanks for the reply. No other comments from me. > > > > Jun > > > > On Tue

Failing ZkMigrationIntegrationTest

Hey folks, I've noticed that ZkMigrationIntegrationTest#testDualWrite has been failing a lot lately on trunk and PR builds. Earlier this week I tracked down the root cause and have been working on a fix (KAFKA-14805). Basically, what's happening is an AllocateProducerIdRequest is being handled by t

Re: Kafka Cluster WITHOUT Zookeeper

Paul, thanks for the articles. It's great to see someone digging into KRaft! It would be interesting to see your experiments against a newer version of KRaft, such as 3.4.0. Also, we hope to improve on the maximum number of partitions with the addition of https://cwiki.apache.org/confluence/display

Re: Adding reviewers with Github actions

mael Juma wrote: > > > It's a lot more convenient to have it in the commit than having to follow > > links, etc. > > > > David Arthur also wrote a script to help with this step, I believe. > > > > Ismael > > > > On Tue, Apr 18, 2023, 9:29 AM Divij Vaidya

[DISCUSS] Adding non-committers as Github collaborators

Hey folks, I stumbled across this wiki page from the infra team that describes the various features supported in the ".asf.yaml" file: https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features One section that looked particularly interesting was https://cwiki.apache.org/confluenc

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael, I have one major ZK migration improvement (KAFKA-14805) that landed in trunk this week that I'd like to merge to 3.5 (once we fix some test failures it introduced). After that, I have another PR for KAFKA-14840 which is essentially a huge bug in the ZK migration logic that needs to la

Re: [DISCUSS] Recommendations for managing long-running projects

Kirk, 1) I would check out the project management features built into GitHub, e.g., labels, milestones, and projects https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects . 2) Ultimately, the contributions will need to follow the normal PR wor

Re: [DISCUSS] Adding non-committers as Github collaborators

t to everyone's attention. > > >> > > >> There is actually also a way to allow people to re-trigger Jenkins > jobs: > > >> https://github.com/apache/kafka/pull/13578 > > >> > > >> Retriggering test is a little bit more sensitive as our resources

Re: Question: CI Pipeline in Kafka GitHub Pull Requests & Github Actions Usage

Hello, Aaron and welcome to the project! If you look towards the bottom of a Pull Request there will be a section reporting the status of the Checks for the latest commit. There is a "Details" link that takes you to the Jenkins job for that PR. The default Jenkins view is the new UI (called Blue

Re: [DISCUSS] Apache Kafka 3.5.0 release

gt; > > https://issues.apache.org/jira/browse/KAFKA-14946 > > > > https://github.com/apache/kafka/pull/13653 > > > > > > > > Thanks. > > > > Luke > > > > > > > > > > > > > > > > On Fri, Apr 28,

Re: [DISCUSS] Apache Kafka 3.5.0 release

KA-14692. Should I move all of them to the next > release? > Also KAFKA-14698 and KAFKA-14084 are somewhat related to the > migration. Should I move them too? > > Thanks, > Mickael > > On Fri, May 5, 2023 at 4:27 PM David Arthur > wrote: > > > > Hey Mickael

Re: Migration from zookeeper to kraft not working

Elena, Did you provision a KRaft controller quorum before restarting the brokers? If you don't mind, could you create a JIRA and attach the config files used for the brokers before/after the migration along with the controller configs? Please include the sequence of steps you took in the JIRA as

Re: [VOTE] 3.5.0 RC0

Mickael, we have some migration fixes on trunk, is it okay to cherry-pick these to 3.5? KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator KAFKA-15009 Handle new ACLs in KRaft snapshot during migration There is another issue KAFKA-15010 that I'm also investigating to determine the

<    1   2   3   4   5   6   7   8   9   >