Re: [VOTE] 3.5.0 RC0

2023-06-02 Thread David Arthur
Mickael, all of our migration fixes are in. Thanks! David On Thu, Jun 1, 2023 at 6:10 PM Colin McCabe wrote: > Hi Mickael, > > Can you start the new RC tomorrow? There's one last PR we'd like to get in. > > If we can't get it in by tomorrow then let's go ahead anyway. > > Thanks very much, > Co

Re: [VOTE] 3.5.0 RC0

2023-06-01 Thread Colin McCabe
Hi Mickael, Can you start the new RC tomorrow? There's one last PR we'd like to get in. If we can't get it in by tomorrow then let's go ahead anyway. Thanks very much, Colin On Thu, Jun 1, 2023, at 14:15, Mickael Maison wrote: > Hi David, > > The PR you mentioned is merged now. Can I start wor

Re: [VOTE] 3.5.0 RC0

2023-06-01 Thread Mickael Maison
Hi David, The PR you mentioned is merged now. Can I start working on a new RC or is there more work needed? It seems the associated ticket is still open. Thanks, Mickael On Wed, 31 May 2023, 23:52 Justine Olshan, wrote: > Hey Mickael -- > This is done. Thanks! > > On Wed, May 31, 2023 at 11:24

Re: [VOTE] 3.5.0 RC0

2023-05-31 Thread Justine Olshan
Hey Mickael -- This is done. Thanks! On Wed, May 31, 2023 at 11:24 AM Mickael Maison wrote: > Hi Justine, > > Yes you can merge that into 3.5. > > Thanks, > Mickael > > On Wed, May 31, 2023 at 7:56 PM Justine Olshan > wrote: > > > > FYI -- I just saw this PR regarding a dependency for ARM. We m

Re: [VOTE] 3.5.0 RC0

2023-05-31 Thread Mickael Maison
Hi Justine, Yes you can merge that into 3.5. Thanks, Mickael On Wed, May 31, 2023 at 7:56 PM Justine Olshan wrote: > > FYI -- I just saw this PR regarding a dependency for ARM. We may want to > get this in for 3.5 as well. It should be quick. > > https://issues.apache.org/jira/browse/KAFKA-1504

Re: [VOTE] 3.5.0 RC0

2023-05-31 Thread Justine Olshan
FYI -- I just saw this PR regarding a dependency for ARM. We may want to get this in for 3.5 as well. It should be quick. https://issues.apache.org/jira/browse/KAFKA-15044 https://github.com/apache/kafka/pull/13786 Justine On Wed, May 31, 2023 at 9:28 AM David Arthur wrote: > Mickael, > > Coli

Re: [VOTE] 3.5.0 RC0

2023-05-31 Thread David Arthur
Mickael, Colin has approved my patch for KAFKA-15010, I'm just waiting on a build before merging. I'll go ahead and backport the other fixes that need to precede this one into 3.5. -David On Wed, May 31, 2023 at 11:52 AM Mickael Maison wrote: > Hi, > > The issue mentioned by Greg has been fixe

Re: [VOTE] 3.5.0 RC0

2023-05-31 Thread Mickael Maison
Hi, The issue mentioned by Greg has been fixed. As soon as the fix for KAFKA-15010 is merged I'll build another RC. Thanks, Mickael On Tue, May 30, 2023 at 10:33 AM Mickael Maison wrote: > > Hi David, > > Feel free to backport the necessary fixes to 3.5. > > Thanks, > Mickael > > On Tue, May 30

Re: [VOTE] 3.5.0 RC0

2023-05-30 Thread Mickael Maison
Hi David, Feel free to backport the necessary fixes to 3.5. Thanks, Mickael On Tue, May 30, 2023 at 10:32 AM Mickael Maison wrote: > > Hi Greg, > > Thanks for the heads up, this indeed looks like something we want in > 3.5. I've replied in the PR. > > Mickael > > On Sat, May 27, 2023 at 11:44 P

Re: [VOTE] 3.5.0 RC0

2023-05-30 Thread Mickael Maison
Hi Greg, Thanks for the heads up, this indeed looks like something we want in 3.5. I've replied in the PR. Mickael On Sat, May 27, 2023 at 11:44 PM David Arthur wrote: > > Mickael, after looking more closely, I definitely think KAFKA-15010 is a > blocker. It creates the case where the controlle

Re: [VOTE] 3.5.0 RC0

2023-05-27 Thread David Arthur
Mickael, after looking more closely, I definitely think KAFKA-15010 is a blocker. It creates the case where the controller can totally miss a metadata update and not write it back to ZK. Since things like dynamic configs and ACLs are only read from ZK by the ZK brokers, we could have significant pr

Re: [VOTE] 3.5.0 RC0

2023-05-25 Thread Greg Harris
Hey all, A contributor just pointed out a small but noticeable flaw in the implementation of KIP-581 https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value which is planned for this release. Impact: the feature works for root values in a r

Re: [VOTE] 3.5.0 RC0

2023-05-24 Thread Ismael Juma
I agree the migration should be functional - it wasn't obvious if the migration issues are edge cases or not. If they are edge cases, I think 3.5.1 would be fine given the preview status. I understand that a new RC is needed, but that doesn't mean we should let everything in. Each change carries s

Re: [VOTE] 3.5.0 RC0

2023-05-24 Thread Colin McCabe
Hi Ismael, I do personally think they are blockers. But just as importantly, KAFKA-14996 is not related to zk to kraft migration, and is a blocker. So we will need another RC in any case. best, Colin On Wed, May 24, 2023, at 11:50, Ismael Juma wrote: > Are all these blockers? For example, zk t

Re: [VOTE] 3.5.0 RC0

2023-05-24 Thread Josep Prat
Hi there, Is the plan described in KIP-833[1] still valid? In there it states that 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the path to migrate to Kraft should be somewhat functional (in my opinion). If we don't want to deprecate Zookeeper in 3.5.0, then I share Ismael's opini

Re: [VOTE] 3.5.0 RC0

2023-05-24 Thread Ismael Juma
Are all these blockers? For example, zk to kraft migration are is still in preview - can we fix some of these in 3.5.1? Ismael On Wed, May 24, 2023, 10:22 AM Colin McCabe wrote: > Hi Mickael, > > Thanks for putting together this RC. Unfortunately, we've identified > several blocker issues in th

Re: [VOTE] 3.5.0 RC0

2023-05-24 Thread Colin McCabe
Hi Mickael, Thanks for putting together this RC. Unfortunately, we've identified several blocker issues in this release candidate. KAFKA-15009: New ACLs are not written to ZK during migration KAFKA-15007: MV is not set correctly in the MetadataPropagator in migration. KAFKA-15004: Topic config c

Re: [VOTE] 3.5.0 RC0

2023-05-24 Thread Mickael Maison
Hi David, We're already quite a bit behind schedule. If you think these fixes are really important and can be ready in the next couple of days, I'm open to backport them and build another release candidate. Let me know once you've investigated the severity of KAFKA-15010. Thanks, Mickael On Tue

Re: [VOTE] 3.5.0 RC0

2023-05-23 Thread David Arthur
Mickael, we have some migration fixes on trunk, is it okay to cherry-pick these to 3.5? KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator KAFKA-15009 Handle new ACLs in KRaft snapshot during migration There is another issue KAFKA-15010 that I'm also investigating to determine the

Re: [VOTE] 3.5.0 RC0

2023-05-23 Thread Mickael Maison
Hi Christo, Yes this is expected. This happens when nested fields also accept optional tagged fields. The tables list all fields, so they may include _tagged_fields multiple times. Clearly the layout of this page could be improved, if you have ideas how to describe the protocol in a better way, fe

Re: [VOTE] 3.5.0 RC0

2023-05-23 Thread Mickael Maison
Hi Josep, Good catch! I opened a PR to fix this: https://github.com/apache/kafka-site/pull/514 Thanks, Mickael On Tue, May 23, 2023 at 3:36 PM Christo Lolov wrote: > > Hey Mickael! > > I am giving a +1 (non-binding) for this candidate release. > > * Built from the binary tar.gz source with Jav

Re: [VOTE] 3.5.0 RC0

2023-05-23 Thread Christo Lolov
Hey Mickael! I am giving a +1 (non-binding) for this candidate release. * Built from the binary tar.gz source with Java 17 and Scala 2.13 on Intel (m5.4xlarge) and ARM (m6g.4xlarge) machines. * Ran unit and integration tests on Intel and ARM machines. * Ran the Quickstart in both Zookeeper and KR

Re: [VOTE] 3.5.0 RC0

2023-05-23 Thread Josep Prat
Hi Mickael, I just wanted to point out that I think the documentation you recently merged on Kafka site regarding the 3.5.0 version has a problem when it states the version number and the sub-menu that links to previous versions. Left a comment here: https://github.com/apache/kafka-site/pull/513#pu

Re: [VOTE] 3.5.0 RC0

2023-05-23 Thread Josep Prat
Hi Mickael, I can +1 this candidate. I verified the following: - Built from source with Java 17 and Scala 2.13 - Signatures and hashes of the artifacts generated - Navigated through Javadoc including links to JDK classes - Run the unit tests - Run integration tests - Run the quickstart in KRaft an

[VOTE] 3.5.0 RC0

2023-05-22 Thread Mickael Maison
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.5.0. Some of the major features include: - KIP-710: Full support for distributed mode in dedicated MirrorMaker 2.0 clusters - KIP-881: Rack-aware Partition Assignment for Kafka Consumers