Hi Mickael, Thank you for running the release! How do you feel about including https://issues.apache.org/jira/browse/KAFKA-19458 as well in v4.1.0?
Regards, Gaurav > On 18 Jul 2025, at 13:54, Mickael Maison <mickael.mai...@gmail.com> wrote: > > Hi Luke, > > I don't think Kafka uses the ClassUtils.getClass() method from Commons > so it shouldn't be impacted by this CVE. > That said, I'm planning to do another RC on Monday, so if you can get > the dependency update merged in 4.1 by then I'll include it. > > Thanks, > Mickael > > On Fri, Jul 18, 2025 at 2:42 PM Luke Chen <show...@gmail.com> wrote: >> >> Hi Mickael, >> >> Since we're going to have a new RC, do you think this KAFKA-19520 >> <https://issues.apache.org/jira/browse/KAFKA-19520> should be included in >> v4.1.0? >> >> Thanks. >> Luke >> >> On Thu, Jul 17, 2025 at 11:29 PM Lucas Brutschy >> <lbruts...@confluent.io.invalid> wrote: >> >>> Hi Mickael, >>> >>> done. >>> >>> Cheers, >>> Lucas >>> >>> On Thu, Jul 17, 2025 at 2:18 PM Mickael Maison <mickael.mai...@gmail.com> >>> wrote: >>>> >>>> Hi Lucas, >>>> >>>> Ok, let me know once you're done. >>>> >>>> Thanks, >>>> Mickael >>>> >>>> On Thu, Jul 17, 2025 at 11:08 AM Lucas Brutschy >>>> <lbruts...@confluent.io.invalid> wrote: >>>>> >>>>> Hi Mickael, >>>>> >>>>> we have been seeing a race condition happening in the initialization >>>>> of the state updater. We'd like to revert 4d6cf3efef >>>>> which changed the initialization of the state updater and is anyway >>>>> linked to KAFKA-18913 that was moved to 4.2. >>>>> >>>>> Let me know if you accept the change. >>>>> >>>>> Cheers, >>>>> Lucas >>>>> >>>>> On Wed, Jul 16, 2025 at 6:47 PM Paolo Patierno < >>> paolo.patie...@gmail.com> wrote: >>>>>> >>>>>> Hi Jun, >>>>>> oh yes sorry!! It's NOW fixed :-D Thanks! >>>>>> >>>>>> On Wed, 16 Jul 2025 at 18:38, Jun Rao <j...@confluent.io.invalid> >>> wrote: >>>>>> >>>>>>> Hi, Paolo, >>>>>>> >>>>>>> Thanks for the reply. I guess that you mean the metadata version >>> issue is >>>>>>> NOW fixed, instead of NOT fixed? >>>>>>> >>>>>>> Jun >>>>>>> >>>>>>> On Wed, Jul 16, 2025 at 8:57 AM Paolo Patierno < >>> paolo.patie...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> I already did on the PR but let me confirm here as well that the >>> metadata >>>>>>>> version issues is not fixed, I was able to test it locally >>> within our >>>>>>>> Strimzi unit tests. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Paolo >>>>>>>> >>>>>>>> Twitter : @ppatierno <http://twitter.com/ppatierno> >>>>>>>> Linkedin : paolopatierno < >>> http://it.linkedin.com/in/paolopatierno> >>>>>>>> GitHub : ppatierno <https://github.com/ppatierno> >>>>>>>> >>>>>>>> On Wed, 16 Jul 2025, 17:43 Mickael Maison, < >>> mickael.mai...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Chia-Ping, >>>>>>>>> >>>>>>>>> Sure, thanks >>>>>>>>> >>>>>>>>> Mickael >>>>>>>>> >>>>>>>>> On Wed, Jul 16, 2025 at 4:07 PM Chia-Ping Tsai < >>> chia7...@apache.org> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> hi Mickael >>>>>>>>>> >>>>>>>>>> I'm going to merge the patch for >>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-19427. The bug >>> could lead >>>>>>> to >>>>>>>>> an OOM in 4.x. The risk is low since the patch is small. >>> Hence, I plan >>>>>>> to >>>>>>>>> backport it to 4.1.0 if you agree to accept it. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Chia-Ping >>>>>>>>>> >>>>>>>>>> On 2025/07/15 08:43:40 Mickael Maison wrote: >>>>>>>>>>> Hi Bill, >>>>>>>>>>> >>>>>>>>>>> Yes you can go ahead and backport that to 4.1. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Mickael >>>>>>>>>>> >>>>>>>>>>> On Mon, Jul 14, 2025 at 8:42 PM Bill Bejeck < >>> bbej...@gmail.com> >>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Mickael, >>>>>>>>>>>> >>>>>>>>>>>> I've found another blocker as well KAFKA-19504 >>>>>>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-19504>, >>> and we >>>>>>> have a >>>>>>>>> PR ( >>>>>>>>>>>> https://github.com/apache/kafka/pull/20166). >>>>>>>>>>>> While this is not a blocker in the strict sense, it's >>> severe >>>>>>> enough >>>>>>>>> that we >>>>>>>>>>>> shouldn't let this ship as is. It's minor change with >>> very low >>>>>>>> risk. >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jul 14, 2025 at 2:09 PM Jun Rao >>> <j...@confluent.io.invalid >>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, Paolo, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for identifying the issue. We now have a PR ( >>>>>>>>>>>>> https://github.com/apache/kafka/pull/20137) to bump >>> up the >>>>>>>> latest >>>>>>>>>>>>> production metadata version. >>>>>>>>>>>>> >>>>>>>>>>>>> Jun >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Jul 14, 2025 at 1:58 AM Paolo Patierno < >>>>>>>>> paolo.patie...@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> I think I found something which could be a blocker >>> for the >>>>>>>>> release. >>>>>>>>>>>>>> I was adding the support for Apache Kafka 4.1.0 to >>> Strimzi >>>>>>> and >>>>>>>>> running >>>>>>>>>>>>> our >>>>>>>>>>>>>> tests but some of them are failing with the >>> following error. >>>>>>>>>>>>>> >>>>>>>>>>>>>> java.lang.IllegalArgumentException: Version 4.1 is >>> not a >>>>>>> valid >>>>>>>>> version. >>>>>>>>>>>>> The >>>>>>>>>>>>>> minimum version is 3.3-IV3 and the maximum version >>> is 4.2-IV1 >>>>>>>>>>>>>> at >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> org.apache.kafka.server.common.MetadataVersion.lambda$fromVersionString$0(MetadataVersion.java:356) >>>>>>>>>>>>>> at >>>>>>>>> java.base/java.util.Optional.orElseThrow(Optional.java:403) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> org.apache.kafka.server.common.MetadataVersion.fromVersionString(MetadataVersion.java:354) >>>>>>>>>>>>>> >>>>>>>>>>>>>> In these tests we use the MetadataVersion class to >>> verify the >>>>>>>>> validity of >>>>>>>>>>>>>> metadata but it fails when passing something like >>> "4.1" >>>>>>> (while >>>>>>>>> they work >>>>>>>>>>>>>> just fine with all previous releases, i.e. "4.0", >>> "3.9" and >>>>>>> so >>>>>>>>> on). >>>>>>>>>>>>>> >>>>>>>>>>>>>> The exception, thrown by the fromVersionString >>> method, seems >>>>>>> to >>>>>>>>> be >>>>>>>>>>>>> related >>>>>>>>>>>>>> to "4.1" missing as key entry within the >>> IBP_VERSIONS map and >>>>>>>>> such entry >>>>>>>>>>>>>> seems to be missing because, if you go through the >>> static >>>>>>>>> initialization >>>>>>>>>>>>> of >>>>>>>>>>>>>> the IBP_VERSIONS map, the "4.1" entry is not added >>> due to the >>>>>>>>>>>>>> metadataVersion.isProduction() which results false >>> for "4.1" >>>>>>>>> because the >>>>>>>>>>>>>> LATEST_PRODUCTION constant is defined as IBP_4_0_IV3 >>> while I >>>>>>>>> would assume >>>>>>>>>>>>>> should be something like IBP_4_1_IV1 instead. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think this issue wasn't caught during Kafka >>> upstream >>>>>>> testing >>>>>>>>> because >>>>>>>>>>>>> the >>>>>>>>>>>>>> MetadataVersionTest.testFromVersionString is missing >>> the test >>>>>>>>> for "4.1" >>>>>>>>>>>>> (as >>>>>>>>>>>>>> we have for all other versions). So it's missing >>> something >>>>>>> like >>>>>>>>>>>>>> >>>>>>>>>>>>>> assertEquals(IBP_4_1_IV1, >>>>>>>>> MetadataVersion.fromVersionString("4.1")); >>>>>>>>>>>>>> assertEquals(IBP_4_1_IV0, >>>>>>>>>>>>>> MetadataVersion.fromVersionString("4.1-IV0")); >>>>>>>>>>>>>> assertEquals(IBP_4_1_IV1, >>>>>>>>>>>>>> MetadataVersion.fromVersionString("4.1-IV1")); >>>>>>>>>>>>>> >>>>>>>>>>>>>> Let's add that this issue seems to break the storage >>> tool >>>>>>> where >>>>>>>>> you can >>>>>>>>>>>>>> pass the --release-version, but if you run it by >>> passing >>>>>>> "4.1" >>>>>>>>> you get >>>>>>>>>>>>>> exactly the same exception. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> bin/kafka-storage.sh format --standalone -t >>>>>>>>> kEzc4vk3TIKhCQKsh40klQ -c >>>>>>>>>>>>>> config/server.properties --release-version 4.1 >>>>>>>>>>>>>> Exception in thread "main" >>>>>>> java.lang.IllegalArgumentException: >>>>>>>>> Version >>>>>>>>>>>>> 4.1 >>>>>>>>>>>>>> is not a valid version. The minimum version is >>> 3.3-IV3 and >>>>>>> the >>>>>>>>> maximum >>>>>>>>>>>>>> version is 4.2-IV1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Finally, I would add that the error message itself is >>>>>>>> misleading >>>>>>>>> as it >>>>>>>>>>>>>> mentions 4.2-IV1 as maximum version which would not >>> exist at >>>>>>>> all >>>>>>>>> if you >>>>>>>>>>>>> are >>>>>>>>>>>>>> dealing with a 4.1 release. This misleading message >>> should be >>>>>>>>> related to >>>>>>>>>>>>>> MetadataVersion.latestTesting() method which is >>> getting last >>>>>>>>> value from >>>>>>>>>>>>>> VERSIONS which is filled with all MetadataVersion >>> enums >>>>>>>>> including the 4.2 >>>>>>>>>>>>>> placeholders (which AFAIU is actually a new >>> practice). All of >>>>>>>>> that has >>>>>>>>>>>>> also >>>>>>>>>>>>>> an impact on the ClusterTest annotation class as >>> well as >>>>>>>>>>>>> FeatureCommandTest >>>>>>>>>>>>>> messages to be assert. So that 4.2 seems to be there >>> on >>>>>>> purpose >>>>>>>>> but not >>>>>>>>>>>>>> sure why. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would be happy to contribute about the first issue >>> which >>>>>>>>> raises the >>>>>>>>>>>>>> exception. Not sure about the misleading error >>> message >>>>>>> because >>>>>>>>> of the >>>>>>>>>>>>> usage >>>>>>>>>>>>>> of the 4.2 placeholders within the VERSIONS >>> collection. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> Paolo Patierno >>>>>>>>>>>>>> >>>>>>>>>>>>>> ---------- Forwarded message --------- >>>>>>>>>>>>>> From: Mickael Maison <mickael.mai...@gmail.com> >>>>>>>>>>>>>> Date: Fri, Jul 11, 2025 at 11:53 PM >>>>>>>>>>>>>> Subject: [VOTE] 4.1.0 RC0 >>>>>>>>>>>>>> To: dev <dev@kafka.apache.org>, Users < >>>>>>> us...@kafka.apache.org >>>>>>>>> , >>>>>>>>>>>>>> kafka-clients <kafka-clie...@googlegroups.com> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hello Kafka users, developers and client-developers, >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is the first candidate for release of Apache >>> Kafka >>>>>>> 4.1.0. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The release brings many new features and >>> improvements: >>>>>>>>>>>>>> - Mechanism for plugin to register metrics >>>>>>>>>>>>>> - Allow running multiple versions of connector >>> plugins >>>>>>>>>>>>>> - Queue are now in preview access >>>>>>>>>>>>>> - New Streams rebalance protocol in early access >>>>>>>>>>>>>> - Support for OAuth jwt-bearer grant type >>>>>>>>>>>>>> - Improved metadata replication >>>>>>>>>>>>>> >>>>>>>>>>>>>> Release notes for the 4.1.0 release: >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/RELEASE_NOTES.html >>>>>>>>>>>>>> >>>>>>>>>>>>>> *** Please download, test and vote by Monday July 21 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Kafka's KEYS file containing PGP keys we use to sign >>> the >>>>>>>> release: >>>>>>>>>>>>>> https://kafka.apache.org/KEYS >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Release artifacts to be voted upon (source and >>> binary): >>>>>>>>>>>>>> >>> https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Docker release artifacts to be voted upon: >>>>>>>>>>>>>> apache/kafka:4.1.0-rc0 >>>>>>>>>>>>>> apache/kafka-native:4.1.0-rc0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Maven artifacts to be voted upon: >>>>>>>>>>>>>> >>>>>>>>> >>> https://repository.apache.org/content/groups/staging/org/apache/kafka/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Javadoc: >>>>>>>>>>>>>> >>>>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/javadoc/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Tag to be voted upon (off 4.1 branch) is the 4.1.0 >>> tag: >>>>>>>>>>>>>> >>> https://github.com/apache/kafka/releases/tag/4.1.0-rc0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Documentation: >>>>>>>>>>>>>> https://kafka.apache.org/41/documentation.html >>>>>>>>>>>>>> PR (https://github.com/apache/kafka-site/pull/702) >>> is not >>>>>>>>> merged yet >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Protocol: >>>>>>>>>>>>>> https://kafka.apache.org/41/protocol.html >>>>>>>>>>>>>> PR (https://github.com/apache/kafka-site/pull/702) >>> is not >>>>>>>>> merged yet >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Successful CI builds for the 4.1 branch: >>>>>>>>>>>>>> Unit/integration tests: >>>>>>>>>>>>>> >>> https://github.com/apache/kafka/actions/runs/16198079614 >>>>>>>>>>>>>> * System tests results: >>>>>>>>>>>>>> - core: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> https://drive.google.com/file/d/1JLpFBFbPV1N6cdh9EA9TGw9S5SZwkIpv/view?usp=drive_link >>>>>>>>>>>>>> - non-core: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> https://drive.google.com/file/d/15jCnTS-dssSHauW7uWxroiupkWaeUK7A/view?usp=drive_link >>>>>>>>>>>>>> I got 2 failures in the system tests: >>>>>>>>>>>>>> - >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> 'tests/kafkatest/tests/client/consumer_test.py::OffsetValidationTest.test_broker_rolling_bounce@ >>>>>>>>>>>>>> >>>>>>>> {"metadata_quorum":"ISOLATED_KRAFT","group_protocol":"classic"}' >>>>>>>>>>>>>> - >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> 'tests/kafkatest/tests/core/network_degrade_test.py::NetworkDegradeTest.test_rate@ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>> {"task_name":"rate-1000-latency-50","device_name":"eth0","latency_ms":50,"rate_limit_kbit":1000000,"metadata_quorum":"COMBINED_KRAFT"}' >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Successful Docker Image Github Actions Pipeline >>> for 4.1 >>>>>>>> branch: >>>>>>>>>>>>>> Docker Build Test Pipeline (JVM): >>>>>>>>>>>>>> >>> https://github.com/apache/kafka/actions/runs/16172687407/ >>>>>>>>>>>>>> Docker Build Test Pipeline (Native): >>>>>>>>>>>>>> >>> https://github.com/apache/kafka/actions/runs/16220242161 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Note: >>>>>>>>>>>>>> As described in the release thread, the native image >>>>>>>>> configuration >>>>>>>>>>>>>> files had to be updated. I'm not able to run the >>> full system >>>>>>>> test >>>>>>>>>>>>>> suite with the GraalVM agent to generate the new >>>>>>> configuration >>>>>>>>> files. >>>>>>>>>>>>>> Luke has kindly run a few commands to produce a fix >>> to the >>>>>>>> config >>>>>>>>>>>>>> files. I validated that the kafka-native:4.1.0-rc0 >>> image >>>>>>> passes >>>>>>>>> the >>>>>>>>>>>>>> quickstart (clients, connect and streams), but as >>> that image >>>>>>> is >>>>>>>>> only >>>>>>>>>>>>>> for development, I've not tested it any further. >>>>>>>>>>>>>> Also all the other release artifacts were generated >>> before >>>>>>> the >>>>>>>>> fix to >>>>>>>>>>>>>> the native image configuration files was merged. As >>> these >>>>>>> files >>>>>>>>> are >>>>>>>>>>>>>> not part of any of the other release artifacts and >>> as this is >>>>>>>>> rc0, I >>>>>>>>>>>>>> deemed that acceptable. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Mickael >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Paolo Patierno >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Senior Principal Software Engineer @ Red Hat**CNCF >>>>>>> Ambassador* >>>>>>>>>>>>>> *Microsoft MVP on **Azure* >>>>>>>>>>>>>> >>>>>>>>>>>>>> Twitter : @ppatierno <http://twitter.com/ppatierno> >>>>>>>>>>>>>> Linkedin : paolopatierno < >>>>>>>>> http://it.linkedin.com/in/paolopatierno> >>>>>>>>>>>>>> GitHub : ppatierno <https://github.com/ppatierno> >>>>>>>>>>>>>> Paolo Patierno >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Senior Principal Software Engineer @ Red Hat**CNCF >>>>>>> Ambassador* >>>>>>>>>>>>>> *Microsoft MVP on **Azure* >>>>>>>>>>>>>> >>>>>>>>>>>>>> Twitter : @ppatierno <http://twitter.com/ppatierno> >>>>>>>>>>>>>> Linkedin : paolopatierno < >>>>>>>>> http://it.linkedin.com/in/paolopatierno> >>>>>>>>>>>>>> GitHub : ppatierno <https://github.com/ppatierno> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Paolo Patierno >>>>>> >>>>>> *Senior Principal Software Engineer @ Red Hat**CNCF Ambassador* >>>>>> *Microsoft MVP on **Azure* >>>>>> >>>>>> Twitter : @ppatierno <http://twitter.com/ppatierno> >>>>>> Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno> >>>>>> GitHub : ppatierno <https://github.com/ppatierno> >>>
