Hi Colin, Thanks for the update. I've ran a sanity check as Mickael suggested for the maven artifacts with the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/. I used jarcomp <https://activityworkshop.net/software/jarcomp/index.html> to verify: 1. kafka_2.13-3.9.0.jar 2. kafka-clients-3.9.0.jar 3. kafka-metadata-3.9.0.jar
I confirmed all the class files in the jars have the same content. So +1 (binding) from me. Thank you. Luke On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison <mickael.mai...@gmail.com> wrote: > Hi, > > Can we add sanity checks to ensure all artifacts are present when > running the release? > Also should we use a different folder for each RC? This would > eliminate the risk of artifacts from previous RCs hiding an issue. > Luke spotted the old artifacts this time but this could have been > missed. > > Thanks, > Mickael > > On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cmcc...@apache.org> wrote: > > > > Hi David, > > > > Thanks for checking. I took a look and yes, I did forget to upload the > Scala 2.12 artifacts previously. The script runs gradlewAll, but when I was > trying to do the steps manually, I missed that and just ran gradlew, which > only did Scala 2.13. > > > > This should be fixed now! > > > > The scala 2.12 artifacts are here: > > > > https://repository.apache.org/content/repositories/orgapachekafka-1827/ > > > > and also here: > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/ > > > > I wonder if we should change the release.py script to just run the > gradlew steps manually. The gradlewAll script seems a bit confusing (or > maybe that's just me?) > > > > best, > > Colin > > > > > > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote: > > > Should there be Scala 2.12 artifacts in the Maven staging? > > > > > > I see these for 2.13: > > > * > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/ > > > * > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/ > > > > > > but I don't see 2.12 for 3.9.0 release. > > > > > > I can confirm that the other artifacts (such as kafka-clients) do have > > > today's timestamp for the 3.9.0 artifact. > > > > > > -David > > > > > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cmcc...@apache.org> > wrote: > > > > > >> Thanks for checking, Luke. I did actually visit > > >> https://repository.apache.org/#stagingRepositories and hit close on > the > > >> artifacts as the release.py script says to do. I don't know why they > ended > > >> up not showing up. > > >> > > >> Today I dropped all the maven artifacts for previous 3.9 RCs, > switched to > > >> java 8, and re-ran "gradlew publish" and "mvn deploy -Pgpg-signing". > Then I > > >> closed the two artifacts that were uploaded. You can see them here: > > >> > > >> > https://repository.apache.org/content/repositories/orgapachekafka-1825/ > > >> > https://repository.apache.org/content/repositories/orgapachekafka-1826/ > > >> > > >> Hopefully these should now show up in Maven as expected. > > >> > > >> best, > > >> Colin > > >> > > >> > > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote: > > >> > Hi Colin, > > >> > > > >> > I was trying to test the RC, but I found the artifacts in Maven > artifacts > > >> > to be voted upon are not up-to-date. > > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on > Oct. 10, > > >> > but also the source code in the artifacts didn't include the latest > > >> commits > > >> > in RC5 here <https://github.com/apache/kafka/commits/3.9.0-rc5/>. > > >> > > > >> > Could you help verify it? > > >> > > > >> > But I confirmed the release artifacts in " > > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/"; is > up-to-date. > > >> > I did: > > >> > - checked the checksums and signatures > > >> > - ran upgrade from v3.8.1 to v3.9.0 > > >> > - ran quickstart with the 2.13 binaries > > >> > - verify tiered storage and KIP-950 (dynamically disablement) > > >> > > > >> > I'll be +1 after the maven artifacts are updated. > > >> > > > >> > Thank you for running the release. > > >> > Luke > > >> > > > >> > > > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan > > >> <jols...@confluent.io.invalid> > > >> > wrote: > > >> > > > >> >> Hey Colin, > > >> >> > > >> >> Thanks for the new RC. I've tested the --feature functionality > that was > > >> >> missing before. Seems to be working as expected now. > > >> >> As a side note, I was looking into how to configure kip-853 and it > took > > >> me > > >> >> a moment to find the instructions in the documentation, I wonder > if it > > >> >> could be included in the notable changes section of the upgrade > notes. > > >> Not > > >> >> a blocker as I believe we can still update documentation. > > >> >> > > >> >> I did some other spot checks on the rest of the release. +1 > (binding) > > >> from > > >> >> me > > >> >> > > >> >> Justine > > >> >> > > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <cmcc...@apache.org> > > >> wrote: > > >> >> > > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at > how we > > >> can > > >> >> > improve the docs here... > > >> >> > > > >> >> > best, > > >> >> > Colin > > >> >> > > > >> >> > > > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote: > > >> >> > > Hi Chia-Ping, > > >> >> > > > > >> >> > > Thanks for pointing those two fields out. I retract my -1. > > >> >> > > > > >> >> > > Cheers, > > >> >> > > Anton > > >> >> > > > > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai < > > >> >> chia7...@apache.org > > >> >> > >: > > >> >> > > > > >> >> > >> hi Anton > > >> >> > >> > > >> >> > >> Thanks for sharing your insights on Kafka serialization—it’s > really > > >> >> cool > > >> >> > >> and interesting to me. Additionally, you inspired me to file > a JIRA > > >> >> > issue > > >> >> > >> (KAFKA-17882) to improve the documentation. > > >> >> > >> > > >> >> > >> The most important aspect of Kafka is compatibility, and the > > >> >> > undocumented > > >> >> > >> behavior has been in place for some time [0][1]. This means > > >> there’s no > > >> >> > need > > >> >> > >> to rush your improvement for 3.9, as we’ll need to explicitly > add > > >> >> > default > > >> >> > >> values after applying your patch to ensure we generate the > same > > >> binary > > >> >> > data. > > >> >> > >> > > >> >> > >> In short, we can improve the documentation first. In the > meantime, > > >> we > > >> >> > can > > >> >> > >> continue discussing behavior clarification for 4.0, and RM > can keep > > >> >> > running > > >> >> > >> the RC for 3.9. Everything is on track. > > >> >> > >> > > >> >> > >> Best, > > >> >> > >> Chia-Ping > > >> >> > >> > > >> >> > >> [0] > > >> >> > >> > > >> >> > > > >> >> > > >> > https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43 > > >> >> > >> [1] > > >> >> > >> > > >> >> > > > >> >> > > >> > https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39 > > >> >> > >> > > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote: > > >> >> > >> > -1, refer to comments on the RC 2 thread. > > >> >> > >> > > > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe < > > >> >> cmcc...@apache.org > > >> >> > >: > > >> >> > >> > > > >> >> > >> > > This is the RC5 candidate for the release of Apache Kafka > > >> 3.9.0. > > >> >> > >> > > > > >> >> > >> > > - This is a major release, the final one in the 3.x line. > > >> (There > > >> >> > may of > > >> >> > >> > > course be other minor releases in this line, such as > 3.9.1.) > > >> >> > >> > > - Tiered storage will be considered production-ready in > this > > >> >> > release. > > >> >> > >> > > - This will be the final major release to feature the > > >> deprecated > > >> >> > >> ZooKeeper > > >> >> > >> > > mode. > > >> >> > >> > > > > >> >> > >> > > This release includes the following KIPs: > > >> >> > >> > > - KIP-853: Support dynamically changing KRaft controller > > >> >> membership > > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump log > tool > > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to Kafka > > >> Streams > > >> >> > >> > > - KIP-1040: Improve handling of nullable values in > InsertField, > > >> >> > >> > > ExtractField, and other transformations > > >> >> > >> > > - KIP-1031: Control offset translation in > MirrorSourceConnector > > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for > exceptions > > >> >> > >> occurring > > >> >> > >> > > during processing > > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect > > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and > clientSecret in > > >> >> > >> > > authorization header > > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and TieredOffset > > >> >> > >> > > - KIP-950: Tiered Storage Disablement > > >> >> > >> > > - KIP-956: Tiered Storage Quotas > > >> >> > >> > > > > >> >> > >> > > Release notes for the 3.9.0 release: > > >> >> > >> > > > > >> >> > >> > > >> >> > > > >> >> > > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html > > >> >> > >> > > > > >> >> > >> > > *** Please download, test and vote by October 30, 2024. > > >> >> > >> > > > > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign the > > >> release: > > >> >> > >> > > https://kafka.apache.org/KEYS > > >> >> > >> > > > > >> >> > >> > > * Release artifacts to be voted upon (source and binary): > > >> >> > >> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/ > > >> >> > >> > > > > >> >> > >> > > * Docker release artifacts to be voted upon: > > >> >> > >> > > apache/kafka:3.9.0-rc5 > > >> >> > >> > > apache/kafka-native:3.9.0-rc5 > > >> >> > >> > > > > >> >> > >> > > * Maven artifacts to be voted upon: > > >> >> > >> > > > > >> >> > > > >> > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > >> >> > >> > > > > >> >> > >> > > * Javadoc: > > >> >> > >> > > > > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/ > > >> >> > >> > > > > >> >> > >> > > * Documentation: > > >> >> > >> > > https://kafka.apache.org/39/documentation.html > > >> >> > >> > > > > >> >> > >> > > * Protocol: > > >> >> > >> > > https://kafka.apache.org/39/protocol.html > > >> >> > >> > > > > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc5 > tag: > > >> >> > >> > > https://github.com/apache/kafka/releases/tag/3.9.0-rc5 > > >> >> > >> > > > > >> >> > >> > > * Successful Docker Image Github Actions Pipeline for 3.9 > > >> branch: > > >> >> > >> > > Docker Build Test Pipeline (JVM): > > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535300463 > > >> >> > >> > > Docker Build Test Pipeline (Native): > > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535328957 > > >> >> > >> > > > > >> >> > >> > > Thanks to everyone who helped with this release candidate, > > >> either > > >> >> by > > >> >> > >> > > contributing code, testing, or documentation. > > >> >> > >> > > > > >> >> > >> > > Regards, > > >> >> > >> > > Colin > > >> >> > >> > > > > >> >> > >> > > > >> >> > >> > > >> >> > > > >> >> > > >> > > > > > > > > > -- > > > David Arthur >
