Hi, Can we add sanity checks to ensure all artifacts are present when running the release? Also should we use a different folder for each RC? This would eliminate the risk of artifacts from previous RCs hiding an issue. Luke spotted the old artifacts this time but this could have been missed.
Thanks, Mickael On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cmcc...@apache.org> wrote: > > Hi David, > > Thanks for checking. I took a look and yes, I did forget to upload the Scala > 2.12 artifacts previously. The script runs gradlewAll, but when I was trying > to do the steps manually, I missed that and just ran gradlew, which only did > Scala 2.13. > > This should be fixed now! > > The scala 2.12 artifacts are here: > > https://repository.apache.org/content/repositories/orgapachekafka-1827/ > > and also here: > > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/ > > I wonder if we should change the release.py script to just run the gradlew > steps manually. The gradlewAll script seems a bit confusing (or maybe that's > just me?) > > best, > Colin > > > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote: > > Should there be Scala 2.12 artifacts in the Maven staging? > > > > I see these for 2.13: > > * > > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/ > > * > > https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/ > > > > but I don't see 2.12 for 3.9.0 release. > > > > I can confirm that the other artifacts (such as kafka-clients) do have > > today's timestamp for the 3.9.0 artifact. > > > > -David > > > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe <cmcc...@apache.org> wrote: > > > >> Thanks for checking, Luke. I did actually visit > >> https://repository.apache.org/#stagingRepositories and hit close on the > >> artifacts as the release.py script says to do. I don't know why they ended > >> up not showing up. > >> > >> Today I dropped all the maven artifacts for previous 3.9 RCs, switched to > >> java 8, and re-ran "gradlew publish" and "mvn deploy -Pgpg-signing". Then I > >> closed the two artifacts that were uploaded. You can see them here: > >> > >> https://repository.apache.org/content/repositories/orgapachekafka-1825/ > >> https://repository.apache.org/content/repositories/orgapachekafka-1826/ > >> > >> Hopefully these should now show up in Maven as expected. > >> > >> best, > >> Colin > >> > >> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote: > >> > Hi Colin, > >> > > >> > I was trying to test the RC, but I found the artifacts in Maven artifacts > >> > to be voted upon are not up-to-date. > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are on Oct. 10, > >> > but also the source code in the artifacts didn't include the latest > >> commits > >> > in RC5 here <https://github.com/apache/kafka/commits/3.9.0-rc5/>. > >> > > >> > Could you help verify it? > >> > > >> > But I confirmed the release artifacts in " > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/"; is up-to-date. > >> > I did: > >> > - checked the checksums and signatures > >> > - ran upgrade from v3.8.1 to v3.9.0 > >> > - ran quickstart with the 2.13 binaries > >> > - verify tiered storage and KIP-950 (dynamically disablement) > >> > > >> > I'll be +1 after the maven artifacts are updated. > >> > > >> > Thank you for running the release. > >> > Luke > >> > > >> > > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan > >> <jols...@confluent.io.invalid> > >> > wrote: > >> > > >> >> Hey Colin, > >> >> > >> >> Thanks for the new RC. I've tested the --feature functionality that was > >> >> missing before. Seems to be working as expected now. > >> >> As a side note, I was looking into how to configure kip-853 and it took > >> me > >> >> a moment to find the instructions in the documentation, I wonder if it > >> >> could be included in the notable changes section of the upgrade notes. > >> Not > >> >> a blocker as I believe we can still update documentation. > >> >> > >> >> I did some other spot checks on the rest of the release. +1 (binding) > >> from > >> >> me > >> >> > >> >> Justine > >> >> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe <cmcc...@apache.org> > >> wrote: > >> >> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at how we > >> can > >> >> > improve the docs here... > >> >> > > >> >> > best, > >> >> > Colin > >> >> > > >> >> > > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote: > >> >> > > Hi Chia-Ping, > >> >> > > > >> >> > > Thanks for pointing those two fields out. I retract my -1. > >> >> > > > >> >> > > Cheers, > >> >> > > Anton > >> >> > > > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai < > >> >> chia7...@apache.org > >> >> > >: > >> >> > > > >> >> > >> hi Anton > >> >> > >> > >> >> > >> Thanks for sharing your insights on Kafka serialization—it’s really > >> >> cool > >> >> > >> and interesting to me. Additionally, you inspired me to file a JIRA > >> >> > issue > >> >> > >> (KAFKA-17882) to improve the documentation. > >> >> > >> > >> >> > >> The most important aspect of Kafka is compatibility, and the > >> >> > undocumented > >> >> > >> behavior has been in place for some time [0][1]. This means > >> there’s no > >> >> > need > >> >> > >> to rush your improvement for 3.9, as we’ll need to explicitly add > >> >> > default > >> >> > >> values after applying your patch to ensure we generate the same > >> binary > >> >> > data. > >> >> > >> > >> >> > >> In short, we can improve the documentation first. In the meantime, > >> we > >> >> > can > >> >> > >> continue discussing behavior clarification for 4.0, and RM can keep > >> >> > running > >> >> > >> the RC for 3.9. Everything is on track. > >> >> > >> > >> >> > >> Best, > >> >> > >> Chia-Ping > >> >> > >> > >> >> > >> [0] > >> >> > >> > >> >> > > >> >> > >> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43 > >> >> > >> [1] > >> >> > >> > >> >> > > >> >> > >> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39 > >> >> > >> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote: > >> >> > >> > -1, refer to comments on the RC 2 thread. > >> >> > >> > > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe < > >> >> cmcc...@apache.org > >> >> > >: > >> >> > >> > > >> >> > >> > > This is the RC5 candidate for the release of Apache Kafka > >> 3.9.0. > >> >> > >> > > > >> >> > >> > > - This is a major release, the final one in the 3.x line. > >> (There > >> >> > may of > >> >> > >> > > course be other minor releases in this line, such as 3.9.1.) > >> >> > >> > > - Tiered storage will be considered production-ready in this > >> >> > release. > >> >> > >> > > - This will be the final major release to feature the > >> deprecated > >> >> > >> ZooKeeper > >> >> > >> > > mode. > >> >> > >> > > > >> >> > >> > > This release includes the following KIPs: > >> >> > >> > > - KIP-853: Support dynamically changing KRaft controller > >> >> membership > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the dump log tool > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to Kafka > >> Streams > >> >> > >> > > - KIP-1040: Improve handling of nullable values in InsertField, > >> >> > >> > > ExtractField, and other transformations > >> >> > >> > > - KIP-1031: Control offset translation in MirrorSourceConnector > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for exceptions > >> >> > >> occurring > >> >> > >> > > during processing > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and clientSecret in > >> >> > >> > > authorization header > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and TieredOffset > >> >> > >> > > - KIP-950: Tiered Storage Disablement > >> >> > >> > > - KIP-956: Tiered Storage Quotas > >> >> > >> > > > >> >> > >> > > Release notes for the 3.9.0 release: > >> >> > >> > > > >> >> > >> > >> >> > > >> >> > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html > >> >> > >> > > > >> >> > >> > > *** Please download, test and vote by October 30, 2024. > >> >> > >> > > > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to sign the > >> release: > >> >> > >> > > https://kafka.apache.org/KEYS > >> >> > >> > > > >> >> > >> > > * Release artifacts to be voted upon (source and binary): > >> >> > >> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/ > >> >> > >> > > > >> >> > >> > > * Docker release artifacts to be voted upon: > >> >> > >> > > apache/kafka:3.9.0-rc5 > >> >> > >> > > apache/kafka-native:3.9.0-rc5 > >> >> > >> > > > >> >> > >> > > * Maven artifacts to be voted upon: > >> >> > >> > > > >> >> > > >> https://repository.apache.org/content/groups/staging/org/apache/kafka/ > >> >> > >> > > > >> >> > >> > > * Javadoc: > >> >> > >> > > > >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/ > >> >> > >> > > > >> >> > >> > > * Documentation: > >> >> > >> > > https://kafka.apache.org/39/documentation.html > >> >> > >> > > > >> >> > >> > > * Protocol: > >> >> > >> > > https://kafka.apache.org/39/protocol.html > >> >> > >> > > > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc5 tag: > >> >> > >> > > https://github.com/apache/kafka/releases/tag/3.9.0-rc5 > >> >> > >> > > > >> >> > >> > > * Successful Docker Image Github Actions Pipeline for 3.9 > >> branch: > >> >> > >> > > Docker Build Test Pipeline (JVM): > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535300463 > >> >> > >> > > Docker Build Test Pipeline (Native): > >> >> > >> > > https://github.com/apache/kafka/actions/runs/11535328957 > >> >> > >> > > > >> >> > >> > > Thanks to everyone who helped with this release candidate, > >> either > >> >> by > >> >> > >> > > contributing code, testing, or documentation. > >> >> > >> > > > >> >> > >> > > Regards, > >> >> > >> > > Colin > >> >> > >> > > > >> >> > >> > > >> >> > >> > >> >> > > >> >> > >> > > > > > > -- > > David Arthur
