Thanks, Josep. best, Colin
On Wed, Nov 6, 2024, at 23:58, Josep Prat wrote: > Hi Colin, > I see that you made 3.9 the default docs version in Kafka-Site. However, as > your doc changes were pushed before the 3.8.1 release, now the upgrade.html > page doesn't mention 3.8.1 at all. I'll update the site. > > On Wed, Nov 6, 2024 at 4:48 PM Viktor Somogyi-Vass > <viktor.somo...@cloudera.com.invalid> wrote: > >> Ok, sorry for the noise. It seems like my setup was incorrectly configured, >> one controller had an incorrect voters config and that caused the issue. >> Retested it and it works now. >> But given this, I might as well just give a +1. :) >> >> On Wed, Nov 6, 2024 at 2:53 PM Viktor Somogyi-Vass < >> viktor.somo...@cloudera.com> wrote: >> >> > Hey All, >> > >> > It seems like I ran into an issue. While trying to do the migration (3.9 >> > ZK -> 3.9 KRaft), at the very last step the whole cluster (except the >> > controller I just restarted) fails with a fatal error where a controller >> (I >> > think) requests a truncation that is below the HWM. >> > I tried to capture everything but with the configs and the steps I >> > described it can be reproduced every time. >> > >> > I started looking into it, but since I'm completely unfamiliar with many >> > parts of KRaft, I'd be happy if someone else more familiar with it could >> > look at it. Especially if I didn't follow the migration procedure well >> (but >> > I think I did so). >> > All the info captured here: >> > https://issues.apache.org/jira/browse/KAFKA-17950 >> > >> > Viktor >> > >> > >> > On Fri, Nov 1, 2024 at 11:00 PM Colin McCabe <cmcc...@apache.org> wrote: >> > >> >> Argh. That should read "5 +1 votes". :) >> >> >> >> Thanks, everyone. >> >> Colin >> >> >> >> >> >> On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote: >> >> > Hi all, >> >> > >> >> > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes. >> >> > >> >> > +1 votes >> >> > PMC Members: >> >> > * Justine Olshan >> >> > * Bill Bejeck >> >> > * David Arthur >> >> > * Luke Chen >> >> > >> >> > Community: >> >> > * Federico Valeri >> >> > >> >> > 0 votes >> >> > * No votes >> >> > >> >> > -1 votes >> >> > * No votes >> >> > >> >> > Vote thread: >> >> > https://lists.apache.org/thread/og8l24z8oj87x0zsoqddfxo54qsj4d1o >> >> > >> >> > I'll continue with the release process and the release announcement >> >> > will follow in the next few days. >> >> > >> >> > best, >> >> > Colin >> >> > >> >> > >> >> > On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote: >> >> >> Hi Colin, >> >> >> >> >> >> Thanks for the update. >> >> >> I've ran a sanity check as Mickael suggested for the maven artifacts >> >> with >> >> >> the one in https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/. >> >> >> I used jarcomp < >> >> https://activityworkshop.net/software/jarcomp/index.html> >> >> >> to verify: >> >> >> 1. kafka_2.13-3.9.0.jar >> >> >> 2. kafka-clients-3.9.0.jar >> >> >> 3. kafka-metadata-3.9.0.jar >> >> >> >> >> >> I confirmed all the class files in the jars have the same content. >> >> >> >> >> >> So +1 (binding) from me. >> >> >> >> >> >> Thank you. >> >> >> Luke >> >> >> >> >> >> On Fri, Nov 1, 2024 at 1:08 AM Mickael Maison < >> >> mickael.mai...@gmail.com> >> >> >> wrote: >> >> >> >> >> >>> Hi, >> >> >>> >> >> >>> Can we add sanity checks to ensure all artifacts are present when >> >> >>> running the release? >> >> >>> Also should we use a different folder for each RC? This would >> >> >>> eliminate the risk of artifacts from previous RCs hiding an issue. >> >> >>> Luke spotted the old artifacts this time but this could have been >> >> >>> missed. >> >> >>> >> >> >>> Thanks, >> >> >>> Mickael >> >> >>> >> >> >>> On Thu, Oct 31, 2024 at 5:55 PM Colin McCabe <cmcc...@apache.org> >> >> wrote: >> >> >>> > >> >> >>> > Hi David, >> >> >>> > >> >> >>> > Thanks for checking. I took a look and yes, I did forget to upload >> >> the >> >> >>> Scala 2.12 artifacts previously. The script runs gradlewAll, but >> when >> >> I was >> >> >>> trying to do the steps manually, I missed that and just ran gradlew, >> >> which >> >> >>> only did Scala 2.13. >> >> >>> > >> >> >>> > This should be fixed now! >> >> >>> > >> >> >>> > The scala 2.12 artifacts are here: >> >> >>> > >> >> >>> > >> >> https://repository.apache.org/content/repositories/orgapachekafka-1827/ >> >> >>> > >> >> >>> > and also here: >> >> >>> > >> >> >>> > >> >> >>> >> >> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.12/3.9.0/ >> >> >>> > >> >> >>> > I wonder if we should change the release.py script to just run the >> >> >>> gradlew steps manually. The gradlewAll script seems a bit confusing >> >> (or >> >> >>> maybe that's just me?) >> >> >>> > >> >> >>> > best, >> >> >>> > Colin >> >> >>> > >> >> >>> > >> >> >>> > On Wed, Oct 30, 2024, at 11:02, David Arthur wrote: >> >> >>> > > Should there be Scala 2.12 artifacts in the Maven staging? >> >> >>> > > >> >> >>> > > I see these for 2.13: >> >> >>> > > * >> >> >>> > > >> >> >>> >> >> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.13/3.9.0/ >> >> >>> > > * >> >> >>> > > >> >> >>> >> >> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-streams-scala_2.13/3.9.0/ >> >> >>> > > >> >> >>> > > but I don't see 2.12 for 3.9.0 release. >> >> >>> > > >> >> >>> > > I can confirm that the other artifacts (such as kafka-clients) >> do >> >> have >> >> >>> > > today's timestamp for the 3.9.0 artifact. >> >> >>> > > >> >> >>> > > -David >> >> >>> > > >> >> >>> > > On Wed, Oct 30, 2024 at 12:36 PM Colin McCabe < >> cmcc...@apache.org >> >> > >> >> >>> wrote: >> >> >>> > > >> >> >>> > >> Thanks for checking, Luke. I did actually visit >> >> >>> > >> https://repository.apache.org/#stagingRepositories and hit >> >> close on >> >> >>> the >> >> >>> > >> artifacts as the release.py script says to do. I don't know why >> >> they >> >> >>> ended >> >> >>> > >> up not showing up. >> >> >>> > >> >> >> >>> > >> Today I dropped all the maven artifacts for previous 3.9 RCs, >> >> >>> switched to >> >> >>> > >> java 8, and re-ran "gradlew publish" and "mvn deploy >> >> -Pgpg-signing". >> >> >>> Then I >> >> >>> > >> closed the two artifacts that were uploaded. You can see them >> >> here: >> >> >>> > >> >> >> >>> > >> >> >> >>> >> >> https://repository.apache.org/content/repositories/orgapachekafka-1825/ >> >> >>> > >> >> >> >>> >> >> https://repository.apache.org/content/repositories/orgapachekafka-1826/ >> >> >>> > >> >> >> >>> > >> Hopefully these should now show up in Maven as expected. >> >> >>> > >> >> >> >>> > >> best, >> >> >>> > >> Colin >> >> >>> > >> >> >> >>> > >> >> >> >>> > >> On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote: >> >> >>> > >> > Hi Colin, >> >> >>> > >> > >> >> >>> > >> > I was trying to test the RC, but I found the artifacts in >> Maven >> >> >>> artifacts >> >> >>> > >> > to be voted upon are not up-to-date. >> >> >>> > >> > Not only the "Last Modified" time in the 3.9.0 artifacts are >> on >> >> >>> Oct. 10, >> >> >>> > >> > but also the source code in the artifacts didn't include the >> >> latest >> >> >>> > >> commits >> >> >>> > >> > in RC5 here < >> >> https://github.com/apache/kafka/commits/3.9.0-rc5/>. >> >> >>> > >> > >> >> >>> > >> > Could you help verify it? >> >> >>> > >> > >> >> >>> > >> > But I confirmed the release artifacts in " >> >> >>> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/"; is >> >> >>> up-to-date. >> >> >>> > >> > I did: >> >> >>> > >> > - checked the checksums and signatures >> >> >>> > >> > - ran upgrade from v3.8.1 to v3.9.0 >> >> >>> > >> > - ran quickstart with the 2.13 binaries >> >> >>> > >> > - verify tiered storage and KIP-950 (dynamically disablement) >> >> >>> > >> > >> >> >>> > >> > I'll be +1 after the maven artifacts are updated. >> >> >>> > >> > >> >> >>> > >> > Thank you for running the release. >> >> >>> > >> > Luke >> >> >>> > >> > >> >> >>> > >> > >> >> >>> > >> > On Wed, Oct 30, 2024 at 5:40 AM Justine Olshan >> >> >>> > >> <jols...@confluent.io.invalid> >> >> >>> > >> > wrote: >> >> >>> > >> > >> >> >>> > >> >> Hey Colin, >> >> >>> > >> >> >> >> >>> > >> >> Thanks for the new RC. I've tested the --feature >> functionality >> >> >>> that was >> >> >>> > >> >> missing before. Seems to be working as expected now. >> >> >>> > >> >> As a side note, I was looking into how to configure kip-853 >> >> and it >> >> >>> took >> >> >>> > >> me >> >> >>> > >> >> a moment to find the instructions in the documentation, I >> >> wonder >> >> >>> if it >> >> >>> > >> >> could be included in the notable changes section of the >> >> upgrade >> >> >>> notes. >> >> >>> > >> Not >> >> >>> > >> >> a blocker as I believe we can still update documentation. >> >> >>> > >> >> >> >> >>> > >> >> I did some other spot checks on the rest of the release. +1 >> >> >>> (binding) >> >> >>> > >> from >> >> >>> > >> >> me >> >> >>> > >> >> >> >> >>> > >> >> Justine >> >> >>> > >> >> >> >> >>> > >> >> On Tue, Oct 29, 2024 at 12:45 PM Colin McCabe < >> >> cmcc...@apache.org> >> >> >>> > >> wrote: >> >> >>> > >> >> >> >> >>> > >> >> > Thanks, Anton. And thanks Chia-Ping Tsai for taking a look >> >> at >> >> >>> how we >> >> >>> > >> can >> >> >>> > >> >> > improve the docs here... >> >> >>> > >> >> > >> >> >>> > >> >> > best, >> >> >>> > >> >> > Colin >> >> >>> > >> >> > >> >> >>> > >> >> > >> >> >>> > >> >> > On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote: >> >> >>> > >> >> > > Hi Chia-Ping, >> >> >>> > >> >> > > >> >> >>> > >> >> > > Thanks for pointing those two fields out. I retract my >> -1. >> >> >>> > >> >> > > >> >> >>> > >> >> > > Cheers, >> >> >>> > >> >> > > Anton >> >> >>> > >> >> > > >> >> >>> > >> >> > > Den sön 27 okt. 2024 kl 17:40 skrev Chia-Ping Tsai < >> >> >>> > >> >> chia7...@apache.org >> >> >>> > >> >> > >: >> >> >>> > >> >> > > >> >> >>> > >> >> > >> hi Anton >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> Thanks for sharing your insights on Kafka >> >> serialization—it’s >> >> >>> really >> >> >>> > >> >> cool >> >> >>> > >> >> > >> and interesting to me. Additionally, you inspired me to >> >> file >> >> >>> a JIRA >> >> >>> > >> >> > issue >> >> >>> > >> >> > >> (KAFKA-17882) to improve the documentation. >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> The most important aspect of Kafka is compatibility, >> and >> >> the >> >> >>> > >> >> > undocumented >> >> >>> > >> >> > >> behavior has been in place for some time [0][1]. This >> >> means >> >> >>> > >> there’s no >> >> >>> > >> >> > need >> >> >>> > >> >> > >> to rush your improvement for 3.9, as we’ll need to >> >> explicitly >> >> >>> add >> >> >>> > >> >> > default >> >> >>> > >> >> > >> values after applying your patch to ensure we generate >> >> the >> >> >>> same >> >> >>> > >> binary >> >> >>> > >> >> > data. >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> In short, we can improve the documentation first. In >> the >> >> >>> meantime, >> >> >>> > >> we >> >> >>> > >> >> > can >> >> >>> > >> >> > >> continue discussing behavior clarification for 4.0, and >> >> RM >> >> >>> can keep >> >> >>> > >> >> > running >> >> >>> > >> >> > >> the RC for 3.9. Everything is on track. >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> Best, >> >> >>> > >> >> > >> Chia-Ping >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> [0] >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> >> >>> > >> >> >> >> >>> > >> >> >> >>> >> >> >> https://github.com/apache/kafka/blob/3.8/clients/src/main/resources/common/message/FetchSnapshotResponse.json#L43 >> >> >>> > >> >> > >> [1] >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> >> >>> > >> >> >> >> >>> > >> >> >> >>> >> >> >> https://github.com/apache/kafka/blob/3.8/group-coordinator/src/main/resources/common/message/ConsumerGroupMemberMetadataValue.json#L39 >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> On 2024/10/27 15:28:05 Anton Agestam wrote: >> >> >>> > >> >> > >> > -1, refer to comments on the RC 2 thread. >> >> >>> > >> >> > >> > >> >> >>> > >> >> > >> > Den sön 27 okt. 2024 kl 02:51 skrev Colin McCabe < >> >> >>> > >> >> cmcc...@apache.org >> >> >>> > >> >> > >: >> >> >>> > >> >> > >> > >> >> >>> > >> >> > >> > > This is the RC5 candidate for the release of Apache >> >> Kafka >> >> >>> > >> 3.9.0. >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > - This is a major release, the final one in the 3.x >> >> line. >> >> >>> > >> (There >> >> >>> > >> >> > may of >> >> >>> > >> >> > >> > > course be other minor releases in this line, such >> as >> >> >>> 3.9.1.) >> >> >>> > >> >> > >> > > - Tiered storage will be considered >> production-ready >> >> in >> >> >>> this >> >> >>> > >> >> > release. >> >> >>> > >> >> > >> > > - This will be the final major release to feature >> the >> >> >>> > >> deprecated >> >> >>> > >> >> > >> ZooKeeper >> >> >>> > >> >> > >> > > mode. >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > This release includes the following KIPs: >> >> >>> > >> >> > >> > > - KIP-853: Support dynamically changing KRaft >> >> controller >> >> >>> > >> >> membership >> >> >>> > >> >> > >> > > - KIP-1057: Add remote log metadata flag to the >> dump >> >> log >> >> >>> tool >> >> >>> > >> >> > >> > > - KIP-1049: Add config log.summary.interval.ms to >> >> Kafka >> >> >>> > >> Streams >> >> >>> > >> >> > >> > > - KIP-1040: Improve handling of nullable values in >> >> >>> InsertField, >> >> >>> > >> >> > >> > > ExtractField, and other transformations >> >> >>> > >> >> > >> > > - KIP-1031: Control offset translation in >> >> >>> MirrorSourceConnector >> >> >>> > >> >> > >> > > - KIP-1033: Add Kafka Streams exception handler for >> >> >>> exceptions >> >> >>> > >> >> > >> occurring >> >> >>> > >> >> > >> > > during processing >> >> >>> > >> >> > >> > > - KIP-1017: Health check endpoint for Kafka Connect >> >> >>> > >> >> > >> > > - KIP-1025: Optionally URL-encode clientID and >> >> >>> clientSecret in >> >> >>> > >> >> > >> > > authorization header >> >> >>> > >> >> > >> > > - KIP-1005: Expose EarliestLocalOffset and >> >> TieredOffset >> >> >>> > >> >> > >> > > - KIP-950: Tiered Storage Disablement >> >> >>> > >> >> > >> > > - KIP-956: Tiered Storage Quotas >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > Release notes for the 3.9.0 release: >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> >> >>> > >> >> >> >> >>> > >> >> >> >>> >> >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/RELEASE_NOTES.html >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > *** Please download, test and vote by October 30, >> >> 2024. >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > Kafka's KEYS file containing PGP keys we use to >> sign >> >> the >> >> >>> > >> release: >> >> >>> > >> >> > >> > > https://kafka.apache.org/KEYS >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Release artifacts to be voted upon (source and >> >> binary): >> >> >>> > >> >> > >> > > >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/ >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Docker release artifacts to be voted upon: >> >> >>> > >> >> > >> > > apache/kafka:3.9.0-rc5 >> >> >>> > >> >> > >> > > apache/kafka-native:3.9.0-rc5 >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Maven artifacts to be voted upon: >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> >> >>> > >> >> >> >>> >> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/ >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Javadoc: >> >> >>> > >> >> > >> > > >> >> >>> > >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc5/javadoc/ >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Documentation: >> >> >>> > >> >> > >> > > https://kafka.apache.org/39/documentation.html >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Protocol: >> >> >>> > >> >> > >> > > https://kafka.apache.org/39/protocol.html >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Tag to be voted upon (off 3.9 branch) is the >> >> 3.9.0-rc5 >> >> >>> tag: >> >> >>> > >> >> > >> > > >> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc5 >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > * Successful Docker Image Github Actions Pipeline >> >> for 3.9 >> >> >>> > >> branch: >> >> >>> > >> >> > >> > > Docker Build Test Pipeline (JVM): >> >> >>> > >> >> > >> > > >> >> https://github.com/apache/kafka/actions/runs/11535300463 >> >> >>> > >> >> > >> > > Docker Build Test Pipeline (Native): >> >> >>> > >> >> > >> > > >> >> https://github.com/apache/kafka/actions/runs/11535328957 >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > Thanks to everyone who helped with this release >> >> candidate, >> >> >>> > >> either >> >> >>> > >> >> by >> >> >>> > >> >> > >> > > contributing code, testing, or documentation. >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > > Regards, >> >> >>> > >> >> > >> > > Colin >> >> >>> > >> >> > >> > > >> >> >>> > >> >> > >> > >> >> >>> > >> >> > >> >> >> >>> > >> >> > >> >> >>> > >> >> >> >> >>> > >> >> >> >>> > > >> >> >>> > > >> >> >>> > > -- >> >> >>> > > David Arthur >> >> >>> >> >> >> > >> > > > -- > [image: Aiven] <https://www.aiven.io> > > *Josep Prat* > Open Source Engineering Director, *Aiven* > josep.p...@aiven.io | +491715557497 > aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> > *Aiven Deutschland GmbH* > Alexanderufer 3-7, 10117 Berlin > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > Anna Richardson, Kenneth Chen > Amtsgericht Charlottenburg, HRB 209739 B
