Hi Justine, I looked at this again, and it seems like we somehow dropped the code to handle --feature. I filed a blocker JIRA to fix: Do not ignore --feature flag in kafka-storage.sh
I will also add some more junit tests in this area. Sadly, I guess that means we'll need an RC4. thanks, Colin On Thu, Oct 24, 2024, at 10:10, Colin McCabe wrote: > On Wed, Oct 23, 2024, at 23:22, Justine Olshan wrote: >> Hey Colin, >> >> Thanks for running the release. >> >> I checked the keys and scanned the docs >> I built from source, used kraft quickstart, ran a transactions workload, >> played around with a few other things including running some of the feature >> version commands. >> > Hi Justine, > > Thanks for taking a look at the RC! > > [responses below] > >> I noticed that some of the KIP-1022 work that was meant to make it in this >> release was removed. >> Without any specifications, the cluster is meant to take the latest version >> of all available features. The code to do that was removed. Perhaps there >> is a reason, but we should either change the KIP or remedy this. >> Not only that, but the ability to configure a feature via the --feature >> flag in the storage tool also seems to be removed but the text remains when >> running with no arguments. I don't think it is a blocker, but this should >> be fixed for 4.0. > > For kraft.version (aka KIP-853) we opted to have the formatting tool > enable the feature automatically if controller.quorum.voter is not > specified. In other words, if you configure a static quorum, the > formatter gives you a static quorum; otherwise, the formatter creates a > dynamic quorum. We felt that this was more intuitive than making the > user manually supply --feature kraft.version=1 when dynamic quorums > were in use, and --feature kraft.version=0 when static quorums were in > use. > > Another issue is that if the formatter had automatically set everyone > to dynamic quorums, it would have been an incompatible change. > Configuration files without controller.quorum.bootstrap.servers can't > be used when dynamic quorums are in use. I think people would have > perceived this as "3.9 is broken" since their old configurations would > have stopped working. > > Note that it may still be useful to specify --feature kraft.version=1 > since that will force the formatter to always give you a dynamic > quorum, and error out if that is not possible. > >> >> I looked at two runs of the system tests and they looked fine to me. >> > > Yes, I also did a ducktape run on rc3. I should have mentioned that earlier. > > best, > Colin > > >> +1 (binding) from me, >> >> Justine >> >> On Tue, Oct 22, 2024 at 1:21 PM Bill Bejeck <bbej...@apache.org> wrote: >> >>> Hi Colin, >>> >>> Thanks for running the release. >>> >>> I did the following verification steps: >>> >>> 1. Built from source >>> 2. Validated all checksums >>> 3. Ran all the unit tests >>> 4. Pulled both docker images and started up Kafka >>> 5. Ran the KRaft quick start >>> 6. Ran the ZK quick start >>> 7. Ran the Kafka Streams quick start >>> 8. Spot checked the documentation >>> 9. Spot checked the javadocs >>> >>> +1(binding) >>> >>> Thanks, >>> Bill >>> >>> >>> >>> On Tue, Oct 22, 2024 at 9:08 AM Federico Valeri <fedeval...@gmail.com> >>> wrote: >>> >>> > Hi, I did the following tests: >>> > >>> > - Ran all unit and integration tests >>> > - Spot checked the documentation >>> > - Tried out new Maven dependencies >>> > - Tested KIP-853, KIP-950, KIP-956 and KIP-1057 >>> > >>> > +1 (non binding) >>> > >>> > Thanks >>> > Fede >>> > >>> > >>> > On Mon, Oct 21, 2024 at 11:12 PM Colin McCabe <cmcc...@apache.org> >>> wrote: >>> > > >>> > > This is the third candidate for the release of Apache Kafka 3.9.0. I >>> > have titled it rc3 since I had an rc1 which got very far, even to the >>> point >>> > of pushing tags and docker images, before I spotted an issue. So rather >>> > than mutate the tags, I decided to skip over rc1. >>> > > >>> > > - This is a major release, the final one in the 3.x line. (There may of >>> > course be other minor releases in this line, such as 3.9.1.) >>> > > - Tiered storage will be considered production-ready in this release. >>> > > - This will be the final major release to feature the deprecated >>> > ZooKeeper mode. >>> > > >>> > > This release includes the following KIPs: >>> > > - KIP-853: Support dynamically changing KRaft controller membership >>> > > - KIP-1057: Add remote log metadata flag to the dump log tool >>> > > - KIP-1049: Add config log.summary.interval.ms to Kafka Streams >>> > > - KIP-1040: Improve handling of nullable values in InsertField, >>> > ExtractField, and other transformations >>> > > - KIP-1031: Control offset translation in MirrorSourceConnector >>> > > - KIP-1033: Add Kafka Streams exception handler for exceptions >>> occurring >>> > during processing >>> > > - KIP-1017: Health check endpoint for Kafka Connect >>> > > - KIP-1025: Optionally URL-encode clientID and clientSecret in >>> > authorization header >>> > > - KIP-1005: Expose EarliestLocalOffset and TieredOffset >>> > > - KIP-950: Tiered Storage Disablement >>> > > - KIP-956: Tiered Storage Quotas >>> > > >>> > > Release notes for the 3.9.0 release: >>> > > >>> > >>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc3/RELEASE_NOTES.html >>> > > >>> > > *** Please download, test and vote by October 24, 2024. >>> > > >>> > > Kafka's KEYS file containing PGP keys we use to sign the release: >>> > > https://kafka.apache.org/KEYS >>> > > >>> > > * Release artifacts to be voted upon (source and binary): >>> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc3/ >>> > > >>> > > * Docker release artifacts to be voted upon: >>> > > apache/kafka:3.9.0-rc3 >>> > > apache/kafka-native:3.9.0-rc3 >>> > > >>> > > * Maven artifacts to be voted upon: >>> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ >>> > > >>> > > * Javadoc: >>> > > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc3/javadoc/ >>> > > >>> > > * Documentation: >>> > > https://kafka.apache.org/39/documentation.html >>> > > >>> > > * Protocol: >>> > > https://kafka.apache.org/39/protocol.html >>> > > >>> > > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc3 tag: >>> > > https://github.com/apache/kafka/releases/tag/3.9.0-rc3 >>> > > >>> > > * Successful Docker Image Github Actions Pipeline for 3.9 branch: >>> > > Docker Build Test Pipeline (JVM): >>> > https://github.com/apache/kafka/actions/runs/11448325059 >>> > > Docker Build Test Pipeline (Native): >>> > https://github.com/apache/kafka/actions/runs/11448338981 >>> > > >>> > > Thanks to everyone who helped with this release candidate, either by >>> > contributing code, testing, or documentation. >>> > > >>> > > Regards, >>> > > Colin >>> > >>>
