The different name of the controller listener for KRaft controllers and control plane listener in ZooKeeper-based cluster was not required before and it is not simple to change to handle now at the "last minute". So given that this is called production-ready already for some time, I think this is breaking API change and should be treated as such.
Thanks & Regards Jakub On Mon, Oct 14, 2024 at 7:55 PM Colin McCabe <cmcc...@apache.org> wrote: > Hi Jakub, > > After looking through the attached file on the JIRA, I can say that this > is a misconfiguration. control.plane.listener is a totally separate concept > from control.plane.listener.name. They should never be set to the same > value. The controller listener must have a different name and value than > the control plane listener (if any). > > I also tested myself that KRaft migration works with > control.plane.listener configured. It works on both 3.8 and 3.9-RC2. > > My initial statement that control.plane.listener was not supported during > ZK migration was incorrect. As you said, it is supported during migration > up to the point that we are in KRaft mode. (Another reason why having the > control plane listener = controller listener would not make sense.) > > Thanks for the bug report and discussion. I've closed this as invalid now > that I have tested migration using control.plane.listener for myself and > verified that it works. > > best, > Colin > > On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote: > >> control.plane.listener is not (and never has been) supported in KRaft > > mode. > > > > You mean control.plane.listener.name is not supported in KRaft I guess? > > Well, this is not KRaft, this is migration, so it uses the settings that > it > > used before for the Zoo-based cluster and that includes using dedicated > > control plane listener. I don't think I can "just remove it" because the > > other nodes will use it during the rolling update. > > > > This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not > supported > > now, it is a breaking API change I guess which should be a blocker. > > > > Thanks & Regards > > Jakub > > > > On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe <cmcc...@apache.org> wrote: > > > >> Hi Jakub, > >> > >> Thanks for testing. control.plane.listener is not (and never has been) > >> supported in KRaft mode. You have to remove control.plane.listener > >> configurations before migrating. I filed KAFKA-17790 to document this in > >> the migration instructions. (This is not a blocker for the release, > though.) > >> > >> best, > >> Colin > >> > >> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote: > >> > Hi Colin, > >> > > >> > Thanks for the RC. I did some testing of it and run into > >> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a > >> > regression in the migration to KRaft process. > >> > > >> > Can someone who understands this part of the codebase look into it > >> please? > >> > > >> > Thanks & Regards > >> > Jakub > >> > > >> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe <cmcc...@apache.org> > >> wrote: > >> > > >> >> This is the second candidate for the release of Apache Kafka 3.9.0. I > >> have > >> >> titled it rc2 since I had an rc1 which got very far, even to the > point > >> of > >> >> pushing tags and docker images, before I spotted an issue. So rather > >> than > >> >> mutate the tags, I decided to skip over rc1. > >> >> > >> >> - This is a major release, the final one in the 3.x line. (There may > of > >> >> course be other minor releases in this line, such as 3.9.1.) > >> >> - Tiered storage will be considered production-ready in this release. > >> >> - This will be the final major release to feature the deprecated > >> ZooKeeper > >> >> mode. > >> >> > >> >> This release includes the following KIPs: > >> >> - KIP-853: Support dynamically changing KRaft controller membership > >> >> - KIP-1057: Add remote log metadata flag to the dump log tool > >> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams > >> >> - KIP-1040: Improve handling of nullable values in InsertField, > >> >> ExtractField, and other transformations > >> >> - KIP-1031: Control offset translation in MirrorSourceConnector > >> >> - KIP-1033: Add Kafka Streams exception handler for exceptions > occurring > >> >> during processing > >> >> - KIP-1017: Health check endpoint for Kafka Connect > >> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in > >> >> authorization header > >> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset > >> >> - KIP-950: Tiered Storage Disablement > >> >> - KIP-956: Tiered Storage Quotas > >> >> > >> >> Release notes for the 3.9.0 release: > >> >> > >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html > >> >> > >> >> *** Please download, test and vote by October 16, 2024. > >> >> > >> >> Kafka's KEYS file containing PGP keys we use to sign the release: > >> >> https://kafka.apache.org/KEYS > >> >> > >> >> * Release artifacts to be voted upon (source and binary): > >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/ > >> >> > >> >> * Docker release artifacts to be voted upon: > >> >> apache/kafka:3.9.0-rc2 > >> >> apache/kafka-native:3.9.0-rc2 > >> >> > >> >> * Maven artifacts to be voted upon: > >> >> > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > >> >> > >> >> * Javadoc: > >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/ > >> >> > >> >> * Documentation: > >> >> https://kafka.apache.org/39/documentation.html > >> >> > >> >> * Protocol: > >> >> https://kafka.apache.org/39/protocol.html > >> >> > >> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag: > >> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc2 > >> >> > >> >> * Successful Docker Image Github Actions Pipeline for 3.9 branch: > >> >> Docker Build Test Pipeline (JVM): > >> >> https://github.com/apache/kafka/actions/runs/11281563007 > >> >> Docker Build Test Pipeline (Native): > >> >> https://github.com/apache/kafka/actions/runs/11281608809 > >> >> > >> >> Thanks to everyone who helped with this release candidate, either by > >> >> contributing code, testing, or documentation. > >> >> > >> >> Regards, > >> >> Colin > >> >> > >> >
