I also posted a PR to add removing control.plane.listener to the docs: https://github.com/apache/kafka/pull/17501
This needs to be done for each broker process before before it is restarted in KRaft mode. (However, it is fine to BEGIN the migration process with control plane listeners enabled.) best, Colin On Mon, Oct 14, 2024, at 10:54, Colin McCabe wrote: > Hi Jakub, > > After looking through the attached file on the JIRA, I can say that > this is a misconfiguration. control.plane.listener is a totally > separate concept from control.plane.listener.name. They should never be > set to the same value. The controller listener must have a different > name and value than the control plane listener (if any). > > I also tested myself that KRaft migration works with > control.plane.listener configured. It works on both 3.8 and 3.9-RC2. > > My initial statement that control.plane.listener was not supported > during ZK migration was incorrect. As you said, it is supported during > migration up to the point that we are in KRaft mode. (Another reason > why having the control plane listener = controller listener would not > make sense.) > > Thanks for the bug report and discussion. I've closed this as invalid > now that I have tested migration using control.plane.listener for > myself and verified that it works. > > best, > Colin > > On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote: >>> control.plane.listener is not (and never has been) supported in KRaft >> mode. >> >> You mean control.plane.listener.name is not supported in KRaft I guess? >> Well, this is not KRaft, this is migration, so it uses the settings that it >> used before for the Zoo-based cluster and that includes using dedicated >> control plane listener. I don't think I can "just remove it" because the >> other nodes will use it during the rolling update. >> >> This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not supported >> now, it is a breaking API change I guess which should be a blocker. >> >> Thanks & Regards >> Jakub >> >> On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe <cmcc...@apache.org> wrote: >> >>> Hi Jakub, >>> >>> Thanks for testing. control.plane.listener is not (and never has been) >>> supported in KRaft mode. You have to remove control.plane.listener >>> configurations before migrating. I filed KAFKA-17790 to document this in >>> the migration instructions. (This is not a blocker for the release, though.) >>> >>> best, >>> Colin >>> >>> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote: >>> > Hi Colin, >>> > >>> > Thanks for the RC. I did some testing of it and run into >>> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a >>> > regression in the migration to KRaft process. >>> > >>> > Can someone who understands this part of the codebase look into it >>> please? >>> > >>> > Thanks & Regards >>> > Jakub >>> > >>> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe <cmcc...@apache.org> >>> wrote: >>> > >>> >> This is the second candidate for the release of Apache Kafka 3.9.0. I >>> have >>> >> titled it rc2 since I had an rc1 which got very far, even to the point >>> of >>> >> pushing tags and docker images, before I spotted an issue. So rather >>> than >>> >> mutate the tags, I decided to skip over rc1. >>> >> >>> >> - This is a major release, the final one in the 3.x line. (There may of >>> >> course be other minor releases in this line, such as 3.9.1.) >>> >> - Tiered storage will be considered production-ready in this release. >>> >> - This will be the final major release to feature the deprecated >>> ZooKeeper >>> >> mode. >>> >> >>> >> This release includes the following KIPs: >>> >> - KIP-853: Support dynamically changing KRaft controller membership >>> >> - KIP-1057: Add remote log metadata flag to the dump log tool >>> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams >>> >> - KIP-1040: Improve handling of nullable values in InsertField, >>> >> ExtractField, and other transformations >>> >> - KIP-1031: Control offset translation in MirrorSourceConnector >>> >> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring >>> >> during processing >>> >> - KIP-1017: Health check endpoint for Kafka Connect >>> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in >>> >> authorization header >>> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset >>> >> - KIP-950: Tiered Storage Disablement >>> >> - KIP-956: Tiered Storage Quotas >>> >> >>> >> Release notes for the 3.9.0 release: >>> >> >>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html >>> >> >>> >> *** Please download, test and vote by October 16, 2024. >>> >> >>> >> Kafka's KEYS file containing PGP keys we use to sign the release: >>> >> https://kafka.apache.org/KEYS >>> >> >>> >> * Release artifacts to be voted upon (source and binary): >>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/ >>> >> >>> >> * Docker release artifacts to be voted upon: >>> >> apache/kafka:3.9.0-rc2 >>> >> apache/kafka-native:3.9.0-rc2 >>> >> >>> >> * Maven artifacts to be voted upon: >>> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/ >>> >> >>> >> * Javadoc: >>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/ >>> >> >>> >> * Documentation: >>> >> https://kafka.apache.org/39/documentation.html >>> >> >>> >> * Protocol: >>> >> https://kafka.apache.org/39/protocol.html >>> >> >>> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag: >>> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc2 >>> >> >>> >> * Successful Docker Image Github Actions Pipeline for 3.9 branch: >>> >> Docker Build Test Pipeline (JVM): >>> >> https://github.com/apache/kafka/actions/runs/11281563007 >>> >> Docker Build Test Pipeline (Native): >>> >> https://github.com/apache/kafka/actions/runs/11281608809 >>> >> >>> >> Thanks to everyone who helped with this release candidate, either by >>> >> contributing code, testing, or documentation. >>> >> >>> >> Regards, >>> >> Colin >>> >> >>>
