Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Hi Jakub,

It has always been required to separate control plane listeners and controller 
listeners. Failing to do this is a major misconfiguration. It may not have been 
caught sometimes, but that is a bug.

It should be simple to fix the configuration you posted -- simply have a 
different name for the controller listener than the control plane listener.

best,
Colin

On Mon, Oct 14, 2024, at 11:16, Jakub Scholz wrote:
> The different name of the controller listener for KRaft controllers and
> control plane listener in ZooKeeper-based cluster was not required before
> and it is not simple to change to handle now at the "last minute". So given
> that this is called production-ready already for some time, I think this is
> breaking API change and should be treated as such.
>
> Thanks & Regards
> Jakub
>
> On Mon, Oct 14, 2024 at 7:55 PM Colin McCabe  wrote:
>
>> Hi Jakub,
>>
>> After looking through the attached file on the JIRA, I can say that this
>> is a misconfiguration. control.plane.listener is a totally separate concept
>> from control.plane.listener.name. They should never be set to the same
>> value. The controller listener must have a different name and value than
>> the control plane listener (if any).
>>
>> I also tested myself that KRaft migration works with
>> control.plane.listener configured. It works on both 3.8 and 3.9-RC2.
>>
>> My initial statement that control.plane.listener was not supported during
>> ZK migration was incorrect. As you said, it is supported during migration
>> up to the point that we are in KRaft mode. (Another reason why having the
>> control plane listener = controller listener would not make sense.)
>>
>> Thanks for the bug report and discussion. I've closed this as invalid now
>> that I have tested migration using control.plane.listener for myself and
>> verified that it works.
>>
>> best,
>> Colin
>>
>> On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote:
>> >> control.plane.listener is not (and never has been) supported in KRaft
>> > mode.
>> >
>> > You mean control.plane.listener.name is not supported in KRaft I guess?
>> > Well, this is not KRaft, this is migration, so it uses the settings that
>> it
>> > used before for the Zoo-based cluster and that includes using dedicated
>> > control plane listener. I don't think I can "just remove it" because the
>> > other nodes will use it during the rolling update.
>> >
>> > This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not
>> supported
>> > now, it is a breaking API change I guess which should be a blocker.
>> >
>> > Thanks & Regards
>> > Jakub
>> >
>> > On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe  wrote:
>> >
>> >> Hi Jakub,
>> >>
>> >> Thanks for testing. control.plane.listener is not (and never has been)
>> >> supported in KRaft mode. You have to remove control.plane.listener
>> >> configurations before migrating. I filed KAFKA-17790 to document this in
>> >> the migration instructions. (This is not a blocker for the release,
>> though.)
>> >>
>> >> best,
>> >> Colin
>> >>
>> >> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
>> >> > Hi Colin,
>> >> >
>> >> > Thanks for the RC. I did some testing of it and run into
>> >> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
>> >> > regression in the migration to KRaft process.
>> >> >
>> >> > Can someone who understands this part of the codebase look into it
>> >> please?
>> >> >
>> >> > Thanks & Regards
>> >> > Jakub
>> >> >
>> >> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe 
>> >> wrote:
>> >> >
>> >> >> This is the second candidate for the release of Apache Kafka 3.9.0. I
>> >> have
>> >> >> titled it rc2 since I had an rc1 which got very far, even to the
>> point
>> >> of
>> >> >> pushing tags and docker images, before I spotted an issue. So rather
>> >> than
>> >> >> mutate the tags, I decided to skip over rc1.
>> >> >>
>> >> >> - This is a major release, the final one in the 3.x line. (There may
>> of
>> >> >> course be other minor releases in this line, such as 3.9.1.)
>> >> >> - Tiered storage will be considered production-ready in this release.
>> >> >> - This will be the final major release to feature the deprecated
>> >> ZooKeeper
>> >> >> mode.
>> >> >>
>> >> >> This release includes the following KIPs:
>> >> >> - KIP-853: Support dynamically changing KRaft controller membership
>> >> >> - KIP-1057: Add remote log metadata flag to the dump log tool
>> >> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> >> >> - KIP-1040: Improve handling of nullable values in InsertField,
>> >> >> ExtractField, and other transformations
>> >> >> - KIP-1031: Control offset translation in MirrorSourceConnector
>> >> >> - KIP-1033: Add Kafka Streams exception handler for exceptions
>> occurring
>> >> >> during processing
>> >> >> - KIP-1017: Health check endpoint for Kafka Connect
>> >> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in
>> >> >> authorization header
>> >> >> - KIP-1005: Expose Ea

Re: [DISCUSS] KIP-1088: Replace KafkaClientSupplier with KafkaClientInterceptor

[Bill Bejeck]

HI All,

Matthias - thanks for the KIP!

I'm in favor of this proposal, but I'd like to summarize my understanding
to ensure I'm getting everything.

1. KafkaStreams will remain in control of creating the client unless the
user overrides that behavior (I think this would be a rare case).
Although we can't wholly enforce this creation rule, it's evident that
doing so breaks the contract, and users will need to understand they are on
their own at that point.
Of course, providing a MockConsumer instance would be supported or expected
for testing.

2. It allows users to override given method(s), in other words,
"cherry-pick" the behavior to modify easily, rather than implementing their
own client with overridden methods. (Admittedly, the difference between the
two approaches is minor.)

3. While we could only provide a `StreamsConsumer` and add methods as
needed, providing an interceptor seems more direct in guiding users on how
to modify behavior.

I'm good with the KIP as it stands now. I get what Sophie and Bruno are
alluding to with the naming, so maybe a name of `WrappedX` would be more
appropriate, but I don't have a strong opinion.

Thanks,
Bill

On Mon, Oct 14, 2024 at 4:17 AM Bruno Cadonna  wrote:

> Hi,
>
> Thanks for the KIP!
>
>
> BC1.
> I echo point S3 from Sophie. I find it quite confusing having those
> methods return an interceptor that is used as a client in the Streams
> code. Naming might already do the trick. Something like
> Intercepted(Admin|StreamsConsumer|Consumer|Producer) or
> Wrapped(Admin|StreamsConsumer|Consumer|Producer) come to my mind.
>
> Related to this: Why is the interface with all the wrap* methods also
> called Interceptor? The functionality is quite different from the other
> interceptors.
>
>
> BC2.
> Why do you not use interfaces instead of the classes for the clients in
> the wrap* signatures? We just want to intercept the calls to the
> interface, right?
>
> Related to this, why do you pass-in also the configs of the clients, if
> you only want to wrap the calls to them?
>
>
> BC3.
> Why do we need KafkaStreamsConsumer in the public API? Isn't the
> StreamsConsumer interface enough?
>
>
> BC4.
> I am wondering, whether a kind of observer pattern would be better. That
> means, in Streams we do not call the interceptor but we call the client
> and for each call we also call the corresponding method on the passed in
> object -- basically the user implements callbacks.
>
> That would make it even clearer that you should not inject your own
> client implementations since we remove the user's object from the direct
> call to the client. The injected object cannot participate into the RPCs
> at all. The trade-off is that the user only gets a call before the
> actual call to the client or afterwards. To alleviate that, we could
> provide an interface for the passed-in object that has a pre-call and
> post-call for each client call.
> Internally, we would have a wrapper around the clients that calls the
> corresponding method of the user-provided method. Something like:
>
> public ConsumerRecords poll(Duration timeout) {
>  ConsumerRecords records = actualConsumer.poll(timeout);
>  return userConsumerObserver.poll(timeout, records);
> }
>
> or with the pre and post calls
>
> public ConsumerRecords poll(Duration timeout) {
>  userConsumerObserver.prePoll(timeout);
>  ConsumerRecords records = actualConsumer.poll(timeout);
>  return userConsumerObserver.postPoll(timeout, records);
> }
>
>
> Best,
> Bruno
>
>
> On 05.10.24 02:31, Sophie Blee-Goldman wrote:
> > Thanks for the update Matthias! I'm totally in agreement with the new
> > proposal and have mainly only cosmetic points and minor nits remaining.
> > Once we come to agreement on these I would be happy to move this to a
> vote
> > (unless others chime in with new concerns of course)
> >
> > S1. One thing that jumped out at me is that only the main consumer
> > implements the new StreamsConsumer interface, whereas the restore
> consumer
> > and global consumers both remain on the original plain 'Consumer' type.
> > This decision in itself makes sense to me, but it makes me question the
> > naming and whether it might confuse users to have only 1 of 3 consumer
> > types in Kafka Streams use the "StreamsConsumer". Have you considered any
> > other names for the interface yet? I suppose "StreamsMainConsumer" is the
> > most literal option, but I would prefer to hint at the underlying
> behavior
> > difference in some way and call it something like "StreamsGroupConsumer"
> > since that is fundamentally what it is.
> >
> > S2. Building off of S1, personally I always thought something like
> > "groupConsumer" or "streamsGroupConsumer" would be a better, more
> > descriptive name than "mainConsumer" for the consumer whose job is to
> join
> > the application's consumer group (or "streams group" after KIP-1071). So
> > perhaps we can also take the opportunity to do this renaming and change
> the
> > interceptor's

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #89

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-17794) kafka-storage format must require KIP-853 flags unless controller.quorum.voters is set

[Colin McCabe (Jira)]

Colin McCabe created KAFKA-17794:


 Summary: kafka-storage format must require KIP-853 flags unless 
controller.quorum.voters is set
 Key: KAFKA-17794
 URL: https://issues.apache.org/jira/browse/KAFKA-17794
 Project: Kafka
  Issue Type: Bug
Affects Versions: 3.9.0
Reporter: Colin McCabe
Assignee: Colin McCabe
 Fix For: 3.9.0


kafka-storage format must require KIP-853 flags unless controller.quorum.voters 
is set



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

On Mon, Oct 14, 2024, at 15:04, Jakub Scholz wrote:
> Hi Colin,
>
> So, how exactly does this major misconfiguration that was not documented
> for over a year and nobody complained manifest itself? What should I look
> for in the logs? What are the problems it manifests itself through? There
> are plenty of users who went through the migration with this "major
> misconfiguration". So what should they be looking for? Did they lose
> messages? Does it have security consequences that require a CVE?  How
> does one recover from the problems it caused?
>
> I do not think changing the controller name is as simple as you suggest.
> And it is especially not simple if you need to roll it out to thousands of
> users.
>
> I also feel that issues like this need to be taken more seriously than just
> shrugging it off and quickly closing JIRA someone else opened as invalid.
> This is not the first thing that was found as undocumented. It indicates
> significant quality issues, at least on the documentation side. How many
> other "major misconfigurations" are still left?

Hi Jakub,

I understand your frustration. Let me explain. Kafka wants to divide up all 
listeners into either broker listeners or controller listeners. The sets are 
disjoint: a listener can't be both.

BrokerServer will try to open the ports that belong to the broker; 
ControllerServer will try to open the ports that belong to the controller. You 
obviously can't open the same port twice (in standard UNIX, at least) or if you 
somehow did, the result would be nonsense. This isn't such a big deal on broker 
or controller nodes, but it becomes a bigger deal when in "combined" mode, 
where a process functions as both broker and controller.

You are getting the error message "requirement failed: 
control.plane.listener.name must be a listener name defined in 
advertised.listeners" because the broker is looking for a listener named 
CONTROLPLANE-9090, but it already knows that this is NOT an advertised listener 
for the broker, it is an advertised listener for the controller. You would get 
the same error if you tried to set the inter.broker.listener or the broker 
replication listener to the CONTROLLER listener.

The reason why this wasn't an issue in 3.8.0, and is in 3.9-RC2 is that in 
3.8.0 and earlier, advertised.listeners was just for broker listeners. There 
was never a need to put a controller listener in there because controller 
listeners were statically configured by controller.quorum.voters. In fact, when 
in KRaft mode, it would be a fatal configuration error to put a controller 
listener into advertised.listeners. Unfortunately, we failed to enforce this 
when the broker was in migration mode.

I guess for the purpose of being bug-compatible with 3.8, we could make an 
exception here and force the listener specified by control.plane.listener to 
appear in effectiveAdvertisedBrokerListeners. Since control plane listeners are 
going away anyway, we won't have to support this exception for very long.

By the way, I wasn't trying to be dismissive of your bug report (in fact I 
spent several hours on it). I felt (in fact I still feel) that it's a 
configuration error. But your compatibility argument is a reasonable one. So 
let's be compatible.

best,
Colin

>
> Jakub
>
> On Mon, Oct 14, 2024 at 11:33 PM Colin McCabe  wrote:
>
>> Hi Jakub,
>>
>> It has always been required to separate control plane listeners and
>> controller listeners. Failing to do this is a major misconfiguration. It
>> may not have been caught sometimes, but that is a bug.
>>
>> It should be simple to fix the configuration you posted -- simply have a
>> different name for the controller listener than the control plane listener.
>>
>> best,
>> Colin
>>
>> On Mon, Oct 14, 2024, at 11:16, Jakub Scholz wrote:
>> > The different name of the controller listener for KRaft controllers and
>> > control plane listener in ZooKeeper-based cluster was not required before
>> > and it is not simple to change to handle now at the "last minute". So
>> given
>> > that this is called production-ready already for some time, I think this
>> is
>> > breaking API change and should be treated as such.
>> >
>> > Thanks & Regards
>> > Jakub
>> >
>> > On Mon, Oct 14, 2024 at 7:55 PM Colin McCabe  wrote:
>> >
>> >> Hi Jakub,
>> >>
>> >> After looking through the attached file on the JIRA, I can say that this
>> >> is a misconfiguration. control.plane.listener is a totally separate
>> concept
>> >> from control.plane.listener.name. They should never be set to the same
>> >> value. The controller listener must have a different name and value than
>> >> the control plane listener (if any).
>> >>
>> >> I also tested myself that KRaft migration works with
>> >> control.plane.listener configured. It works on both 3.8 and 3.9-RC2.
>> >>
>> >> My initial statement that control.plane.listener was not supported
>> during
>> >> ZK migration was incorrect. As you said, it is supported during
>> migration
>> >> up to 

[jira] [Reopened] (KAFKA-17788) Migration fails in Kafka 3.9.0-RC2 when control plane listener is used

[Colin McCabe (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin McCabe reopened KAFKA-17788:
--
  Assignee: Colin McCabe

> Migration fails in Kafka 3.9.0-RC2 when control plane listener is used
> --
>
> Key: KAFKA-17788
> URL: https://issues.apache.org/jira/browse/KAFKA-17788
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 3.9.0
>Reporter: Jakub Scholz
>Assignee: Colin McCabe
>Priority: Blocker
> Attachments: logs-pod-cluster-9833cba6-kafka-1-container-kafka.log
>
>
> When testing migration with Kafka 3.9.0-RC2, the broker fail to start when 
> they are first rolled to start the migration with the following error:
> {code}
> 2024-10-11 21:37:04,060 ERROR Exiting Kafka due to fatal exception 
> (kafka.Kafka$) [main]
> java.lang.IllegalArgumentException: requirement failed: 
> control.plane.listener.name must be a listener name defined in 
> advertised.listeners. The valid options based on currently configured 
> listeners are REPLICATION-9091,PLAIN-9092,TLS-9093
>   at scala.Predef$.require(Predef.scala:337)
>   at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1019)
>   at kafka.server.KafkaConfig.(KafkaConfig.scala:843)
>   at kafka.server.KafkaConfig.(KafkaConfig.scala:185)
>   at kafka.Kafka$.buildServer(Kafka.scala:71)
>   at kafka.Kafka$.main(Kafka.scala:90)
>   at kafka.Kafka.main(Kafka.scala)
> {code}
> This is despite our configuration having the {control.plane.listener.name} 
> properly configured:
> {code}
> listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,PLAIN-9092:SASL_PLAINTEXT,TLS-9093:SSL
> listeners=CONTROLPLANE-9090://0.0.0.0:9090,REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
> advertised.listeners=CONTROLPLANE-9090://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9090,REPLICATION-9091://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9091,PLAIN-9092://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9092,TLS-9093://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9093
> inter.broker.listener.name=REPLICATION-9091
> control.plane.listener.name=CONTROLPLANE-9090
> {code}
> It looks like 3.9.0-RC2 filters out the control plane listener (maybe because 
> it is used by the KRaft controllers as well?) and runs into this error. This 
> worked fine in 3.8.0, so this seems like a regression in 3.9.0 that should be 
> fixed.
> The full log from the broker node is attached. It includes the full 
> configuration of the broker as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

While messing around with the release candidate, I found a separate blocker 
bug: KAFKA-17794.

Basically, our behavior is confusing when we format the storage directory with 
KIP-853 (dynamic controller quorum), but no initial controllers are supplied. I 
have a PR which requires the initial controllers to be initialized in this case 
(since otherwise, it does not work).

best,
Colin

On Mon, Oct 14, 2024, at 16:34, Colin McCabe wrote:
> On Mon, Oct 14, 2024, at 15:04, Jakub Scholz wrote:
>> Hi Colin,
>>
>> So, how exactly does this major misconfiguration that was not documented
>> for over a year and nobody complained manifest itself? What should I look
>> for in the logs? What are the problems it manifests itself through? There
>> are plenty of users who went through the migration with this "major
>> misconfiguration". So what should they be looking for? Did they lose
>> messages? Does it have security consequences that require a CVE?  How
>> does one recover from the problems it caused?
>>
>> I do not think changing the controller name is as simple as you suggest.
>> And it is especially not simple if you need to roll it out to thousands of
>> users.
>>
>> I also feel that issues like this need to be taken more seriously than just
>> shrugging it off and quickly closing JIRA someone else opened as invalid.
>> This is not the first thing that was found as undocumented. It indicates
>> significant quality issues, at least on the documentation side. How many
>> other "major misconfigurations" are still left?
>
> Hi Jakub,
>
> I understand your frustration. Let me explain. Kafka wants to divide up 
> all listeners into either broker listeners or controller listeners. The 
> sets are disjoint: a listener can't be both.
>
> BrokerServer will try to open the ports that belong to the broker; 
> ControllerServer will try to open the ports that belong to the 
> controller. You obviously can't open the same port twice (in standard 
> UNIX, at least) or if you somehow did, the result would be nonsense. 
> This isn't such a big deal on broker or controller nodes, but it 
> becomes a bigger deal when in "combined" mode, where a process 
> functions as both broker and controller.
>
> You are getting the error message "requirement failed: 
> control.plane.listener.name must be a listener name defined in 
> advertised.listeners" because the broker is looking for a listener 
> named CONTROLPLANE-9090, but it already knows that this is NOT an 
> advertised listener for the broker, it is an advertised listener for 
> the controller. You would get the same error if you tried to set the 
> inter.broker.listener or the broker replication listener to the 
> CONTROLLER listener.
>
> The reason why this wasn't an issue in 3.8.0, and is in 3.9-RC2 is that 
> in 3.8.0 and earlier, advertised.listeners was just for broker 
> listeners. There was never a need to put a controller listener in there 
> because controller listeners were statically configured by 
> controller.quorum.voters. In fact, when in KRaft mode, it would be a 
> fatal configuration error to put a controller listener into 
> advertised.listeners. Unfortunately, we failed to enforce this when the 
> broker was in migration mode.
>
> I guess for the purpose of being bug-compatible with 3.8, we could make 
> an exception here and force the listener specified by 
> control.plane.listener to appear in effectiveAdvertisedBrokerListeners. 
> Since control plane listeners are going away anyway, we won't have to 
> support this exception for very long.
>
> By the way, I wasn't trying to be dismissive of your bug report (in 
> fact I spent several hours on it). I felt (in fact I still feel) that 
> it's a configuration error. But your compatibility argument is a 
> reasonable one. So let's be compatible.
>
> best,
> Colin
>
>>
>> Jakub
>>
>> On Mon, Oct 14, 2024 at 11:33 PM Colin McCabe  wrote:
>>
>>> Hi Jakub,
>>>
>>> It has always been required to separate control plane listeners and
>>> controller listeners. Failing to do this is a major misconfiguration. It
>>> may not have been caught sometimes, but that is a bug.
>>>
>>> It should be simple to fix the configuration you posted -- simply have a
>>> different name for the controller listener than the control plane listener.
>>>
>>> best,
>>> Colin
>>>
>>> On Mon, Oct 14, 2024, at 11:16, Jakub Scholz wrote:
>>> > The different name of the controller listener for KRaft controllers and
>>> > control plane listener in ZooKeeper-based cluster was not required before
>>> > and it is not simple to change to handle now at the "last minute". So
>>> given
>>> > that this is called production-ready already for some time, I think this
>>> is
>>> > breaking API change and should be treated as such.
>>> >
>>> > Thanks & Regards
>>> > Jakub
>>> >
>>> > On Mon, Oct 14, 2024 at 7:55 PM Colin McCabe  wrote:
>>> >
>>> >> Hi Jakub,
>>> >>
>>> >> After looking through the attached file on the JIRA, I can say that this
>>> >> is a misconfigur

[jira] [Resolved] (KAFKA-17693) Remove testCheckEarliestLocalTimestampVersion and testCheckLatestTieredTimestampVersion

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17693.

Fix Version/s: 4.0.0
   Resolution: Fixed

> Remove testCheckEarliestLocalTimestampVersion and 
> testCheckLatestTieredTimestampVersion
> ---
>
> Key: KAFKA-17693
> URL: https://issues.apache.org/jira/browse/KAFKA-17693
> Project: Kafka
>  Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Ming-Yen Chung
>Priority: Minor
> Fix For: 4.0.0
>
>
> Also, `testListOffsetsRequestOldestVersion` should be moved to java 
> `ListOffsetRequestTest`



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is still unstable: Kafka » Kafka PowerPC Daily » test-powerpc #87

[Apache Jenkins Server]

See 

[jira] [Resolved] (KAFKA-17748) Remove scala-java8-compat

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17748.

Resolution: Fixed

> Remove scala-java8-compat 
> --
>
> Key: KAFKA-17748
> URL: https://issues.apache.org/jira/browse/KAFKA-17748
> Project: Kafka
>  Issue Type: Improvement
>Reporter: TengYao Chi
>Assignee: TengYao Chi
>Priority: Major
> Fix For: 4.0.0
>
>
> We should remove `scala-java8-compat` lib after removing Java 8 and zk.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Jakub Scholz]

Hi Colin,

So, how exactly does this major misconfiguration that was not documented
for over a year and nobody complained manifest itself? What should I look
for in the logs? What are the problems it manifests itself through? There
are plenty of users who went through the migration with this "major
misconfiguration". So what should they be looking for? Did they lose
messages? Does it have security consequences that require a CVE?  How
does one recover from the problems it caused?

I do not think changing the controller name is as simple as you suggest.
And it is especially not simple if you need to roll it out to thousands of
users.

I also feel that issues like this need to be taken more seriously than just
shrugging it off and quickly closing JIRA someone else opened as invalid.
This is not the first thing that was found as undocumented. It indicates
significant quality issues, at least on the documentation side. How many
other "major misconfigurations" are still left?

Jakub

On Mon, Oct 14, 2024 at 11:33 PM Colin McCabe  wrote:

> Hi Jakub,
>
> It has always been required to separate control plane listeners and
> controller listeners. Failing to do this is a major misconfiguration. It
> may not have been caught sometimes, but that is a bug.
>
> It should be simple to fix the configuration you posted -- simply have a
> different name for the controller listener than the control plane listener.
>
> best,
> Colin
>
> On Mon, Oct 14, 2024, at 11:16, Jakub Scholz wrote:
> > The different name of the controller listener for KRaft controllers and
> > control plane listener in ZooKeeper-based cluster was not required before
> > and it is not simple to change to handle now at the "last minute". So
> given
> > that this is called production-ready already for some time, I think this
> is
> > breaking API change and should be treated as such.
> >
> > Thanks & Regards
> > Jakub
> >
> > On Mon, Oct 14, 2024 at 7:55 PM Colin McCabe  wrote:
> >
> >> Hi Jakub,
> >>
> >> After looking through the attached file on the JIRA, I can say that this
> >> is a misconfiguration. control.plane.listener is a totally separate
> concept
> >> from control.plane.listener.name. They should never be set to the same
> >> value. The controller listener must have a different name and value than
> >> the control plane listener (if any).
> >>
> >> I also tested myself that KRaft migration works with
> >> control.plane.listener configured. It works on both 3.8 and 3.9-RC2.
> >>
> >> My initial statement that control.plane.listener was not supported
> during
> >> ZK migration was incorrect. As you said, it is supported during
> migration
> >> up to the point that we are in KRaft mode. (Another reason why having
> the
> >> control plane listener = controller listener would not make sense.)
> >>
> >> Thanks for the bug report and discussion. I've closed this as invalid
> now
> >> that I have tested migration using control.plane.listener for myself and
> >> verified that it works.
> >>
> >> best,
> >> Colin
> >>
> >> On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote:
> >> >> control.plane.listener is not (and never has been) supported in KRaft
> >> > mode.
> >> >
> >> > You mean control.plane.listener.name is not supported in KRaft I
> guess?
> >> > Well, this is not KRaft, this is migration, so it uses the settings
> that
> >> it
> >> > used before for the Zoo-based cluster and that includes using
> dedicated
> >> > control plane listener. I don't think I can "just remove it" because
> the
> >> > other nodes will use it during the rolling update.
> >> >
> >> > This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not
> >> supported
> >> > now, it is a breaking API change I guess which should be a blocker.
> >> >
> >> > Thanks & Regards
> >> > Jakub
> >> >
> >> > On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe 
> wrote:
> >> >
> >> >> Hi Jakub,
> >> >>
> >> >> Thanks for testing. control.plane.listener is not (and never has
> been)
> >> >> supported in KRaft mode. You have to remove control.plane.listener
> >> >> configurations before migrating. I filed KAFKA-17790 to document
> this in
> >> >> the migration instructions. (This is not a blocker for the release,
> >> though.)
> >> >>
> >> >> best,
> >> >> Colin
> >> >>
> >> >> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
> >> >> > Hi Colin,
> >> >> >
> >> >> > Thanks for the RC. I did some testing of it and run into
> >> >> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to
> be a
> >> >> > regression in the migration to KRaft process.
> >> >> >
> >> >> > Can someone who understands this part of the codebase look into it
> >> >> please?
> >> >> >
> >> >> > Thanks & Regards
> >> >> > Jakub
> >> >> >
> >> >> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe 
> >> >> wrote:
> >> >> >
> >> >> >> This is the second candidate for the release of Apache Kafka
> 3.9.0. I
> >> >> have
> >> >> >> titled it rc2 since I had an rc1 which got very far, even to the
> >> point
> >> >> of

Re: [VOTE] 3.9.0 RC2

[Jakub Scholz]

Hi Colin,

> BrokerServer will try to open the ports that belong to the broker;
ControllerServer will try to open the ports that belong to the controller.
You obviously can't open the same port twice (in standard UNIX, at least)
or if you somehow did, the result would be nonsense. This isn't such a big
deal on broker or controller nodes, but it becomes a bigger deal when in
"combined" mode, where a process functions as both broker and controller.

Thanks for looking into it. Yeah, what you describe would be an issue. And
we do not use this configuration when running in full KRaft mode (whether
with combined nodes or not). It is only during the migration that this
configuration is set like this as you have parts of the old configuration
that mix up with the new configuration. So maybe the migration state can be
used to control whether the validation is applied or not? (The migration
anyway does not support combined nodes)

Thanks & Regards
Jakub

On Tue, Oct 15, 2024 at 1:35 AM Colin McCabe  wrote:

> On Mon, Oct 14, 2024, at 15:04, Jakub Scholz wrote:
> > Hi Colin,
> >
> > So, how exactly does this major misconfiguration that was not documented
> > for over a year and nobody complained manifest itself? What should I look
> > for in the logs? What are the problems it manifests itself through? There
> > are plenty of users who went through the migration with this "major
> > misconfiguration". So what should they be looking for? Did they lose
> > messages? Does it have security consequences that require a CVE?  How
> > does one recover from the problems it caused?
> >
> > I do not think changing the controller name is as simple as you suggest.
> > And it is especially not simple if you need to roll it out to thousands
> of
> > users.
> >
> > I also feel that issues like this need to be taken more seriously than
> just
> > shrugging it off and quickly closing JIRA someone else opened as invalid.
> > This is not the first thing that was found as undocumented. It indicates
> > significant quality issues, at least on the documentation side. How many
> > other "major misconfigurations" are still left?
>
> Hi Jakub,
>
> I understand your frustration. Let me explain. Kafka wants to divide up
> all listeners into either broker listeners or controller listeners. The
> sets are disjoint: a listener can't be both.
>
> BrokerServer will try to open the ports that belong to the broker;
> ControllerServer will try to open the ports that belong to the controller.
> You obviously can't open the same port twice (in standard UNIX, at least)
> or if you somehow did, the result would be nonsense. This isn't such a big
> deal on broker or controller nodes, but it becomes a bigger deal when in
> "combined" mode, where a process functions as both broker and controller.
>
> You are getting the error message "requirement failed:
> control.plane.listener.name must be a listener name defined in
> advertised.listeners" because the broker is looking for a listener named
> CONTROLPLANE-9090, but it already knows that this is NOT an advertised
> listener for the broker, it is an advertised listener for the controller.
> You would get the same error if you tried to set the inter.broker.listener
> or the broker replication listener to the CONTROLLER listener.
>
> The reason why this wasn't an issue in 3.8.0, and is in 3.9-RC2 is that in
> 3.8.0 and earlier, advertised.listeners was just for broker listeners.
> There was never a need to put a controller listener in there because
> controller listeners were statically configured by
> controller.quorum.voters. In fact, when in KRaft mode, it would be a fatal
> configuration error to put a controller listener into advertised.listeners.
> Unfortunately, we failed to enforce this when the broker was in migration
> mode.
>
> I guess for the purpose of being bug-compatible with 3.8, we could make an
> exception here and force the listener specified by control.plane.listener
> to appear in effectiveAdvertisedBrokerListeners. Since control plane
> listeners are going away anyway, we won't have to support this exception
> for very long.
>
> By the way, I wasn't trying to be dismissive of your bug report (in fact I
> spent several hours on it). I felt (in fact I still feel) that it's a
> configuration error. But your compatibility argument is a reasonable one.
> So let's be compatible.
>
> best,
> Colin
>
> >
> > Jakub
> >
> > On Mon, Oct 14, 2024 at 11:33 PM Colin McCabe 
> wrote:
> >
> >> Hi Jakub,
> >>
> >> It has always been required to separate control plane listeners and
> >> controller listeners. Failing to do this is a major misconfiguration. It
> >> may not have been caught sometimes, but that is a bug.
> >>
> >> It should be simple to fix the configuration you posted -- simply have a
> >> different name for the controller listener than the control plane
> listener.
> >>
> >> best,
> >> Colin
> >>
> >> On Mon, Oct 14, 2024, at 11:16, Jakub Scholz wrote:
> >> > The different name of the controll

[jira] [Resolved] (KAFKA-17786) align the low bound of ducktape version for 3.8.x

[Josep Prat (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josep Prat resolved KAFKA-17786.

Resolution: Fixed

Done via https://github.com/apache/kafka/pull/17482

> align the low bound of ducktape version for 3.8.x
> -
>
> Key: KAFKA-17786
> URL: https://issues.apache.org/jira/browse/KAFKA-17786
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Josep Prat
>Priority: Major
> Fix For: 3.8.1
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-17786) align the low bound of ducktape version for 3.8.x

[Josep Prat (Jira)]

Josep Prat created KAFKA-17786:
--

 Summary: align the low bound of ducktape version for 3.8.x
 Key: KAFKA-17786
 URL: https://issues.apache.org/jira/browse/KAFKA-17786
 Project: Kafka
  Issue Type: Sub-task
Reporter: Josep Prat
 Fix For: 3.8.1






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] KIP-1088: Replace KafkaClientSupplier with KafkaClientInterceptor

[Bruno Cadonna]

Hi,

Thanks for the KIP!


BC1.
I echo point S3 from Sophie. I find it quite confusing having those 
methods return an interceptor that is used as a client in the Streams 
code. Naming might already do the trick. Something like 
Intercepted(Admin|StreamsConsumer|Consumer|Producer) or 
Wrapped(Admin|StreamsConsumer|Consumer|Producer) come to my mind.


Related to this: Why is the interface with all the wrap* methods also 
called Interceptor? The functionality is quite different from the other 
interceptors.



BC2.
Why do you not use interfaces instead of the classes for the clients in 
the wrap* signatures? We just want to intercept the calls to the 
interface, right?


Related to this, why do you pass-in also the configs of the clients, if 
you only want to wrap the calls to them?



BC3.
Why do we need KafkaStreamsConsumer in the public API? Isn't the 
StreamsConsumer interface enough?



BC4.
I am wondering, whether a kind of observer pattern would be better. That 
means, in Streams we do not call the interceptor but we call the client 
and for each call we also call the corresponding method on the passed in 
object -- basically the user implements callbacks.


That would make it even clearer that you should not inject your own 
client implementations since we remove the user's object from the direct 
call to the client. The injected object cannot participate into the RPCs 
at all. The trade-off is that the user only gets a call before the 
actual call to the client or afterwards. To alleviate that, we could 
provide an interface for the passed-in object that has a pre-call and 
post-call for each client call.
Internally, we would have a wrapper around the clients that calls the 
corresponding method of the user-provided method. Something like:


public ConsumerRecords poll(Duration timeout) {
ConsumerRecords records = actualConsumer.poll(timeout);
return userConsumerObserver.poll(timeout, records);
}

or with the pre and post calls

public ConsumerRecords poll(Duration timeout) {
userConsumerObserver.prePoll(timeout);
ConsumerRecords records = actualConsumer.poll(timeout);
return userConsumerObserver.postPoll(timeout, records);
}


Best,
Bruno


On 05.10.24 02:31, Sophie Blee-Goldman wrote:

Thanks for the update Matthias! I'm totally in agreement with the new
proposal and have mainly only cosmetic points and minor nits remaining.
Once we come to agreement on these I would be happy to move this to a vote
(unless others chime in with new concerns of course)

S1. One thing that jumped out at me is that only the main consumer
implements the new StreamsConsumer interface, whereas the restore consumer
and global consumers both remain on the original plain 'Consumer' type.
This decision in itself makes sense to me, but it makes me question the
naming and whether it might confuse users to have only 1 of 3 consumer
types in Kafka Streams use the "StreamsConsumer". Have you considered any
other names for the interface yet? I suppose "StreamsMainConsumer" is the
most literal option, but I would prefer to hint at the underlying behavior
difference in some way and call it something like "StreamsGroupConsumer"
since that is fundamentally what it is.

S2. Building off of S1, personally I always thought something like
"groupConsumer" or "streamsGroupConsumer" would be a better, more
descriptive name than "mainConsumer" for the consumer whose job is to join
the application's consumer group (or "streams group" after KIP-1071). So
perhaps we can also take the opportunity to do this renaming and change the
interceptor's method name from "#wrapMainConsumer" to
"#wrapStreamsGroupConsumer" or something like that? (Would also be nice to
rename the variable names in the code to reflect this but that's an
implementation detail, and can be done in a standalone PR after the KIP is
completed. I'll do it myself if I have to :P )

S3. I'm a little confused about the need to introduce the intermediary
"Interceptor" classes, like AdminInterceptor and
ConsumerInterceptor and so on. Can't these methods just return the client
type directly? Doesn't seem like we're adding anything to these interfaces
and just extending the client class. Are they supposed to be marker
interfaces or something like that? It feels like we're adding unnecessary
noise to the API so I'd just like to understand the motivation behind this
choice

S4. Can you fill in the javadocs for the Interceptor class? Just want to
make sure we include some specific things. Mainly, if we're going to
include a StreamsConfig method of injecting the interceptor, we should make
sure users know to have their interceptor implementation extend
Configurable in order to obtain the application configs for any setup they
need to do when instantiated via the default constructor. Alternatively we
can just have the KafkaClientInterceptor extend Configurable itself as we
do in some similar cases.

S5. On the subject of the new config:
5a. Since it requires a class or cla

[jira] [Created] (KAFKA-17788) Migration fails in Kafka 3.9.0-RC2 when control plane listener is used

[Jakub Scholz (Jira)]

Jakub Scholz created KAFKA-17788:


 Summary: Migration fails in Kafka 3.9.0-RC2 when control plane 
listener is used
 Key: KAFKA-17788
 URL: https://issues.apache.org/jira/browse/KAFKA-17788
 Project: Kafka
  Issue Type: Bug
Reporter: Jakub Scholz
 Attachments: logs-pod-cluster-9833cba6-kafka-1-container-kafka.log

When testing migration with Kafka 3.9.0-RC2, the broker fail to start when they 
are first rolled to start the migration with the following error:

{code}
2024-10-11 21:37:04,060 ERROR Exiting Kafka due to fatal exception 
(kafka.Kafka$) [main]
java.lang.IllegalArgumentException: requirement failed: 
control.plane.listener.name must be a listener name defined in 
advertised.listeners. The valid options based on currently configured listeners 
are REPLICATION-9091,PLAIN-9092,TLS-9093
at scala.Predef$.require(Predef.scala:337)
at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1019)
at kafka.server.KafkaConfig.(KafkaConfig.scala:843)
at kafka.server.KafkaConfig.(KafkaConfig.scala:185)
at kafka.Kafka$.buildServer(Kafka.scala:71)
at kafka.Kafka$.main(Kafka.scala:90)
at kafka.Kafka.main(Kafka.scala)
{code}

This is despite our configuration having the {control.plane.listener.name} 
properly configured:

{code}
listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,PLAIN-9092:SASL_PLAINTEXT,TLS-9093:SSL
listeners=CONTROLPLANE-9090://0.0.0.0:9090,REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
advertised.listeners=CONTROLPLANE-9090://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9090,REPLICATION-9091://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9091,PLAIN-9092://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9092,TLS-9093://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9093
inter.broker.listener.name=REPLICATION-9091
control.plane.listener.name=CONTROLPLANE-9090
{code}

It looks like 3.9.0-RC2 filters out the control plane listener (maybe because 
it is used by the KRaft controllers as well?) and runs into this error. This 
worked fine in 3.8.0, so this seems like a regression in 3.9.0 that should be 
fixed.

The full log from the broker node is attached. It includes the full 
configuration of the broker as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Jakub Scholz]

Hi Colin,

Thanks for the RC. I did some testing of it and run into
https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
regression in the migration to KRaft process.

Can someone who understands this part of the codebase look into it please?

Thanks & Regards
Jakub

On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe  wrote:

> This is the second candidate for the release of Apache Kafka 3.9.0. I have
> titled it rc2 since I had an rc1 which got very far, even to the point of
> pushing tags and docker images, before I spotted an issue. So rather than
> mutate the tags, I decided to skip over rc1.
>
> - This is a major release, the final one in the 3.x line. (There may of
> course be other minor releases in this line, such as 3.9.1.)
> - Tiered storage will be considered production-ready in this release.
> - This will be the final major release to feature the deprecated ZooKeeper
> mode.
>
> This release includes the following KIPs:
> - KIP-853: Support dynamically changing KRaft controller membership
> - KIP-1057: Add remote log metadata flag to the dump log tool
> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
> - KIP-1040: Improve handling of nullable values in InsertField,
> ExtractField, and other transformations
> - KIP-1031: Control offset translation in MirrorSourceConnector
> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring
> during processing
> - KIP-1017: Health check endpoint for Kafka Connect
> - KIP-1025: Optionally URL-encode clientID and clientSecret in
> authorization header
> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> - KIP-950: Tiered Storage Disablement
> - KIP-956: Tiered Storage Quotas
>
> Release notes for the 3.9.0 release:
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>
> *** Please download, test and vote by October 16, 2024.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>
> * Docker release artifacts to be voted upon:
> apache/kafka:3.9.0-rc2
> apache/kafka-native:3.9.0-rc2
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>
> * Documentation:
> https://kafka.apache.org/39/documentation.html
>
> * Protocol:
> https://kafka.apache.org/39/protocol.html
>
> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
>
> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/11281563007
> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/11281608809
>
> Thanks to everyone who helped with this release candidate, either by
> contributing code, testing, or documentation.
>
> Regards,
> Colin
>

[jira] [Resolved] (KAFKA-17746) Replace JavaConverters with CollectionConverters

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17746.

Fix Version/s: 4.0.0
   Resolution: Fixed

> Replace JavaConverters with CollectionConverters
> 
>
> Key: KAFKA-17746
> URL: https://issues.apache.org/jira/browse/KAFKA-17746
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Mickael Maison
>Assignee: Lin Siyuan
>Priority: Major
> Fix For: 4.0.0
>
>
> scala.collection.JavaConverters is deprecated, since Scala 2.13 we can use 
> scala.jdk.CollectionConverters instead



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-17787) Remove --zookeeper option and logic from ConfigCommand

[Mickael Maison (Jira)]

Mickael Maison created KAFKA-17787:
--

 Summary: Remove --zookeeper option and logic from ConfigCommand
 Key: KAFKA-17787
 URL: https://issues.apache.org/jira/browse/KAFKA-17787
 Project: Kafka
  Issue Type: Sub-task
  Components: tools
Reporter: Mickael Maison






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-17617) New GitHub Actions build builds Java 8 with 2.13 instead of 2.12

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17617.

Fix Version/s: (was: 3.7.2)
   (was: 3.9.1)
   Resolution: Won't Fix

> New GitHub Actions build builds Java 8 with 2.13 instead of 2.12
> 
>
> Key: KAFKA-17617
> URL: https://issues.apache.org/jira/browse/KAFKA-17617
> Project: Kafka
>  Issue Type: Task
>Reporter: Justine Olshan
>Assignee: David Arthur
>Priority: Minor
>
> I noticed a PR that failed for Jenkins but not for the GitHub Actions build. 
> Tracing it down, it looks like Jenkins was using Scala 2.12 and GitHub 
> actions is using 2.13.
> We still should support 2.12, so we should fix the GitHub actions now that 
> Jenkins is removed. Until we fix, folks can merge in code that breaks 2.12 
> builds.
> See Jenkins for 
> failure:[https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-17093/5/cloudbees-pipeline-explorer/?filter=37]
> see for success in GH Actions: 
> [https://github.com/apache/kafka/actions/runs/11039342155/job/30672276597?pr=17093]
> see raw build for 2.13: 
> [https://productionresultssa2.blob.core.windows.net/actions-results/4f08a774-9253-4d15-8617-a627e9961b76/workflow-job-run-716dd2a8-3073-58cf-0d26-4a389b46b592/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-09-26T01%3A05%3A04Z&sig=klIXSOTwKN9WrBvtdsN6j45DbSqg7ikwow%2FGETJy5pc%3D&ske=2024-09-26T12%3A26%3A46Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-09-26T00%3A26%3A46Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2024-05-04&sp=r&spr=https&sr=b&st=2024-09-26T00%3A54%3A59Z&sv=2024-05-04]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-17374) add bootstrap.controller to kafka-reassign-partitions.sh

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17374.

Fix Version/s: 4.0.0
   Resolution: Fixed

> add bootstrap.controller to kafka-reassign-partitions.sh
> 
>
> Key: KAFKA-17374
> URL: https://issues.apache.org/jira/browse/KAFKA-17374
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: 黃竣陽
>Priority: Major
> Fix For: 4.0.0
>
>
> as title



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-17791) Dockerfile should use `requirements.txt` to ensure dependencies

[Chia-Ping Tsai (Jira)]

Chia-Ping Tsai created KAFKA-17791:
--

 Summary: Dockerfile should use `requirements.txt` to ensure 
dependencies
 Key: KAFKA-17791
 URL: https://issues.apache.org/jira/browse/KAFKA-17791
 Project: Kafka
  Issue Type: Improvement
Reporter: Chia-Ping Tsai
Assignee: 黃竣陽


see https://github.com/apache/kafka/pull/17481#issuecomment-2411631883



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[TengYao Chi]

Hi Colin,

I have tested 3.9 RC2 with Amazon Corretto 8-21 and Zulu 8-21, and
everything seems to be working fine.
Therefore, I am +1 on this RC (non-binding).

Best regards,
TengYao

Colin McCabe  於 2024年10月14日 週一 下午11:42寫道：

> Hi Chia-Ping Tsai,
>
> Thanks for looking into this. I approved the PR.
>
> best,
> Colin
>
> On Mon, Oct 14, 2024, at 08:34, Chia-Ping Tsai wrote:
> >> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk
> seems to be using now (ducktape 0.11.4) We will not advance the version of
> ducktape in the 3.x branch, I think.
> >
> > Yes, I agree not to advance the version of Ducktape, but the Ducktape
> > version was unpinned from 3.2 (see
> >
> https://github.com/apache/kafka/commit/7e683852b46c0fe70f5100e953e3cd61d1697c06),
>
> > so I suspect 3.9 hasn't been using Ducktape 0.8.14 for a while.
> >
> > The PR (https://github.com/apache/kafka/pull/17481) locks Ducktape to
> > version 0.11.4, which is the latest release that works well with the
> > 3.9 Dockerfile (and it is waiting for your review). Additionally, the
> > trunk can use Ducktape 0.12.0, which supports newer Python versions,
> > allowing us to upgrade the base image without relying on outdated
> > Python versions.
> >
> > Best,
> > Chia-Ping
> >
> > On 2024/10/14 15:08:06 Colin McCabe wrote:
> >> Hi Chia-Ping Tsai,
> >>
> >> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk
> seems to be using now (ducktape 0.11.4) We will not advance the version of
> ducktape in the 3.x branch, I think.
> >>
> >> cheers,
> >> Colin
> >>
> >>
> >> On Sat, Oct 12, 2024, at 01:49, Chia-Ping Tsai wrote:
> >> > hi Colin
> >> >
> >> > I'm sending this email to remind developers who plan to test E2E for
> RC2.
> >> >
> >> > There is an issue with the E2E tests due to the new release of
> >> > Ducktape. I've filed KAFKA-17781 to address it. Only developers who
> >> > recently rebuilt the E2E image will encounter this issue, so I assume
> >> > it is NOT a blocker for RC2.
> >> >
> >> > Best,
> >> > Chia-Ping
> >> >
> >> >
> >> > On 2024/10/10 21:14:55 Colin McCabe wrote:
> >> >> This is the second candidate for the release of Apache Kafka 3.9.0.
> I have titled it rc2 since I had an rc1 which got very far, even to the
> point of pushing tags and docker images, before I spotted an issue. So
> rather than mutate the tags, I decided to skip over rc1.
> >> >>
> >> >> - This is a major release, the final one in the 3.x line. (There may
> of course be other minor releases in this line, such as 3.9.1.)
> >> >> - Tiered storage will be considered production-ready in this release.
> >> >> - This will be the final major release to feature the deprecated
> ZooKeeper mode.
> >> >>
> >> >> This release includes the following KIPs:
> >> >> - KIP-853: Support dynamically changing KRaft controller membership
> >> >> - KIP-1057: Add remote log metadata flag to the dump log tool
> >> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
> >> >> - KIP-1040: Improve handling of nullable values in InsertField,
> ExtractField, and other transformations
> >> >> - KIP-1031: Control offset translation in MirrorSourceConnector
> >> >> - KIP-1033: Add Kafka Streams exception handler for exceptions
> occurring during processing
> >> >> - KIP-1017: Health check endpoint for Kafka Connect
> >> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in
> authorization header
> >> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> >> >> - KIP-950: Tiered Storage Disablement
> >> >> - KIP-956: Tiered Storage Quotas
> >> >>
> >> >> Release notes for the 3.9.0 release:
> >> >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
> >> >>
> >> >> *** Please download, test and vote by October 16, 2024.
> >> >>
> >> >> Kafka's KEYS file containing PGP keys we use to sign the release:
> >> >> https://kafka.apache.org/KEYS
> >> >>
> >> >> * Release artifacts to be voted upon (source and binary):
> >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
> >> >>
> >> >> * Docker release artifacts to be voted upon:
> >> >> apache/kafka:3.9.0-rc2
> >> >> apache/kafka-native:3.9.0-rc2
> >> >>
> >> >> * Maven artifacts to be voted upon:
> >> >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >> >>
> >> >> * Javadoc:
> >> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
> >> >>
> >> >> * Documentation:
> >> >> https://kafka.apache.org/39/documentation.html
> >> >>
> >> >> * Protocol:
> >> >> https://kafka.apache.org/39/protocol.html
> >> >>
> >> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
> >> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
> >> >>
> >> >> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
> >> >> Docker Build Test Pipeline (JVM):
> https://github.com/apache/kafka/actions/runs/11281563007
> >> >> Docker Build Test Pipeline (Native):
> https://github.com/apache/kafka/actions/runs/11281608809
> >

Re: [VOTE] 3.9.0 RC2

[Federico Valeri]

On Mon, Oct 14, 2024 at 5:11 PM Colin McCabe  wrote:
>
> Hi Federico,
>
> Thanks for taking a look! Can you please file a JIRA for the documentation 
> changes you'd like to make?
>

There was already a Jira about this, which was closed my mistake:

https://issues.apache.org/jira/browse/KAFKA-17031

I opened a PR to fix it:

https://github.com/apache/kafka/pull/17499

> best,
> Colin
>
>
> On Sun, Oct 13, 2024, at 09:21, Federico Valeri wrote:
> > Hi Colin, thanks for putting out the new RC.
> >
> > According to KIP-950, remote.log.manager.thread.pool.size should be
> > marked as deprecated and replaced by two new configurations:
> > remote.log.manager.copier.thread.pool.size and
> > remote.log.manager.expiration.thread.pool.size. I don't see that
> > reflected in the documentation, but I guess it can be fixed without
> > another RC.
> >
> >
> >
> >
> >
> > On Sat, Oct 12, 2024 at 10:50 AM Chia-Ping Tsai  wrote:
> >>
> >> hi Colin
> >>
> >> I'm sending this email to remind developers who plan to test E2E for RC2.
> >>
> >> There is an issue with the E2E tests due to the new release of Ducktape. 
> >> I've filed KAFKA-17781 to address it. Only developers who recently rebuilt 
> >> the E2E image will encounter this issue, so I assume it is NOT a blocker 
> >> for RC2.
> >>
> >> Best,
> >> Chia-Ping
> >>
> >>
> >> On 2024/10/10 21:14:55 Colin McCabe wrote:
> >> > This is the second candidate for the release of Apache Kafka 3.9.0. I 
> >> > have titled it rc2 since I had an rc1 which got very far, even to the 
> >> > point of pushing tags and docker images, before I spotted an issue. So 
> >> > rather than mutate the tags, I decided to skip over rc1.
> >> >
> >> > - This is a major release, the final one in the 3.x line. (There may of 
> >> > course be other minor releases in this line, such as 3.9.1.)
> >> > - Tiered storage will be considered production-ready in this release.
> >> > - This will be the final major release to feature the deprecated 
> >> > ZooKeeper mode.
> >> >
> >> > This release includes the following KIPs:
> >> > - KIP-853: Support dynamically changing KRaft controller membership
> >> > - KIP-1057: Add remote log metadata flag to the dump log tool
> >> > - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
> >> > - KIP-1040: Improve handling of nullable values in InsertField, 
> >> > ExtractField, and other transformations
> >> > - KIP-1031: Control offset translation in MirrorSourceConnector
> >> > - KIP-1033: Add Kafka Streams exception handler for exceptions occurring 
> >> > during processing
> >> > - KIP-1017: Health check endpoint for Kafka Connect
> >> > - KIP-1025: Optionally URL-encode clientID and clientSecret in 
> >> > authorization header
> >> > - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> >> > - KIP-950: Tiered Storage Disablement
> >> > - KIP-956: Tiered Storage Quotas
> >> >
> >> > Release notes for the 3.9.0 release:
> >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
> >> >
> >> > *** Please download, test and vote by October 16, 2024.
> >> >
> >> > Kafka's KEYS file containing PGP keys we use to sign the release:
> >> > https://kafka.apache.org/KEYS
> >> >
> >> > * Release artifacts to be voted upon (source and binary):
> >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
> >> >
> >> > * Docker release artifacts to be voted upon:
> >> > apache/kafka:3.9.0-rc2
> >> > apache/kafka-native:3.9.0-rc2
> >> >
> >> > * Maven artifacts to be voted upon:
> >> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >> >
> >> > * Javadoc:
> >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
> >> >
> >> > * Documentation:
> >> > https://kafka.apache.org/39/documentation.html
> >> >
> >> > * Protocol:
> >> > https://kafka.apache.org/39/protocol.html
> >> >
> >> > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
> >> > https://github.com/apache/kafka/releases/tag/3.9.0-rc2
> >> >
> >> > * Successful Docker Image Github Actions Pipeline for 3.9 branch:
> >> > Docker Build Test Pipeline (JVM): 
> >> > https://github.com/apache/kafka/actions/runs/11281563007
> >> > Docker Build Test Pipeline (Native): 
> >> > https://github.com/apache/kafka/actions/runs/11281608809
> >> >
> >> > Thanks to everyone who helped with this release candidate, either by 
> >> > contributing code, testing, or documentation.
> >> >
> >> > Regards,
> >> > Colin
> >> >

[jira] [Resolved] (KAFKA-17520) align the low bound of ducktape version

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17520.

Resolution: Fixed

> align the low bound of ducktape version 
> 
>
> Key: KAFKA-17520
> URL: https://issues.apache.org/jira/browse/KAFKA-17520
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Eric Chang
>Priority: Minor
> Fix For: 4.0.0, 3.7.2, 3.9.1
>
>
> There was a discussion for it: 
> https://github.com/apache/kafka/pull/14216#issuecomment-2117519600
> and I noticed this issue when trying to upgrade base image for e2e



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Reopened] (KAFKA-17031) Make configurations public

[Federico Valeri (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Federico Valeri reopened KAFKA-17031:
-

> Make configurations public
> --
>
> Key: KAFKA-17031
> URL: https://issues.apache.org/jira/browse/KAFKA-17031
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Christo Lolov
>Priority: Major
> Fix For: 3.9.0
>
>
> *Summary*
> We introduce 3 new configurations as part of this KIP. Up to now they have 
> been defined as internal only. We need to change said configurations to 
> public prior to the release.
>  
> Also need to deprecate "remote.log.manager.thread.pool.size" config.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-17763) Remove Utils.covariantCast

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17763.

Fix Version/s: 4.0.0
   Resolution: Fixed

> Remove Utils.covariantCast
> --
>
> Key: KAFKA-17763
> URL: https://issues.apache.org/jira/browse/KAFKA-17763
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Logan Zhu
>Priority: Major
> Fix For: 4.0.0
>
>
> we can use `var` to replace casting.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-17782) Fix ConsumerProtocolTest#deserializeOldSubscriptionVersion

[Lin Siyuan (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lin Siyuan resolved KAFKA-17782.

Resolution: Done

> Fix ConsumerProtocolTest#deserializeOldSubscriptionVersion
> --
>
> Key: KAFKA-17782
> URL: https://issues.apache.org/jira/browse/KAFKA-17782
> Project: Kafka
>  Issue Type: Test
>  Components: clients
>Reporter: Lin Siyuan
>Assignee: Lin Siyuan
>Priority: Major
>
> The expected code should be `assertEquals(toSet(subscription.topics()), 
> toSet(parsedSubscription.topics()));`
> https://github.com/apache/kafka/blob/ca422268649136b7207c0843fea28d594c761fe9/clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerProtocolTest.java#L156



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-17789) State updater stuck when starting with empty state folder

[Antoine Michaud (Jira)]

Antoine Michaud created KAFKA-17789:
---

 Summary: State updater stuck when starting with empty state folder
 Key: KAFKA-17789
 URL: https://issues.apache.org/jira/browse/KAFKA-17789
 Project: Kafka
  Issue Type: Bug
Reporter: Antoine Michaud






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Hi Chia-Ping Tsai,

Thanks for looking into this. I approved the PR.

best,
Colin

On Mon, Oct 14, 2024, at 08:34, Chia-Ping Tsai wrote:
>> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems 
>> to be using now (ducktape 0.11.4) We will not advance the version of 
>> ducktape in the 3.x branch, I think.
>
> Yes, I agree not to advance the version of Ducktape, but the Ducktape 
> version was unpinned from 3.2 (see 
> https://github.com/apache/kafka/commit/7e683852b46c0fe70f5100e953e3cd61d1697c06),
>  
> so I suspect 3.9 hasn't been using Ducktape 0.8.14 for a while.
>
> The PR (https://github.com/apache/kafka/pull/17481) locks Ducktape to 
> version 0.11.4, which is the latest release that works well with the 
> 3.9 Dockerfile (and it is waiting for your review). Additionally, the 
> trunk can use Ducktape 0.12.0, which supports newer Python versions, 
> allowing us to upgrade the base image without relying on outdated 
> Python versions.
>
> Best,
> Chia-Ping
>
> On 2024/10/14 15:08:06 Colin McCabe wrote:
>> Hi Chia-Ping Tsai,
>> 
>> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems 
>> to be using now (ducktape 0.11.4) We will not advance the version of 
>> ducktape in the 3.x branch, I think.
>> 
>> cheers,
>> Colin
>> 
>> 
>> On Sat, Oct 12, 2024, at 01:49, Chia-Ping Tsai wrote:
>> > hi Colin
>> >
>> > I'm sending this email to remind developers who plan to test E2E for RC2.
>> >
>> > There is an issue with the E2E tests due to the new release of 
>> > Ducktape. I've filed KAFKA-17781 to address it. Only developers who 
>> > recently rebuilt the E2E image will encounter this issue, so I assume 
>> > it is NOT a blocker for RC2.
>> >
>> > Best,
>> > Chia-Ping
>> >
>> >
>> > On 2024/10/10 21:14:55 Colin McCabe wrote:
>> >> This is the second candidate for the release of Apache Kafka 3.9.0. I 
>> >> have titled it rc2 since I had an rc1 which got very far, even to the 
>> >> point of pushing tags and docker images, before I spotted an issue. So 
>> >> rather than mutate the tags, I decided to skip over rc1.
>> >> 
>> >> - This is a major release, the final one in the 3.x line. (There may of 
>> >> course be other minor releases in this line, such as 3.9.1.)
>> >> - Tiered storage will be considered production-ready in this release.
>> >> - This will be the final major release to feature the deprecated 
>> >> ZooKeeper mode.
>> >> 
>> >> This release includes the following KIPs:
>> >> - KIP-853: Support dynamically changing KRaft controller membership
>> >> - KIP-1057: Add remote log metadata flag to the dump log tool
>> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> >> - KIP-1040: Improve handling of nullable values in InsertField, 
>> >> ExtractField, and other transformations
>> >> - KIP-1031: Control offset translation in MirrorSourceConnector
>> >> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring 
>> >> during processing
>> >> - KIP-1017: Health check endpoint for Kafka Connect
>> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in 
>> >> authorization header
>> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> >> - KIP-950: Tiered Storage Disablement
>> >> - KIP-956: Tiered Storage Quotas
>> >> 
>> >> Release notes for the 3.9.0 release:
>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>> >> 
>> >> *** Please download, test and vote by October 16, 2024.
>> >> 
>> >> Kafka's KEYS file containing PGP keys we use to sign the release:
>> >> https://kafka.apache.org/KEYS
>> >> 
>> >> * Release artifacts to be voted upon (source and binary):
>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>> >> 
>> >> * Docker release artifacts to be voted upon:
>> >> apache/kafka:3.9.0-rc2
>> >> apache/kafka-native:3.9.0-rc2
>> >> 
>> >> * Maven artifacts to be voted upon:
>> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >> 
>> >> * Javadoc:
>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>> >> 
>> >> * Documentation:
>> >> https://kafka.apache.org/39/documentation.html
>> >> 
>> >> * Protocol:
>> >> https://kafka.apache.org/39/protocol.html
>> >> 
>> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
>> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
>> >> 
>> >> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
>> >> Docker Build Test Pipeline (JVM): 
>> >> https://github.com/apache/kafka/actions/runs/11281563007
>> >> Docker Build Test Pipeline (Native): 
>> >> https://github.com/apache/kafka/actions/runs/11281608809
>> >> 
>> >> Thanks to everyone who helped with this release candidate, either by 
>> >> contributing code, testing, or documentation.
>> >> 
>> >> Regards,
>> >> Colin
>> >>
>>

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Hi Jakub,

Thanks for testing. control.plane.listener is not (and never has been) 
supported in KRaft mode. You have to remove control.plane.listener 
configurations before migrating. I filed KAFKA-17790 to document this in the 
migration instructions. (This is not a blocker for the release, though.)

best,
Colin

On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
> Hi Colin,
>
> Thanks for the RC. I did some testing of it and run into
> https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
> regression in the migration to KRaft process.
>
> Can someone who understands this part of the codebase look into it please?
>
> Thanks & Regards
> Jakub
>
> On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe  wrote:
>
>> This is the second candidate for the release of Apache Kafka 3.9.0. I have
>> titled it rc2 since I had an rc1 which got very far, even to the point of
>> pushing tags and docker images, before I spotted an issue. So rather than
>> mutate the tags, I decided to skip over rc1.
>>
>> - This is a major release, the final one in the 3.x line. (There may of
>> course be other minor releases in this line, such as 3.9.1.)
>> - Tiered storage will be considered production-ready in this release.
>> - This will be the final major release to feature the deprecated ZooKeeper
>> mode.
>>
>> This release includes the following KIPs:
>> - KIP-853: Support dynamically changing KRaft controller membership
>> - KIP-1057: Add remote log metadata flag to the dump log tool
>> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> - KIP-1040: Improve handling of nullable values in InsertField,
>> ExtractField, and other transformations
>> - KIP-1031: Control offset translation in MirrorSourceConnector
>> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring
>> during processing
>> - KIP-1017: Health check endpoint for Kafka Connect
>> - KIP-1025: Optionally URL-encode clientID and clientSecret in
>> authorization header
>> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> - KIP-950: Tiered Storage Disablement
>> - KIP-956: Tiered Storage Quotas
>>
>> Release notes for the 3.9.0 release:
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by October 16, 2024.
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>>
>> * Docker release artifacts to be voted upon:
>> apache/kafka:3.9.0-rc2
>> apache/kafka-native:3.9.0-rc2
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>
>> * Javadoc:
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>>
>> * Documentation:
>> https://kafka.apache.org/39/documentation.html
>>
>> * Protocol:
>> https://kafka.apache.org/39/protocol.html
>>
>> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
>> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
>>
>> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
>> Docker Build Test Pipeline (JVM):
>> https://github.com/apache/kafka/actions/runs/11281563007
>> Docker Build Test Pipeline (Native):
>> https://github.com/apache/kafka/actions/runs/11281608809
>>
>> Thanks to everyone who helped with this release candidate, either by
>> contributing code, testing, or documentation.
>>
>> Regards,
>> Colin
>>

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Hi Chia-Ping Tsai,

Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems to 
be using now (ducktape 0.11.4) We will not advance the version of ducktape in 
the 3.x branch, I think.

cheers,
Colin


On Sat, Oct 12, 2024, at 01:49, Chia-Ping Tsai wrote:
> hi Colin
>
> I'm sending this email to remind developers who plan to test E2E for RC2.
>
> There is an issue with the E2E tests due to the new release of 
> Ducktape. I've filed KAFKA-17781 to address it. Only developers who 
> recently rebuilt the E2E image will encounter this issue, so I assume 
> it is NOT a blocker for RC2.
>
> Best,
> Chia-Ping
>
>
> On 2024/10/10 21:14:55 Colin McCabe wrote:
>> This is the second candidate for the release of Apache Kafka 3.9.0. I have 
>> titled it rc2 since I had an rc1 which got very far, even to the point of 
>> pushing tags and docker images, before I spotted an issue. So rather than 
>> mutate the tags, I decided to skip over rc1.
>> 
>> - This is a major release, the final one in the 3.x line. (There may of 
>> course be other minor releases in this line, such as 3.9.1.)
>> - Tiered storage will be considered production-ready in this release.
>> - This will be the final major release to feature the deprecated ZooKeeper 
>> mode.
>> 
>> This release includes the following KIPs:
>> - KIP-853: Support dynamically changing KRaft controller membership
>> - KIP-1057: Add remote log metadata flag to the dump log tool
>> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> - KIP-1040: Improve handling of nullable values in InsertField, 
>> ExtractField, and other transformations
>> - KIP-1031: Control offset translation in MirrorSourceConnector
>> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring 
>> during processing
>> - KIP-1017: Health check endpoint for Kafka Connect
>> - KIP-1025: Optionally URL-encode clientID and clientSecret in authorization 
>> header
>> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> - KIP-950: Tiered Storage Disablement
>> - KIP-956: Tiered Storage Quotas
>> 
>> Release notes for the 3.9.0 release:
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>> 
>> *** Please download, test and vote by October 16, 2024.
>> 
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>> 
>> * Release artifacts to be voted upon (source and binary):
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>> 
>> * Docker release artifacts to be voted upon:
>> apache/kafka:3.9.0-rc2
>> apache/kafka-native:3.9.0-rc2
>> 
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> 
>> * Javadoc:
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>> 
>> * Documentation:
>> https://kafka.apache.org/39/documentation.html
>> 
>> * Protocol:
>> https://kafka.apache.org/39/protocol.html
>> 
>> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
>> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
>> 
>> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
>> Docker Build Test Pipeline (JVM): 
>> https://github.com/apache/kafka/actions/runs/11281563007
>> Docker Build Test Pipeline (Native): 
>> https://github.com/apache/kafka/actions/runs/11281608809
>> 
>> Thanks to everyone who helped with this release candidate, either by 
>> contributing code, testing, or documentation.
>> 
>> Regards,
>> Colin
>>

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Hi Federico,

Thanks for taking a look! Can you please file a JIRA for the documentation 
changes you'd like to make?

best,
Colin


On Sun, Oct 13, 2024, at 09:21, Federico Valeri wrote:
> Hi Colin, thanks for putting out the new RC.
>
> According to KIP-950, remote.log.manager.thread.pool.size should be
> marked as deprecated and replaced by two new configurations:
> remote.log.manager.copier.thread.pool.size and
> remote.log.manager.expiration.thread.pool.size. I don't see that
> reflected in the documentation, but I guess it can be fixed without
> another RC.
>
>
>
>
>
> On Sat, Oct 12, 2024 at 10:50 AM Chia-Ping Tsai  wrote:
>>
>> hi Colin
>>
>> I'm sending this email to remind developers who plan to test E2E for RC2.
>>
>> There is an issue with the E2E tests due to the new release of Ducktape. 
>> I've filed KAFKA-17781 to address it. Only developers who recently rebuilt 
>> the E2E image will encounter this issue, so I assume it is NOT a blocker for 
>> RC2.
>>
>> Best,
>> Chia-Ping
>>
>>
>> On 2024/10/10 21:14:55 Colin McCabe wrote:
>> > This is the second candidate for the release of Apache Kafka 3.9.0. I have 
>> > titled it rc2 since I had an rc1 which got very far, even to the point of 
>> > pushing tags and docker images, before I spotted an issue. So rather than 
>> > mutate the tags, I decided to skip over rc1.
>> >
>> > - This is a major release, the final one in the 3.x line. (There may of 
>> > course be other minor releases in this line, such as 3.9.1.)
>> > - Tiered storage will be considered production-ready in this release.
>> > - This will be the final major release to feature the deprecated ZooKeeper 
>> > mode.
>> >
>> > This release includes the following KIPs:
>> > - KIP-853: Support dynamically changing KRaft controller membership
>> > - KIP-1057: Add remote log metadata flag to the dump log tool
>> > - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> > - KIP-1040: Improve handling of nullable values in InsertField, 
>> > ExtractField, and other transformations
>> > - KIP-1031: Control offset translation in MirrorSourceConnector
>> > - KIP-1033: Add Kafka Streams exception handler for exceptions occurring 
>> > during processing
>> > - KIP-1017: Health check endpoint for Kafka Connect
>> > - KIP-1025: Optionally URL-encode clientID and clientSecret in 
>> > authorization header
>> > - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> > - KIP-950: Tiered Storage Disablement
>> > - KIP-956: Tiered Storage Quotas
>> >
>> > Release notes for the 3.9.0 release:
>> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>> >
>> > *** Please download, test and vote by October 16, 2024.
>> >
>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > https://kafka.apache.org/KEYS
>> >
>> > * Release artifacts to be voted upon (source and binary):
>> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>> >
>> > * Docker release artifacts to be voted upon:
>> > apache/kafka:3.9.0-rc2
>> > apache/kafka-native:3.9.0-rc2
>> >
>> > * Maven artifacts to be voted upon:
>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >
>> > * Javadoc:
>> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>> >
>> > * Documentation:
>> > https://kafka.apache.org/39/documentation.html
>> >
>> > * Protocol:
>> > https://kafka.apache.org/39/protocol.html
>> >
>> > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
>> > https://github.com/apache/kafka/releases/tag/3.9.0-rc2
>> >
>> > * Successful Docker Image Github Actions Pipeline for 3.9 branch:
>> > Docker Build Test Pipeline (JVM): 
>> > https://github.com/apache/kafka/actions/runs/11281563007
>> > Docker Build Test Pipeline (Native): 
>> > https://github.com/apache/kafka/actions/runs/11281608809
>> >
>> > Thanks to everyone who helped with this release candidate, either by 
>> > contributing code, testing, or documentation.
>> >
>> > Regards,
>> > Colin
>> >

[jira] [Created] (KAFKA-17790) Document that control.plane.listener should be removed before ZK migration

[Colin McCabe (Jira)]

Colin McCabe created KAFKA-17790:


 Summary: Document that control.plane.listener should be removed 
before ZK migration
 Key: KAFKA-17790
 URL: https://issues.apache.org/jira/browse/KAFKA-17790
 Project: Kafka
  Issue Type: Bug
Affects Versions: 3.7.2, 3.8.1, 3.9.1
Reporter: Colin McCabe
Assignee: Colin McCabe






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-17792) header parsing ends up timing out and using large quantities of memory if the string looks like a number

[Martin Sillence (Jira)]

Martin Sillence created KAFKA-17792:
---

 Summary: header parsing ends up timing out and using large 
quantities of memory if the string looks like a number
 Key: KAFKA-17792
 URL: https://issues.apache.org/jira/browse/KAFKA-17792
 Project: Kafka
  Issue Type: Bug
  Components: connect
Reporter: Martin Sillence


{color:#172b4d}We have trace headers such as:{color}

{color:#172b4d}"X-B3-SpanId": "74320e6e26adc8f8"{color}

{color:#172b4d}if however the value happens to be: "407127e212797209"{color}

{color:#172b4d}This is then treated as a numeric value and it tries to convert 
this as a numeric representation and an exact value using BigDecimal{color}

we end up with the trace:

    BigDecimal.setScale(int, RoundingMode) line: 2876    
    Values$ValueParser.parseAsExactDecimal(BigDecimal) line: 1044    
    Values$ValueParser.parseAsNumber(String) line: 1025    
    Values$ValueParser.parseNextToken(boolean, String) line: 892    
    Values$ValueParser.parse(boolean) line: 875    
    Values.parseString(String) line: 415    
    SimpleHeaderConverter.toConnectHeader(String, String, byte[]) line: 68    
    WorkerSinkTask.convertHeadersFor(ConsumerRecord) line: 578   
 

this takes a long time to convert to an exact representation of a 212 billion 
digit integer



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-17788) Migration fails in Kafka 3.9.0-RC2 when control plane listener is used

[Colin McCabe (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin McCabe resolved KAFKA-17788.
--
Resolution: Invalid

> Migration fails in Kafka 3.9.0-RC2 when control plane listener is used
> --
>
> Key: KAFKA-17788
> URL: https://issues.apache.org/jira/browse/KAFKA-17788
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 3.9.0
>Reporter: Jakub Scholz
>Priority: Blocker
> Attachments: logs-pod-cluster-9833cba6-kafka-1-container-kafka.log
>
>
> When testing migration with Kafka 3.9.0-RC2, the broker fail to start when 
> they are first rolled to start the migration with the following error:
> {code}
> 2024-10-11 21:37:04,060 ERROR Exiting Kafka due to fatal exception 
> (kafka.Kafka$) [main]
> java.lang.IllegalArgumentException: requirement failed: 
> control.plane.listener.name must be a listener name defined in 
> advertised.listeners. The valid options based on currently configured 
> listeners are REPLICATION-9091,PLAIN-9092,TLS-9093
>   at scala.Predef$.require(Predef.scala:337)
>   at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1019)
>   at kafka.server.KafkaConfig.(KafkaConfig.scala:843)
>   at kafka.server.KafkaConfig.(KafkaConfig.scala:185)
>   at kafka.Kafka$.buildServer(Kafka.scala:71)
>   at kafka.Kafka$.main(Kafka.scala:90)
>   at kafka.Kafka.main(Kafka.scala)
> {code}
> This is despite our configuration having the {control.plane.listener.name} 
> properly configured:
> {code}
> listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,PLAIN-9092:SASL_PLAINTEXT,TLS-9093:SSL
> listeners=CONTROLPLANE-9090://0.0.0.0:9090,REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
> advertised.listeners=CONTROLPLANE-9090://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9090,REPLICATION-9091://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9091,PLAIN-9092://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9092,TLS-9093://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9093
> inter.broker.listener.name=REPLICATION-9091
> control.plane.listener.name=CONTROLPLANE-9090
> {code}
> It looks like 3.9.0-RC2 filters out the control plane listener (maybe because 
> it is used by the KRaft controllers as well?) and runs into this error. This 
> worked fine in 3.8.0, so this seems like a regression in 3.9.0 that should be 
> fixed.
> The full log from the broker node is attached. It includes the full 
> configuration of the broker as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Jakub Scholz]

The different name of the controller listener for KRaft controllers and
control plane listener in ZooKeeper-based cluster was not required before
and it is not simple to change to handle now at the "last minute". So given
that this is called production-ready already for some time, I think this is
breaking API change and should be treated as such.

Thanks & Regards
Jakub

On Mon, Oct 14, 2024 at 7:55 PM Colin McCabe  wrote:

> Hi Jakub,
>
> After looking through the attached file on the JIRA, I can say that this
> is a misconfiguration. control.plane.listener is a totally separate concept
> from control.plane.listener.name. They should never be set to the same
> value. The controller listener must have a different name and value than
> the control plane listener (if any).
>
> I also tested myself that KRaft migration works with
> control.plane.listener configured. It works on both 3.8 and 3.9-RC2.
>
> My initial statement that control.plane.listener was not supported during
> ZK migration was incorrect. As you said, it is supported during migration
> up to the point that we are in KRaft mode. (Another reason why having the
> control plane listener = controller listener would not make sense.)
>
> Thanks for the bug report and discussion. I've closed this as invalid now
> that I have tested migration using control.plane.listener for myself and
> verified that it works.
>
> best,
> Colin
>
> On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote:
> >> control.plane.listener is not (and never has been) supported in KRaft
> > mode.
> >
> > You mean control.plane.listener.name is not supported in KRaft I guess?
> > Well, this is not KRaft, this is migration, so it uses the settings that
> it
> > used before for the Zoo-based cluster and that includes using dedicated
> > control plane listener. I don't think I can "just remove it" because the
> > other nodes will use it during the rolling update.
> >
> > This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not
> supported
> > now, it is a breaking API change I guess which should be a blocker.
> >
> > Thanks & Regards
> > Jakub
> >
> > On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe  wrote:
> >
> >> Hi Jakub,
> >>
> >> Thanks for testing. control.plane.listener is not (and never has been)
> >> supported in KRaft mode. You have to remove control.plane.listener
> >> configurations before migrating. I filed KAFKA-17790 to document this in
> >> the migration instructions. (This is not a blocker for the release,
> though.)
> >>
> >> best,
> >> Colin
> >>
> >> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
> >> > Hi Colin,
> >> >
> >> > Thanks for the RC. I did some testing of it and run into
> >> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
> >> > regression in the migration to KRaft process.
> >> >
> >> > Can someone who understands this part of the codebase look into it
> >> please?
> >> >
> >> > Thanks & Regards
> >> > Jakub
> >> >
> >> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe 
> >> wrote:
> >> >
> >> >> This is the second candidate for the release of Apache Kafka 3.9.0. I
> >> have
> >> >> titled it rc2 since I had an rc1 which got very far, even to the
> point
> >> of
> >> >> pushing tags and docker images, before I spotted an issue. So rather
> >> than
> >> >> mutate the tags, I decided to skip over rc1.
> >> >>
> >> >> - This is a major release, the final one in the 3.x line. (There may
> of
> >> >> course be other minor releases in this line, such as 3.9.1.)
> >> >> - Tiered storage will be considered production-ready in this release.
> >> >> - This will be the final major release to feature the deprecated
> >> ZooKeeper
> >> >> mode.
> >> >>
> >> >> This release includes the following KIPs:
> >> >> - KIP-853: Support dynamically changing KRaft controller membership
> >> >> - KIP-1057: Add remote log metadata flag to the dump log tool
> >> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
> >> >> - KIP-1040: Improve handling of nullable values in InsertField,
> >> >> ExtractField, and other transformations
> >> >> - KIP-1031: Control offset translation in MirrorSourceConnector
> >> >> - KIP-1033: Add Kafka Streams exception handler for exceptions
> occurring
> >> >> during processing
> >> >> - KIP-1017: Health check endpoint for Kafka Connect
> >> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in
> >> >> authorization header
> >> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> >> >> - KIP-950: Tiered Storage Disablement
> >> >> - KIP-956: Tiered Storage Quotas
> >> >>
> >> >> Release notes for the 3.9.0 release:
> >> >>
> >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
> >> >>
> >> >> *** Please download, test and vote by October 16, 2024.
> >> >>
> >> >> Kafka's KEYS file containing PGP keys we use to sign the release:
> >> >> https://kafka.apache.org/KEYS
> >> >>
> >> >> * Release artifacts to be voted upon (source and binary):
> >> >> https://

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

I also posted a PR to add removing control.plane.listener to the docs:

https://github.com/apache/kafka/pull/17501

This needs to be done for each broker process before before it is restarted in 
KRaft mode. (However, it is fine to BEGIN the migration process with control 
plane listeners enabled.)

best,
Colin


On Mon, Oct 14, 2024, at 10:54, Colin McCabe wrote:
> Hi Jakub,
>
> After looking through the attached file on the JIRA, I can say that 
> this is a misconfiguration. control.plane.listener is a totally 
> separate concept from control.plane.listener.name. They should never be 
> set to the same value. The controller listener must have a different 
> name and value than the control plane listener (if any).
>
> I also tested myself that KRaft migration works with 
> control.plane.listener configured. It works on both 3.8 and 3.9-RC2.
>
> My initial statement that control.plane.listener was not supported 
> during ZK migration was incorrect. As you said, it is supported during 
> migration up to the point that we are in KRaft mode. (Another reason 
> why having the control plane listener = controller listener would not 
> make sense.)
>
> Thanks for the bug report and discussion. I've closed this as invalid 
> now that I have tested migration using control.plane.listener for 
> myself and verified that it works.
>
> best,
> Colin
>
> On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote:
>>> control.plane.listener is not (and never has been) supported in KRaft
>> mode.
>>
>> You mean control.plane.listener.name is not supported in KRaft I guess?
>> Well, this is not KRaft, this is migration, so it uses the settings that it
>> used before for the Zoo-based cluster and that includes using dedicated
>> control plane listener. I don't think I can "just remove it" because the
>> other nodes will use it during the rolling update.
>>
>> This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not supported
>> now, it is a breaking API change I guess which should be a blocker.
>>
>> Thanks & Regards
>> Jakub
>>
>> On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe  wrote:
>>
>>> Hi Jakub,
>>>
>>> Thanks for testing. control.plane.listener is not (and never has been)
>>> supported in KRaft mode. You have to remove control.plane.listener
>>> configurations before migrating. I filed KAFKA-17790 to document this in
>>> the migration instructions. (This is not a blocker for the release, though.)
>>>
>>> best,
>>> Colin
>>>
>>> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
>>> > Hi Colin,
>>> >
>>> > Thanks for the RC. I did some testing of it and run into
>>> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
>>> > regression in the migration to KRaft process.
>>> >
>>> > Can someone who understands this part of the codebase look into it
>>> please?
>>> >
>>> > Thanks & Regards
>>> > Jakub
>>> >
>>> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe 
>>> wrote:
>>> >
>>> >> This is the second candidate for the release of Apache Kafka 3.9.0. I
>>> have
>>> >> titled it rc2 since I had an rc1 which got very far, even to the point
>>> of
>>> >> pushing tags and docker images, before I spotted an issue. So rather
>>> than
>>> >> mutate the tags, I decided to skip over rc1.
>>> >>
>>> >> - This is a major release, the final one in the 3.x line. (There may of
>>> >> course be other minor releases in this line, such as 3.9.1.)
>>> >> - Tiered storage will be considered production-ready in this release.
>>> >> - This will be the final major release to feature the deprecated
>>> ZooKeeper
>>> >> mode.
>>> >>
>>> >> This release includes the following KIPs:
>>> >> - KIP-853: Support dynamically changing KRaft controller membership
>>> >> - KIP-1057: Add remote log metadata flag to the dump log tool
>>> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>>> >> - KIP-1040: Improve handling of nullable values in InsertField,
>>> >> ExtractField, and other transformations
>>> >> - KIP-1031: Control offset translation in MirrorSourceConnector
>>> >> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring
>>> >> during processing
>>> >> - KIP-1017: Health check endpoint for Kafka Connect
>>> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in
>>> >> authorization header
>>> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>>> >> - KIP-950: Tiered Storage Disablement
>>> >> - KIP-956: Tiered Storage Quotas
>>> >>
>>> >> Release notes for the 3.9.0 release:
>>> >>
>>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>>> >>
>>> >> *** Please download, test and vote by October 16, 2024.
>>> >>
>>> >> Kafka's KEYS file containing PGP keys we use to sign the release:
>>> >> https://kafka.apache.org/KEYS
>>> >>
>>> >> * Release artifacts to be voted upon (source and binary):
>>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>>> >>
>>> >> * Docker release artifacts to be voted upon:
>>> >> apache/kafka:3.9.0-rc2
>>> >> apa

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Thanks, Federico.

best,
Colin


On Mon, Oct 14, 2024, at 09:23, Federico Valeri wrote:
> On Mon, Oct 14, 2024 at 5:11 PM Colin McCabe  wrote:
>>
>> Hi Federico,
>>
>> Thanks for taking a look! Can you please file a JIRA for the documentation 
>> changes you'd like to make?
>>
>
> There was already a Jira about this, which was closed my mistake:
>
> https://issues.apache.org/jira/browse/KAFKA-17031
>
> I opened a PR to fix it:
>
> https://github.com/apache/kafka/pull/17499
>
>> best,
>> Colin
>>
>>
>> On Sun, Oct 13, 2024, at 09:21, Federico Valeri wrote:
>> > Hi Colin, thanks for putting out the new RC.
>> >
>> > According to KIP-950, remote.log.manager.thread.pool.size should be
>> > marked as deprecated and replaced by two new configurations:
>> > remote.log.manager.copier.thread.pool.size and
>> > remote.log.manager.expiration.thread.pool.size. I don't see that
>> > reflected in the documentation, but I guess it can be fixed without
>> > another RC.
>> >
>> >
>> >
>> >
>> >
>> > On Sat, Oct 12, 2024 at 10:50 AM Chia-Ping Tsai  
>> > wrote:
>> >>
>> >> hi Colin
>> >>
>> >> I'm sending this email to remind developers who plan to test E2E for RC2.
>> >>
>> >> There is an issue with the E2E tests due to the new release of Ducktape. 
>> >> I've filed KAFKA-17781 to address it. Only developers who recently 
>> >> rebuilt the E2E image will encounter this issue, so I assume it is NOT a 
>> >> blocker for RC2.
>> >>
>> >> Best,
>> >> Chia-Ping
>> >>
>> >>
>> >> On 2024/10/10 21:14:55 Colin McCabe wrote:
>> >> > This is the second candidate for the release of Apache Kafka 3.9.0. I 
>> >> > have titled it rc2 since I had an rc1 which got very far, even to the 
>> >> > point of pushing tags and docker images, before I spotted an issue. So 
>> >> > rather than mutate the tags, I decided to skip over rc1.
>> >> >
>> >> > - This is a major release, the final one in the 3.x line. (There may of 
>> >> > course be other minor releases in this line, such as 3.9.1.)
>> >> > - Tiered storage will be considered production-ready in this release.
>> >> > - This will be the final major release to feature the deprecated 
>> >> > ZooKeeper mode.
>> >> >
>> >> > This release includes the following KIPs:
>> >> > - KIP-853: Support dynamically changing KRaft controller membership
>> >> > - KIP-1057: Add remote log metadata flag to the dump log tool
>> >> > - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> >> > - KIP-1040: Improve handling of nullable values in InsertField, 
>> >> > ExtractField, and other transformations
>> >> > - KIP-1031: Control offset translation in MirrorSourceConnector
>> >> > - KIP-1033: Add Kafka Streams exception handler for exceptions 
>> >> > occurring during processing
>> >> > - KIP-1017: Health check endpoint for Kafka Connect
>> >> > - KIP-1025: Optionally URL-encode clientID and clientSecret in 
>> >> > authorization header
>> >> > - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> >> > - KIP-950: Tiered Storage Disablement
>> >> > - KIP-956: Tiered Storage Quotas
>> >> >
>> >> > Release notes for the 3.9.0 release:
>> >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>> >> >
>> >> > *** Please download, test and vote by October 16, 2024.
>> >> >
>> >> > Kafka's KEYS file containing PGP keys we use to sign the release:
>> >> > https://kafka.apache.org/KEYS
>> >> >
>> >> > * Release artifacts to be voted upon (source and binary):
>> >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>> >> >
>> >> > * Docker release artifacts to be voted upon:
>> >> > apache/kafka:3.9.0-rc2
>> >> > apache/kafka-native:3.9.0-rc2
>> >> >
>> >> > * Maven artifacts to be voted upon:
>> >> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >> >
>> >> > * Javadoc:
>> >> > https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>> >> >
>> >> > * Documentation:
>> >> > https://kafka.apache.org/39/documentation.html
>> >> >
>> >> > * Protocol:
>> >> > https://kafka.apache.org/39/protocol.html
>> >> >
>> >> > * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
>> >> > https://github.com/apache/kafka/releases/tag/3.9.0-rc2
>> >> >
>> >> > * Successful Docker Image Github Actions Pipeline for 3.9 branch:
>> >> > Docker Build Test Pipeline (JVM): 
>> >> > https://github.com/apache/kafka/actions/runs/11281563007
>> >> > Docker Build Test Pipeline (Native): 
>> >> > https://github.com/apache/kafka/actions/runs/11281608809
>> >> >
>> >> > Thanks to everyone who helped with this release candidate, either by 
>> >> > contributing code, testing, or documentation.
>> >> >
>> >> > Regards,
>> >> > Colin
>> >> >

[jira] [Created] (KAFKA-17793) Improve kcontroller robustness against long delays

[Colin McCabe (Jira)]

Colin McCabe created KAFKA-17793:


 Summary: Improve kcontroller robustness against long delays
 Key: KAFKA-17793
 URL: https://issues.apache.org/jira/browse/KAFKA-17793
 Project: Kafka
  Issue Type: Bug
Reporter: Colin McCabe
Assignee: Colin McCabe


As described in KIP-500, the Kafka controller monitors the liveness of each 
broker in the cluster. It gathers this information from heartbeats sent from 
the brokers themselves.

In some rare cases, the main controller thread may get blocked for several 
seconds at a time. In the current code, this will result in the controller 
being unable to update the last contact times for the brokers during this time.

This PR changes the controller heartbeat handling to be partially lockless. 
Specifically, the last contact time for each broker will be updated locklessly 
prior to the rest of the heartbeat handling. This will ensure that heartbeats 
always get through.

Additionally, this PR adds a PeriodicTaskControlManager to better manage 
periodic tasks. This should help handle the very common pattern where we want 
to schedule a background task at some frequency. We also want the background 
task to be immediately rescheduled if there is too much work to be done in one 
event.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Colin McCabe]

Hi Jakub,

After looking through the attached file on the JIRA, I can say that this is a 
misconfiguration. control.plane.listener is a totally separate concept from 
control.plane.listener.name. They should never be set to the same value. The 
controller listener must have a different name and value than the control plane 
listener (if any).

I also tested myself that KRaft migration works with control.plane.listener 
configured. It works on both 3.8 and 3.9-RC2.

My initial statement that control.plane.listener was not supported during ZK 
migration was incorrect. As you said, it is supported during migration up to 
the point that we are in KRaft mode. (Another reason why having the control 
plane listener = controller listener would not make sense.)

Thanks for the bug report and discussion. I've closed this as invalid now that 
I have tested migration using control.plane.listener for myself and verified 
that it works.

best,
Colin

On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote:
>> control.plane.listener is not (and never has been) supported in KRaft
> mode.
>
> You mean control.plane.listener.name is not supported in KRaft I guess?
> Well, this is not KRaft, this is migration, so it uses the settings that it
> used before for the Zoo-based cluster and that includes using dedicated
> control plane listener. I don't think I can "just remove it" because the
> other nodes will use it during the rolling update.
>
> This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not supported
> now, it is a breaking API change I guess which should be a blocker.
>
> Thanks & Regards
> Jakub
>
> On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe  wrote:
>
>> Hi Jakub,
>>
>> Thanks for testing. control.plane.listener is not (and never has been)
>> supported in KRaft mode. You have to remove control.plane.listener
>> configurations before migrating. I filed KAFKA-17790 to document this in
>> the migration instructions. (This is not a blocker for the release, though.)
>>
>> best,
>> Colin
>>
>> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
>> > Hi Colin,
>> >
>> > Thanks for the RC. I did some testing of it and run into
>> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
>> > regression in the migration to KRaft process.
>> >
>> > Can someone who understands this part of the codebase look into it
>> please?
>> >
>> > Thanks & Regards
>> > Jakub
>> >
>> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe 
>> wrote:
>> >
>> >> This is the second candidate for the release of Apache Kafka 3.9.0. I
>> have
>> >> titled it rc2 since I had an rc1 which got very far, even to the point
>> of
>> >> pushing tags and docker images, before I spotted an issue. So rather
>> than
>> >> mutate the tags, I decided to skip over rc1.
>> >>
>> >> - This is a major release, the final one in the 3.x line. (There may of
>> >> course be other minor releases in this line, such as 3.9.1.)
>> >> - Tiered storage will be considered production-ready in this release.
>> >> - This will be the final major release to feature the deprecated
>> ZooKeeper
>> >> mode.
>> >>
>> >> This release includes the following KIPs:
>> >> - KIP-853: Support dynamically changing KRaft controller membership
>> >> - KIP-1057: Add remote log metadata flag to the dump log tool
>> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
>> >> - KIP-1040: Improve handling of nullable values in InsertField,
>> >> ExtractField, and other transformations
>> >> - KIP-1031: Control offset translation in MirrorSourceConnector
>> >> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring
>> >> during processing
>> >> - KIP-1017: Health check endpoint for Kafka Connect
>> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in
>> >> authorization header
>> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
>> >> - KIP-950: Tiered Storage Disablement
>> >> - KIP-956: Tiered Storage Quotas
>> >>
>> >> Release notes for the 3.9.0 release:
>> >>
>> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
>> >>
>> >> *** Please download, test and vote by October 16, 2024.
>> >>
>> >> Kafka's KEYS file containing PGP keys we use to sign the release:
>> >> https://kafka.apache.org/KEYS
>> >>
>> >> * Release artifacts to be voted upon (source and binary):
>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
>> >>
>> >> * Docker release artifacts to be voted upon:
>> >> apache/kafka:3.9.0-rc2
>> >> apache/kafka-native:3.9.0-rc2
>> >>
>> >> * Maven artifacts to be voted upon:
>> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >>
>> >> * Javadoc:
>> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
>> >>
>> >> * Documentation:
>> >> https://kafka.apache.org/39/documentation.html
>> >>
>> >> * Protocol:
>> >> https://kafka.apache.org/39/protocol.html
>> >>
>> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
>> >> https://github.com/apa

[jira] [Resolved] (KAFKA-17765) Remove TestUtils.toSet

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-17765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved KAFKA-17765.

Fix Version/s: 4.0.0
   Resolution: Fixed

> Remove TestUtils.toSet
> --
>
> Key: KAFKA-17765
> URL: https://issues.apache.org/jira/browse/KAFKA-17765
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Lin Siyuan
>Priority: Major
> Fix For: 4.0.0
>
>
> It can be replaced by either `Set.of` or `Set.copyOf` 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] 3.9.0 RC2

[Jakub Scholz]

> control.plane.listener is not (and never has been) supported in KRaft
mode.

You mean control.plane.listener.name is not supported in KRaft I guess?
Well, this is not KRaft, this is migration, so it uses the settings that it
used before for the Zoo-based cluster and that includes using dedicated
control plane listener. I don't think I can "just remove it" because the
other nodes will use it during the rolling update.

This also worked fine with 3.8 (and 3.7, etc.) -> so if it is not supported
now, it is a breaking API change I guess which should be a blocker.

Thanks & Regards
Jakub

On Mon, Oct 14, 2024 at 5:12 PM Colin McCabe  wrote:

> Hi Jakub,
>
> Thanks for testing. control.plane.listener is not (and never has been)
> supported in KRaft mode. You have to remove control.plane.listener
> configurations before migrating. I filed KAFKA-17790 to document this in
> the migration instructions. (This is not a blocker for the release, though.)
>
> best,
> Colin
>
> On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote:
> > Hi Colin,
> >
> > Thanks for the RC. I did some testing of it and run into
> > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
> > regression in the migration to KRaft process.
> >
> > Can someone who understands this part of the codebase look into it
> please?
> >
> > Thanks & Regards
> > Jakub
> >
> > On Thu, Oct 10, 2024 at 11:16 PM Colin McCabe 
> wrote:
> >
> >> This is the second candidate for the release of Apache Kafka 3.9.0. I
> have
> >> titled it rc2 since I had an rc1 which got very far, even to the point
> of
> >> pushing tags and docker images, before I spotted an issue. So rather
> than
> >> mutate the tags, I decided to skip over rc1.
> >>
> >> - This is a major release, the final one in the 3.x line. (There may of
> >> course be other minor releases in this line, such as 3.9.1.)
> >> - Tiered storage will be considered production-ready in this release.
> >> - This will be the final major release to feature the deprecated
> ZooKeeper
> >> mode.
> >>
> >> This release includes the following KIPs:
> >> - KIP-853: Support dynamically changing KRaft controller membership
> >> - KIP-1057: Add remote log metadata flag to the dump log tool
> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
> >> - KIP-1040: Improve handling of nullable values in InsertField,
> >> ExtractField, and other transformations
> >> - KIP-1031: Control offset translation in MirrorSourceConnector
> >> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring
> >> during processing
> >> - KIP-1017: Health check endpoint for Kafka Connect
> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in
> >> authorization header
> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> >> - KIP-950: Tiered Storage Disablement
> >> - KIP-956: Tiered Storage Quotas
> >>
> >> Release notes for the 3.9.0 release:
> >>
> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
> >>
> >> *** Please download, test and vote by October 16, 2024.
> >>
> >> Kafka's KEYS file containing PGP keys we use to sign the release:
> >> https://kafka.apache.org/KEYS
> >>
> >> * Release artifacts to be voted upon (source and binary):
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
> >>
> >> * Docker release artifacts to be voted upon:
> >> apache/kafka:3.9.0-rc2
> >> apache/kafka-native:3.9.0-rc2
> >>
> >> * Maven artifacts to be voted upon:
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>
> >> * Javadoc:
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
> >>
> >> * Documentation:
> >> https://kafka.apache.org/39/documentation.html
> >>
> >> * Protocol:
> >> https://kafka.apache.org/39/protocol.html
> >>
> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
> >>
> >> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
> >> Docker Build Test Pipeline (JVM):
> >> https://github.com/apache/kafka/actions/runs/11281563007
> >> Docker Build Test Pipeline (Native):
> >> https://github.com/apache/kafka/actions/runs/11281608809
> >>
> >> Thanks to everyone who helped with this release candidate, either by
> >> contributing code, testing, or documentation.
> >>
> >> Regards,
> >> Colin
> >>
>

Re: [VOTE] 3.9.0 RC2

[Chia-Ping Tsai]

> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems 
> to be using now (ducktape 0.11.4) We will not advance the version of ducktape 
> in the 3.x branch, I think.

Yes, I agree not to advance the version of Ducktape, but the Ducktape version 
was unpinned from 3.2 (see 
https://github.com/apache/kafka/commit/7e683852b46c0fe70f5100e953e3cd61d1697c06),
 so I suspect 3.9 hasn't been using Ducktape 0.8.14 for a while.

The PR (https://github.com/apache/kafka/pull/17481) locks Ducktape to version 
0.11.4, which is the latest release that works well with the 3.9 Dockerfile 
(and it is waiting for your review). Additionally, the trunk can use Ducktape 
0.12.0, which supports newer Python versions, allowing us to upgrade the base 
image without relying on outdated Python versions.

Best,
Chia-Ping

On 2024/10/14 15:08:06 Colin McCabe wrote:
> Hi Chia-Ping Tsai,
> 
> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems 
> to be using now (ducktape 0.11.4) We will not advance the version of ducktape 
> in the 3.x branch, I think.
> 
> cheers,
> Colin
> 
> 
> On Sat, Oct 12, 2024, at 01:49, Chia-Ping Tsai wrote:
> > hi Colin
> >
> > I'm sending this email to remind developers who plan to test E2E for RC2.
> >
> > There is an issue with the E2E tests due to the new release of 
> > Ducktape. I've filed KAFKA-17781 to address it. Only developers who 
> > recently rebuilt the E2E image will encounter this issue, so I assume 
> > it is NOT a blocker for RC2.
> >
> > Best,
> > Chia-Ping
> >
> >
> > On 2024/10/10 21:14:55 Colin McCabe wrote:
> >> This is the second candidate for the release of Apache Kafka 3.9.0. I have 
> >> titled it rc2 since I had an rc1 which got very far, even to the point of 
> >> pushing tags and docker images, before I spotted an issue. So rather than 
> >> mutate the tags, I decided to skip over rc1.
> >> 
> >> - This is a major release, the final one in the 3.x line. (There may of 
> >> course be other minor releases in this line, such as 3.9.1.)
> >> - Tiered storage will be considered production-ready in this release.
> >> - This will be the final major release to feature the deprecated ZooKeeper 
> >> mode.
> >> 
> >> This release includes the following KIPs:
> >> - KIP-853: Support dynamically changing KRaft controller membership
> >> - KIP-1057: Add remote log metadata flag to the dump log tool
> >> - KIP-1049: Add config log.summary.interval.ms to Kafka Streams
> >> - KIP-1040: Improve handling of nullable values in InsertField, 
> >> ExtractField, and other transformations
> >> - KIP-1031: Control offset translation in MirrorSourceConnector
> >> - KIP-1033: Add Kafka Streams exception handler for exceptions occurring 
> >> during processing
> >> - KIP-1017: Health check endpoint for Kafka Connect
> >> - KIP-1025: Optionally URL-encode clientID and clientSecret in 
> >> authorization header
> >> - KIP-1005: Expose EarliestLocalOffset and TieredOffset
> >> - KIP-950: Tiered Storage Disablement
> >> - KIP-956: Tiered Storage Quotas
> >> 
> >> Release notes for the 3.9.0 release:
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/RELEASE_NOTES.html
> >> 
> >> *** Please download, test and vote by October 16, 2024.
> >> 
> >> Kafka's KEYS file containing PGP keys we use to sign the release:
> >> https://kafka.apache.org/KEYS
> >> 
> >> * Release artifacts to be voted upon (source and binary):
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/
> >> 
> >> * Docker release artifacts to be voted upon:
> >> apache/kafka:3.9.0-rc2
> >> apache/kafka-native:3.9.0-rc2
> >> 
> >> * Maven artifacts to be voted upon:
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >> 
> >> * Javadoc:
> >> https://dist.apache.org/repos/dist/dev/kafka/3.9.0-rc2/javadoc/
> >> 
> >> * Documentation:
> >> https://kafka.apache.org/39/documentation.html
> >> 
> >> * Protocol:
> >> https://kafka.apache.org/39/protocol.html
> >> 
> >> * Tag to be voted upon (off 3.9 branch) is the 3.9.0-rc2 tag:
> >> https://github.com/apache/kafka/releases/tag/3.9.0-rc2
> >> 
> >> * Successful Docker Image Github Actions Pipeline for 3.9 branch:
> >> Docker Build Test Pipeline (JVM): 
> >> https://github.com/apache/kafka/actions/runs/11281563007
> >> Docker Build Test Pipeline (Native): 
> >> https://github.com/apache/kafka/actions/runs/11281608809
> >> 
> >> Thanks to everyone who helped with this release candidate, either by 
> >> contributing code, testing, or documentation.
> >> 
> >> Regards,
> >> Colin
> >>
>