On Tue, Oct 29, 2024, at 02:38, Anton Agestam wrote:
>
> Because rather than forcing API designers to specify good defaults,
> [implicit defaults] force protocol implementers to inject bad defaults.
>
> The good examples from this thread are host and port, empty string is not a
> valid hostname, a
Hi Colin,
> Why is it bad API design?
Because rather than forcing API designers to specify good defaults, it
forces protocol implementers to inject bad defaults.
The good examples from this thread are host and port, empty string is not a
valid hostname, and zero is not a valid port, so those are
Hi,
Chia-Ping shared in the RC5 thread now two cases where the behavior under
discussion actually exists prior to 3.9 so all my arguments here are moot.
I missed those two cases because they have the same name as the struct
introduced in 3.9, and my regex excluded that by name when I was looking
f
On Sun, Oct 27, 2024, at 01:44, Anton Agestam wrote:
> Colin
>
> I have presented four reasons, I'll list them again below. Please let me
> know which ones there isn't already enough information on the thread
> already.
>
> - The behavior is new.
Hi Anton,
This behavior isn't new. I gave an examp
Hi Anton,
Thanks for raising the discussion.
I read through this discussion and I can understand your confusion.
> The behavior is new.
I think like Colin said, this behavior actually stays consistent as in the
tagged field in ApiVersionsResponse.json named SupportedFeatures. It
doesn't specify
Colin
I have presented four reasons, I'll list them again below. Please let me
know which ones there isn't already enough information on the thread
already.
- The behavior is new.
- The behavior is undocumented.
- The behavior is bad API design.
- The behavior does not really save bytes *in pract
Hi Anton,
The behavior where we leave out tagged fields when the default value is present
is intentional. As is the behavior where default values are treated as 0, the
empty array, etc. when defaults are not explicitly specified. These two
behaviors, taken together, save space on the wire, and
Hi Colin,
> The current behavior is older than 3.9
No it is not. The *Java implementation* is older than 3.9. But the behavior
is a sum of Java implementation + JSON schema. It's a new construct of JSON
schema that exposes the new behavior. It is fully possible to adjust
behavior only for the new
Hi Anton,
I appreciate your desire for clarity in the specification. If you think there's
something we could do to clarify the docs, let me know (or better yet, file a
PR!)
But we cannot change the behavior of defaults for tagged fields. The current
behavior is older than 3.9 and a lot of thin
Hi again Colin,
> The implicit defaults have been documented for a long time
Yes, if you read my message you will see I am referencing and quoting that
exact documentation. The new behavior is clearly excluded by the explicit
pre requisite in that exact documentation.
Again in summary (all three
Hi Anton,
The implicit defaults have been documented for a long time. Take a look at this
documentation from 2019 in clients/src/main/resources/common/message/README.md :
> Deserializing Messages
> --
> Message objects may be deserialized using the Message#read method. This
Hi Colin,
I understand your perspective that this is a bug in the client, but I
disagree that this is something that's established, or inferrable from
existing specification and implementation.
The behavior of exposing the implicit defaults on the wire protocol really
is new. The only scenario th
Hi all,
I have posted a new release candidate, RC3. See the RC3 thread.
best,
Colin
On Mon, Oct 21, 2024, at 11:31, Colin McCabe wrote:
> Hi Anton,
>
> I replied on the JIRA. I do not think this is a bug, you just failed to
> account for implicit defaults in your protocol code. That is, 0 is th
Hi Anton,
I replied on the JIRA. I do not think this is a bug, you just failed to account
for implicit defaults in your protocol code. That is, 0 is the default of
numeric fields if no other default is specified, etc.
best,
Colin
On Mon, Oct 21, 2024, at 08:07, Anton Agestam wrote:
> Hi everyo
Hi everyone,
I have found a protocol serialization bug that surfaces only with one of
the entities introduced for KIP-853 (UpdateRaftVoterResponse).
Due to the irreversible implications this might have once merged, I'd argue
that this needs to be considered a release blocker.
https://issues.apac
Hi Chia-Ping Tsai,
Yes, let’s take the jetty CVE fix for 3.9.0.
Best,
Colin
On Wed, Oct 16, 2024, at 08:51, Chia-Ping Tsai wrote:
> hi Colin
>
> Do you think KAFKA-17810 is a blocker for 3.9.0 since it's related to a
> CVE? The PR (https://github.com/apache/kafka/pull/17517) will upgrade
> Je
hi Colin
> Yes, let’s take the jetty CVE fix for 3.9.0.
done (
https://github.com/apache/kafka/commit/de9a7199dfbcfbc63334772f7b4556826fbdf4ce
)
Colin McCabe 於 2024年10月21日 週一 上午2:37寫道：
> Hi Chia-Ping Tsai,
>
> Yes, let’s take the jetty CVE fix for 3.9.0.
>
> Best,
> Colin
>
>
> On Wed, Oct 16,
hi Colin
Do you think KAFKA-17810 is a blocker for 3.9.0 since it's related to a CVE?
The PR (https://github.com/apache/kafka/pull/17517) will upgrade Jetty to
9.4.56.v20240826 to fix one of the CVEs, and we can backport it to 3.9 if
you're okay with rolling RC3
Best,
Chia-Ping
On 2024/10/1
Hi Colin,
> BrokerServer will try to open the ports that belong to the broker;
ControllerServer will try to open the ports that belong to the controller.
You obviously can't open the same port twice (in standard UNIX, at least)
or if you somehow did, the result would be nonsense. This isn't such a
While messing around with the release candidate, I found a separate blocker
bug: KAFKA-17794.
Basically, our behavior is confusing when we format the storage directory with
KIP-853 (dynamic controller quorum), but no initial controllers are supplied. I
have a PR which requires the initial contr
On Mon, Oct 14, 2024, at 15:04, Jakub Scholz wrote:
> Hi Colin,
>
> So, how exactly does this major misconfiguration that was not documented
> for over a year and nobody complained manifest itself? What should I look
> for in the logs? What are the problems it manifests itself through? There
> are
Hi Colin,
So, how exactly does this major misconfiguration that was not documented
for over a year and nobody complained manifest itself? What should I look
for in the logs? What are the problems it manifests itself through? There
are plenty of users who went through the migration with this "major
Hi Jakub,
It has always been required to separate control plane listeners and controller
listeners. Failing to do this is a major misconfiguration. It may not have been
caught sometimes, but that is a bug.
It should be simple to fix the configuration you posted -- simply have a
different name
The different name of the controller listener for KRaft controllers and
control plane listener in ZooKeeper-based cluster was not required before
and it is not simple to change to handle now at the "last minute". So given
that this is called production-ready already for some time, I think this is
b
Thanks, Federico.
best,
Colin
On Mon, Oct 14, 2024, at 09:23, Federico Valeri wrote:
> On Mon, Oct 14, 2024 at 5:11 PM Colin McCabe wrote:
>>
>> Hi Federico,
>>
>> Thanks for taking a look! Can you please file a JIRA for the documentation
>> changes you'd like to make?
>>
>
> There was already
I also posted a PR to add removing control.plane.listener to the docs:
https://github.com/apache/kafka/pull/17501
This needs to be done for each broker process before before it is restarted in
KRaft mode. (However, it is fine to BEGIN the migration process with control
plane listeners enabled.)
Hi Jakub,
After looking through the attached file on the JIRA, I can say that this is a
misconfiguration. control.plane.listener is a totally separate concept from
control.plane.listener.name. They should never be set to the same value. The
controller listener must have a different name and val
On Mon, Oct 14, 2024 at 5:11 PM Colin McCabe wrote:
>
> Hi Federico,
>
> Thanks for taking a look! Can you please file a JIRA for the documentation
> changes you'd like to make?
>
There was already a Jira about this, which was closed my mistake:
https://issues.apache.org/jira/browse/KAFKA-17031
Hi Colin,
I have tested 3.9 RC2 with Amazon Corretto 8-21 and Zulu 8-21, and
everything seems to be working fine.
Therefore, I am +1 on this RC (non-binding).
Best regards,
TengYao
Colin McCabe 於 2024年10月14日 週一 下午11:42寫道：
> Hi Chia-Ping Tsai,
>
> Thanks for looking into this. I approved the PR
Hi Chia-Ping Tsai,
Thanks for looking into this. I approved the PR.
best,
Colin
On Mon, Oct 14, 2024, at 08:34, Chia-Ping Tsai wrote:
>> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems
>> to be using now (ducktape 0.11.4) We will not advance the version of
>> duckt
> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems
> to be using now (ducktape 0.11.4) We will not advance the version of ducktape
> in the 3.x branch, I think.
Yes, I agree not to advance the version of Ducktape, but the Ducktape version
was unpinned from 3.2 (see
> control.plane.listener is not (and never has been) supported in KRaft
mode.
You mean control.plane.listener.name is not supported in KRaft I guess?
Well, this is not KRaft, this is migration, so it uses the settings that it
used before for the Zoo-based cluster and that includes using dedicated
Hi Jakub,
Thanks for testing. control.plane.listener is not (and never has been)
supported in KRaft mode. You have to remove control.plane.listener
configurations before migrating. I filed KAFKA-17790 to document this in the
migration instructions. (This is not a blocker for the release, though
Hi Federico,
Thanks for taking a look! Can you please file a JIRA for the documentation
changes you'd like to make?
best,
Colin
On Sun, Oct 13, 2024, at 09:21, Federico Valeri wrote:
> Hi Colin, thanks for putting out the new RC.
>
> According to KIP-950, remote.log.manager.thread.pool.size sh
Hi Chia-Ping Tsai,
Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems to
be using now (ducktape 0.11.4) We will not advance the version of ducktape in
the 3.x branch, I think.
cheers,
Colin
On Sat, Oct 12, 2024, at 01:49, Chia-Ping Tsai wrote:
> hi Colin
>
> I'm send
Hi Colin,
Thanks for the RC. I did some testing of it and run into
https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a
regression in the migration to KRaft process.
Can someone who understands this part of the codebase look into it please?
Thanks & Regards
Jakub
On Thu, Oct 10
Hi Colin, thanks for putting out the new RC.
According to KIP-950, remote.log.manager.thread.pool.size should be
marked as deprecated and replaced by two new configurations:
remote.log.manager.copier.thread.pool.size and
remote.log.manager.expiration.thread.pool.size. I don't see that
reflected in
hi Colin
I'm sending this email to remind developers who plan to test E2E for RC2.
There is an issue with the E2E tests due to the new release of Ducktape. I've
filed KAFKA-17781 to address it. Only developers who recently rebuilt the E2E
image will encounter this issue, so I assume it is NOT a
I also wanted to note that 3.9.0-RC2 doesn't include David's fix for
KAFKA-17193. Since that PR only affects our integration with Github Actions and
not the Java or Scala source code of Kafka, I did not re-generate the release
candidate tar.gz and/or tags after that fix was made.
Also, thanks D
This is the second candidate for the release of Apache Kafka 3.9.0. I have
titled it rc2 since I had an rc1 which got very far, even to the point of
pushing tags and docker images, before I spotted an issue. So rather than
mutate the tags, I decided to skip over rc1.
- This is a major release,
40 matches
Mail list logo
