Re: [VOTE] KIP-792: Add "generation" field into consumer protocol

2021-12-05 Thread Colin McCabe
. If the coordinator moves back and forth between brokers with different IBPs, it seems that the same epoch numbers could be reused for a group, if things are done in the obvious manner (old IBP = don't read or write epoch, new IBP = do). best, Colin On Fri, Dec 3, 2021, at 18:46, Luke

Re: [DISCUSS] KIP-785 Automatic storage formatting

2021-12-13 Thread Colin McCabe
disk to a node. The behavior of a failing disk is usually pathological. In addition to not persisting the data, you often get very slow operations, kernel errors, and unpredictable behavior. Certainly automatically re-adding a potentially bad disk doesn't do anyone any favors. best, Colin

Re: [DISCUSS] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

2021-12-13 Thread Colin McCabe
some common features, and I think it's somewhat orthogonal to the main point here (authorizer in kraft) If this KIP lands before the bootstrapping one, we can always set super.users to be the broker user as a workaround. best, Colin On Wed, Dec 8, 2021, at 14:15, José Armando García Sancio w

[VOTE] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

2021-12-13 Thread Colin McCabe
ake a look and vote if you can. best, Colin

Re: [DISCUSS] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

2021-12-14 Thread Colin McCabe
On Tue, Dec 14, 2021, at 08:27, José Armando García Sancio wrote: > Thanks for the additional information Colin. > ... > > It sounds to me like KIP-801 is assuming that this "bootstrapping KIP" > will at least generate a snapshot with this information in all of the >

Re: [DISCUSS] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

2021-12-14 Thread Colin McCabe
On Tue, Dec 14, 2021, at 10:13, David Arthur wrote: > Thanks for the KIP, Colin! > > 4) For the various "Type" fields in the AccessControlEntityRecord, is it > worth explicitly enumerating the allowed types in the KIP? E.g., > PermissionType = {Any, Deny, Allow}. If these

Re: [VOTE] KIP-792: Add "generation" field into consumer protocol

2021-12-16 Thread Colin McCabe
Thanks for the explanation, Luke. That makes sense. best, Colin On Thu, Dec 9, 2021, at 13:31, Guozhang Wang wrote: > Thanks Luke, in that case I'm +1 on this KIP. > > On Thu, Dec 9, 2021 at 1:46 AM Luke Chen wrote: > >> Hi Guozhang, >> >> Thanks for your co

Re: [Proposal] It is hoped that Apache APISIX and Apache Kafka will carry out diversified community cooperation

2021-12-16 Thread Colin McCabe
Hi Yeliang Wang, Have you considered submitting a talk to Kafka Summit discussing how Apache APISIX uses Kafka? That might be interesting. I'm not sure what you mean by "fully integrating" the projects. Can you elaborate on what integration you see happening in the future? best

Re: [VOTE] KIP-778 KRaft upgrades

2021-12-16 Thread Colin McCabe
Thanks for the KIP, David! Great work. +1 (binding). Should the "./kafka-features.sh downgrade" command also have a --release flag, to match upgrade? Also, it seems like upgrade should have a --latest flag that upgrades everything to the latest installed version? best, Colin On F

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-09-30 Thread Colin McCabe
On Tue, Sep 29, 2020, at 17:43, Jason Gustafson wrote: > Hey Colin, > > Thanks for the hard work on this proposal. > > I'm gradually coming over to the idea of the controllers having separate > IDs. One of the benefits is that it allows us to separate the notion of >

Re: [VOTE] KIP-673: Emit JSONs with new auto-generated schema

2020-10-02 Thread Colin McCabe
Thanks, Anastasia! This will be a lot easier to maintain. +1 (binding) best, Colin On Wed, Sep 30, 2020, at 23:57, David Jacot wrote: > Thanks for the KIP, Anastasia. > > +1 (non-binding) > > On Thu, Oct 1, 2020 at 8:06 AM Tom Bentley wrote: > > > Thanks Anastasia,

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-10-05 Thread Colin McCabe
On Mon, Sep 28, 2020, at 11:41, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. A few more comments below. > > 62. > 62.1 controller.listener.names: So, is this used for the controller or the > broker trying to connect to the controller? > Hi Jun, It's used

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-10-12 Thread Colin McCabe
On Tue, Oct 6, 2020, at 16:09, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. Made another pass of the KIP. A few more comments > below. > Hi Jun, Thanks for the review. > 55. We discussed earlier why the current behavior where we favor the > current broker registr

Re: [DISCUSS] Kafka 3.0

2020-10-15 Thread Colin McCabe
Hi all, Just to follow up on this... since we're not quite ready for 3.0 yet, it's probably best if we release a 2.8 next, and then go to 3.0 after that. Sorry for any confusion. best, Colin On Mon, Jul 20, 2020, at 12:52, Matthias J. Sax wrote: > Did we reach any conclusion o

Re: [VOTE] KIP-653: Upgrade log4j to log4j2

2020-10-21 Thread Colin McCabe
th Log4j 1.x is provided through the log4j12-api > component, > however it does not implement some of the very implementation specific > classes and methods best, Colin On Fri, Oct 9, 2020, at 02:51, Tom Bentley wrote: > +1 non-binding. > > Thanks for your efforts on this Don

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-10-21 Thread Colin McCabe
On Mon, Oct 19, 2020, at 08:59, Ron Dagostino wrote: > Hi Colin. Thanks for the hard work on this KIP. > > I have some questions about what happens to a broker when it becomes > fenced (e.g. because it can't send a heartbeat request to keep its > lease). The KIP says &quo

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-10-21 Thread Colin McCabe
On Tue, Oct 13, 2020, at 18:30, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. A few more comments below. > > 80.1 controller.listener.names only defines the name of the listener. The > actual listener including host/port/security_protocol is typically defined > i

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-10-23 Thread Colin McCabe
ceBroker, which I think addresses your earlier point that we should not confuse unfencing with registering. best, Colin It is not necessary to communicate On Thu, Oct 22, 2020, at 10:48, Ron Dagostino wrote: > Hi again, Colin. Related to the issue of how to communicate fenced > vs. not f

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-10-23 Thread Colin McCabe
On Wed, Oct 21, 2020, at 05:51, Tom Bentley wrote: > Hi Colin, > > On Mon, Oct 19, 2020, at 08:59, Ron Dagostino wrote: > > > Hi Colin. Thanks for the hard work on this KIP. > > > > > > I have some questions about what happens to a broker when it becomes >

Re: Is AdminClient of Kafka thread-safe?

2020-10-29 Thread Colin McCabe
Yes, KafkaAdminClient is thread-safe. best, Colin On Wed, Oct 7, 2020, at 12:38, Efe Gencer wrote: > Hi All, > > Other than a Stack Overflow comment (see > https://stackoverflow.com/a/61738065) by Colin Patrick McCabe (CC'd), > there is no source that verifies

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-11-30 Thread Colin McCabe
On Fri, Oct 23, 2020, at 16:10, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. A few more comments. Hi Jun, Thanks again for the reply. Sorry for the long hiatus. I was on vacation for a while. > > 55. There is still text that favors new broker registration. "W

Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor

2020-12-02 Thread Colin McCabe
ng, rather than Stable, so that we can change them at will. Basically not give a compatibility guarantee for this. best, Colin On Thu, Nov 26, 2020, at 08:47, Mickael Maison wrote: > Thanks Gwen for following up. > > With this extra bit of context, David's comments make more se

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-02 Thread Colin McCabe
On Wed, Dec 2, 2020, at 14:07, Ron Dagostino wrote: > Hi Colin. Thanks for the updates. It's now clear to me that brokers > keep their broker epoch for the life of their JVM -- they register > once, get their broker epoch in the response, and then never > re-register again.

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-08 Thread Colin McCabe
On Thu, Dec 3, 2020, at 16:37, Jun Rao wrote: > Hi, Colin, > > Thanks for the updated KIP. A few more comments below. > Hi Jun, Thanks again for the reviews. > 80.2 For deprecated configs, we need to include zookeeper.* and > broker.id.generation.enable. > Added. >

Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor

2020-12-08 Thread Colin McCabe
On Thu, Dec 3, 2020, at 07:41, Edoardo Comar wrote: > Hi Colin > thanks for your comments. > I think your objections to creating an interface for replica placement > could be used against similar server-side plug-ins (Authorizer, > QuotaCallback). Hi Edoardo, The Authorizer AP

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-11 Thread Colin McCabe
On Wed, Dec 9, 2020, at 10:10, Jun Rao wrote: > Hi, Colin, > > Thanks for the update. A few more follow up comments. > Hi Jun, Thanks again for the review. > 100. FailedReplicaRecord: Since this is reported by each broker > independently, perhaps we could use a more conc

[VOTE] voting on KIP-631: the quorum-based Kafka controller

2020-12-11 Thread Colin McCabe
3c%40%3Cdev.kafka.apache.org%3E There is also a second email DISCUSS thread, which is here: https://lists.apache.org/thread.html/r1ed098a88c489780016d963b065e8cb450a9080a4736457cd25f323c%40%3Cdev.kafka.apache.org%3E Please take a look and vote if you can. best, Colin

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-14 Thread Colin McCabe
On Fri, Dec 11, 2020, at 17:07, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. Just a couple of more comments below. > > 210. Since we are deprecating zookeeper.connection.timeout.ms, should we > add a new config to bound the time for a broker to connect to the &

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-15 Thread Colin McCabe
On Tue, Dec 15, 2020, at 04:13, Tom Bentley wrote: > Hi Colin, > > The KIP says that "brokers which are fenced will not appear in > MetadataResponses. So clients that have up-to-date metadata will not try > to contact fenced brokers.", which is fine, but it doesn'

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-16 Thread Colin McCabe
On Tue, Dec 15, 2020, at 13:08, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. A few more follow up comments. > > 210. initial.broker.registration.timeout.ms: The default value is 90sec, > which seems long. If a broker fails the registration because of incorrect > con

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-16 Thread Colin McCabe
On Wed, Dec 16, 2020, at 09:59, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. A few more comments below. > Hi Jun, Thanks for the comments. > 206. "RemoveTopic is the last step, that scrubs all metadata about the > topic. In order to get to that last step,

Re: [VOTE] voting on KIP-631: the quorum-based Kafka controller

2020-12-16 Thread Colin McCabe
On Wed, Dec 16, 2020, at 10:10, Jason Gustafson wrote: > +1 Thanks Colin for all the iterations. My only request is to change > "controller.connect" to "controller.quorum.voters." I think it's important > to emphasize that this must be the full set of voters un

Re: [VOTE] voting on KIP-631: the quorum-based Kafka controller

2020-12-16 Thread Colin McCabe
o I'm not sure if "deregister" has an advantage... > 11. Broker metrics typically have a PerSec suffix, should we stick with > that for the `MetadataCommitRate`? Added. > 12. For the lag metrics, would it be clearer if we included "Offset" in the > name? In the

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-16 Thread Colin McCabe
On Wed, Dec 16, 2020, at 13:40, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. A few follow up comments. > > 211. When does the broker send the BrokerRegistration request to the > controller? Is it after the recovery phase? If so, at that point, the > broker has alr

Re: [DISCUSS] KIP-700: Add Describe Cluster API

2020-12-16 Thread Colin McCabe
#ClusterAuthorizedOperations. So it seems like this KIP should specify a new version of MetadataRequest where the related fields are absent, right? best, Colin On Mon, Dec 14, 2020, at 08:10, David Jacot wrote: > Hi all, > > I'd like to propose a small KIP: > https://cwiki.apache.org/conflue

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-16 Thread Colin McCabe
On Wed, Dec 16, 2020, at 18:13, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. Just a couple of more comments. > > 211. Currently, the broker only registers itself in ZK after log recovery. > Is there any benefit to change that? As you mentioned, the broker can&

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-17 Thread Colin McCabe
On Thu, Dec 17, 2020, at 10:19, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. > > 211. Hmm, I still don't see a clear benefit of registering a broker before > recovery. It's possible for the recovery to take time. However, during the > recovery mode, it se

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-12-17 Thread Colin McCabe
On Thu, Dec 17, 2020, at 17:02, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. Sounds good. One more comment. > > 231. Currently, we have sasl.mechanism.inter.broker.protocol for inter > broker communication. It seems that we need a similar config for specifying > t

Re: [VOTE] voting on KIP-631: the quorum-based Kafka controller

2020-12-18 Thread Colin McCabe
Hi all, I'm going to close the vote in a few hours. Thanks to everyone who reviewed and voted. best, Colin On Fri, Dec 18, 2020, at 10:08, Jun Rao wrote: > Thanks, Colin. +1 > > Jun > > On Thu, Dec 17, 2020 at 2:24 AM David Jacot wrote: > > > Thanks for dri

Re: [DISCUSS] KIP-700: Add Describe Cluster API

2020-12-18 Thread Colin McCabe
red. I think we should hold off on adding more epochs right now until we have a more general solution. With KIP-500 we can potentially put an epoch on the whole metadata response, which would allow us to have read-after-write consistency for metadata-- something people have wanted for a while.

Re: [VOTE] voting on KIP-631: the quorum-based Kafka controller

2020-12-21 Thread Colin McCabe
Hi all, With non-binding +1 votes from Ron Dagostino, Tom Bentley and Unmesh Joshi, and binding +1 votes from David Arthur, Boyang Chen, Jason Gusafson, Ismael Juma, David Jacot, Jun Rao, the KIP passes. thanks, all! cheers, Colin On Fri, Dec 18, 2020, at 12:42, Colin McCabe wrote: > Hi

Re: [VOTE] voting on KIP-631: the quorum-based Kafka controller

2021-01-05 Thread Colin McCabe
them int32. This is consistent with what we do in MetadataResponse and a few other places. best, Colin On Mon, Dec 21, 2020, at 14:42, Colin McCabe wrote: > Hi all, > > With non-binding +1 votes from Ron Dagostino, Tom Bentley and Unmesh > Joshi, and binding +1 votes from David Ar

Re: [VOTE] voting on KIP-631: the quorum-based Kafka controller

2021-01-11 Thread Colin McCabe
, Colin On Tue, Jan 5, 2021, at 17:03, Colin McCabe wrote: > Hi all, > > Addendum: some of the port types in this KIP were specified as int16 in > the wire protocol. But this does not gracefully handle ports like > 33,000, which shows up as negative when using a signed 16 bit numb

Re: Optimistic locking

2015-01-05 Thread Colin Clark
g mechanisms, I also recommend not using a queuing mechanism as a primary datastore - mixed semantics. -- *Colin* +1 612 859-6129 On Mon, Jan 5, 2015 at 4:47 AM, Daniel Schierbeck < daniel.schierb...@gmail.com> wrote: > I'm trying to design a system that uses Kafka as its prima

Re: produce request wire format question

2013-05-21 Thread Colin Blower
which partition a message goes to. -- Colin B. On 05/21/2013 11:48 AM, Dave Peterson wrote: > I'm looking at the document entitled "A Guide to the Kafka Protocol" > located here: > > https://cwiki.apache.org/KAFKA/a-guide-to-the-kafka-protocol.html > > It s

Re: produce request wire format question

2013-05-23 Thread Colin Blower
documentation please let us know. Also, in general error codes are int16. -- Colin B. On 05/23/2013 09:43 AM, Dave Peterson wrote: > Ok, thanks for the information. Looking at the wire format for the > metadata response, I see that the right hand side of the TopicMetadata > production c

Re: Initializing fetch offsets

2013-02-21 Thread Colin Blower
this API a bit? I'd like to write a consumer that can start from 0 (like unix cat) or start from the most recent (like unix tail). Thanks! -David -- *Colin Blower* /Software Engineer/ Barracuda Networks Inc. +1 408-342-5576 (o)

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
for the controller listener than the control plane listener. best, Colin On Mon, Oct 14, 2024, at 11:16, Jakub Scholz wrote: > The different name of the controller listener for KRaft controllers and > control plane listener in ZooKeeper-based cluster was not required before > and it is n

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
On Mon, Oct 14, 2024, at 15:04, Jakub Scholz wrote: > Hi Colin, > > So, how exactly does this major misconfiguration that was not documented > for over a year and nobody complained manifest itself? What should I look > for in the logs? What are the problems it manifests itself throu

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
controllers to be initialized in this case (since otherwise, it does not work). best, Colin On Mon, Oct 14, 2024, at 16:34, Colin McCabe wrote: > On Mon, Oct 14, 2024, at 15:04, Jakub Scholz wrote: >> Hi Colin, >> >> So, how exactly does this major misconfiguration that was not d

Re: New release branch 3.9

2024-10-09 Thread Colin McCabe
Hi Josep, I marked this as a blocker. We will take a look. I also found some dependencies that need to be updated due to CVEs. See github.com/apache/kafka/pull/17436 best, Colin On Wed, Oct 9, 2024, at 05:52, Josep Prat wrote: > Hi Colin, > > I want to raise a bug we encountered whil

Re: New release branch 3.9

2024-10-10 Thread Colin McCabe
Thanks for the PR, Josep. I don't think it's critical to re-generate the RC since we can simply update the docs on the website later as we've done in the past. If there is a new RC after RC2, it will be there, though. Also it looks like this will be a good fix for 3.8.1 as well

Re: [VOTE] 3.9.0 RC2

2024-10-21 Thread Colin McCabe
Hi Anton, I replied on the JIRA. I do not think this is a bug, you just failed to account for implicit defaults in your protocol code. That is, 0 is the default of numeric fields if no other default is specified, etc. best, Colin On Mon, Oct 21, 2024, at 08:07, Anton Agestam wrote: >

Re: [DISCUSS] Apache Kafka 4.0.0 release

2024-10-20 Thread Colin McCabe
Hi TengYao Chi, The KIP freeze for Apache Kafka 4.0 is. November 20th. So if KIP-1092 has been accepted already, it should make it into 4.0 without any further discussion (assuming the code is ready by feature freeze.) Cheers, Colin On Fri, Oct 18, 2024, at 20:32, TengYao Chi wrote: >

Re: [VOTE] 3.9.0 RC2

2024-10-20 Thread Colin McCabe
Hi Chia-Ping Tsai, Yes, let’s take the jetty CVE fix for 3.9.0. Best, Colin On Wed, Oct 16, 2024, at 08:51, Chia-Ping Tsai wrote: > hi Colin > > Do you think KAFKA-17810 is a blocker for 3.9.0 since it's related to a > CVE? The PR (https://github.com/apache/kafka/pull/17

[VOTE] 3.9.0 RC3

2024-10-21 Thread Colin McCabe
Test Pipeline (Native): https://github.com/apache/kafka/actions/runs/11448338981 Thanks to everyone who helped with this release candidate, either by contributing code, testing, or documentation. Regards, Colin

Re: [VOTE] 3.9.0 RC2

2024-10-21 Thread Colin McCabe
Hi all, I have posted a new release candidate, RC3. See the RC3 thread. best, Colin On Mon, Oct 21, 2024, at 11:31, Colin McCabe wrote: > Hi Anton, > > I replied on the JIRA. I do not think this is a bug, you just failed to > account for implicit defaults in your protocol code. T

Re: New release branch 3.9

2024-10-10 Thread Colin McCabe
Hi all, I have merged KAFKA-17731 and KAFKA-17749 to 3.9. Also KAFKA-17751. We should be ready for the next RC now. Colin On Thu, Oct 10, 2024, at 08:45, Mickael Maison wrote: > Hi, > > Following up on https://issues.apache.org/jira/browse/KAFKA-17749. We > merged the fix in t

Re: [DISCUSS] Apache Kafka 4.0.0 release

2024-10-10 Thread Colin McCabe
Hi David & Luke, We have been using https://issues.apache.org/jira/browse/KAFKA-17611 as the umbrella JIRA for ZK removal tasks. Progress has been pretty rapid, I do think we will get there by January. (Well hopefully even earlier :) best, Colin On Thu, Oct 10, 2024, at 05:55, David J

Re: [DISCUSS] Apache Kafka 3.7.2 release

2024-10-10 Thread Colin McCabe
+1. Thanks, Matthias. Colin On Thu, Oct 10, 2024, at 08:46, Mickael Maison wrote: > +1, thanks for volunteering > > Mickael > > On Thu, Oct 10, 2024 at 5:42 PM Bill Bejeck wrote: >> >> Thanks for volunteering Matthias! >> +1 >> >> Bill >> >

Re: [DISCUSS] Apache Kafka 4.0.0 release

2024-10-10 Thread Colin McCabe
Yes (although I don't believe there's a JIRA for that specifically yet.) best, Colin On Thu, Oct 10, 2024, at 09:11, David Jacot wrote: > Hi Colin, > > Thanks. That seems perfect. Does it include removing --zookeeper from the > command line tools (e.g. kafka-configs)? &g

[VOTE] 3.9.0 RC2

2024-10-10 Thread Colin McCabe
Test Pipeline (Native): https://github.com/apache/kafka/actions/runs/11281608809 Thanks to everyone who helped with this release candidate, either by contributing code, testing, or documentation. Regards, Colin

Re: [VOTE] 3.9.0 RC2

2024-10-10 Thread Colin McCabe
so, thanks David for fixing the github actions problem quickly! The other fixes that we discussed in the 3.9 branch thread should all be here. best, Colin On Thu, Oct 10, 2024, at 14:14, Colin McCabe wrote: > This is the second candidate for the release of Apache Kafka 3.9.0. I > have tit

Re: New release branch 3.9

2024-10-03 Thread Colin McCabe
Also, as a quick update, the PR for KAFKA-17608 has been merged to 3.9. KAFKA-16927 is the only current remaining blocker. best, Colin On Thu, Oct 3, 2024, at 12:49, Colin McCabe wrote: > Hi José, > > Yes, this seems like a regression in 3.9, so we need to fix it there. > Thanks f

Re: New release branch 3.9

2024-10-03 Thread Colin McCabe
Hi José, Yes, this seems like a regression in 3.9, so we need to fix it there. Thanks for the heads up. best, Colin On Thu, Oct 3, 2024, at 11:59, José Armando García Sancio wrote: > Hi all, > > We found a bug that should be fixed for 3.9.0: > https://issues.apache.org/jira/browse

Re: Cluster ID starting with hyphen, docs not exactly in sync

2024-10-22 Thread Colin McCabe
generate-uuid not create cluster IDs that started with a hyphen. best, Colin On Tue, Oct 22, 2024, at 06:44, Samuli Ruohola wrote: > Hi! > > Thank you for the Apache Kafka! We are grateful for it. > > In our root cause analysis process discussions, we were asked to share > followi

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
Hi Chia-Ping Tsai, Thanks for looking into this. I approved the PR. best, Colin On Mon, Oct 14, 2024, at 08:34, Chia-Ping Tsai wrote: >> Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems >> to be using now (ducktape 0.11.4) We will not advance th

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
, though.) best, Colin On Mon, Oct 14, 2024, at 02:52, Jakub Scholz wrote: > Hi Colin, > > Thanks for the RC. I did some testing of it and run into > https://issues.apache.org/jira/browse/KAFKA-17788 which seems to be a > regression in the migration to KRaft process. > > Can someone

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
Hi Chia-Ping Tsai, Apache Kafka 3.9 uses ducktape 0.8.14, which is well behind what trunk seems to be using now (ducktape 0.11.4) We will not advance the version of ducktape in the 3.x branch, I think. cheers, Colin On Sat, Oct 12, 2024, at 01:49, Chia-Ping Tsai wrote: > hi Colin >

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
Hi Federico, Thanks for taking a look! Can you please file a JIRA for the documentation changes you'd like to make? best, Colin On Sun, Oct 13, 2024, at 09:21, Federico Valeri wrote: > Hi Colin, thanks for putting out the new RC. > > According to KIP-950, remote.log.manager.th

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
.) best, Colin On Mon, Oct 14, 2024, at 10:54, Colin McCabe wrote: > Hi Jakub, > > After looking through the attached file on the JIRA, I can say that > this is a misconfiguration. control.plane.listener is a totally > separate concept from control.plane.listener.name. They

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
Thanks, Federico. best, Colin On Mon, Oct 14, 2024, at 09:23, Federico Valeri wrote: > On Mon, Oct 14, 2024 at 5:11 PM Colin McCabe wrote: >> >> Hi Federico, >> >> Thanks for taking a look! Can you please file a JIRA for the documentation >> changes you

Re: [VOTE] 3.9.0 RC2

2024-10-14 Thread Colin McCabe
tener for myself and verified that it works. best, Colin On Mon, Oct 14, 2024, at 08:31, Jakub Scholz wrote: >> control.plane.listener is not (and never has been) supported in KRaft > mode. > > You mean control.plane.listener.name is not supported in KRaft I guess? > Well, thi

Re: [VOTE] 3.9.0 RC3

2024-10-24 Thread Colin McCabe
On Wed, Oct 23, 2024, at 23:22, Justine Olshan wrote: > Hey Colin, > > Thanks for running the release. > > I checked the keys and scanned the docs > I built from source, used kraft quickstart, ran a transactions workload, > played around with a few other things includin

Re: [VOTE] 3.9.0 RC2

2024-10-24 Thread Colin McCabe
is that untagged fields are always present except in older versions, whereas tagged fields may not be present even in the most recent version. Perhaps there is something we could do to improve these docs? Or somehow make them easier to find? best, Colin On Wed, Oct 23, 2024, at 00:19, An

Re: [VOTE] 3.9.0 RC3

2024-10-24 Thread Colin McCabe
Hi Luke, Unfortunately, we don't have a way to upgrade from static quorums to dynamic quorums yet. We are hoping to include this in the next release. We're trakcing it under https://issues.apache.org/jira/browse/KAFKA-16538 . best, Colin On Thu, Oct 24, 2024, at 05:25, Luke Chen w

Re: [VOTE] 3.9.0 RC3

2024-10-24 Thread Colin McCabe
hanks, Colin On Thu, Oct 24, 2024, at 10:10, Colin McCabe wrote: > On Wed, Oct 23, 2024, at 23:22, Justine Olshan wrote: >> Hey Colin, >> >> Thanks for running the release. >> >> I checked the keys and scanned the docs >> I built from source, used kraft quicks

Re: [VOTE] 3.9.0 RC5

2024-10-29 Thread Colin McCabe
Thanks, Anton. And thanks Chia-Ping Tsai for taking a look at how we can improve the docs here... best, Colin On Tue, Oct 29, 2024, at 02:39, Anton Agestam wrote: > Hi Chia-Ping, > > Thanks for pointing those two fields out. I retract my -1. > > Cheers, > Anton > > De

Re: [VOTE] 3.9.0 RC2

2024-10-29 Thread Colin McCabe
explicit, we would have to introduce a richer syntax for initializing some types, like UUIDs, collections, etc. Collections alone would be daunting since the problem is recursive (I can have a collection containing other collections, etc.) best, Colin > > Cheers, > Anton > >

Re: [DISCUSS] Require KIPs to include "How to teach this section"

2024-10-29 Thread Colin McCabe
t be all of the documentation. We should update protocol.html and possibly other docs in cases where we're changing the protocol. I don't think it necessary needs to be done before the change itself, but it should be done so that the release that includes the protocol changes also includes t

Re: [DISCUSS] Require KIPs to include "How to teach this section"

2024-11-01 Thread Colin McCabe
change. It can be done incrementally if necessary. Hopefully having the section in the KIP would help remind us to do it, though! And motivate discussion about what kind of documentation would be best. best, Colin > Claude > > On Thu, Oct 31, 2024 at 12:21 PM Anton Agestam > wrote

Re: Advice on how to comprehend Kafka source code

2024-11-04 Thread Colin McCabe
hat people will be deploying in the next few years. best, Colin On Fri, Nov 1, 2024, at 21:58, Xiang Zhang wrote: > Hi community, > > After being a Kafka user for several years, I want to know Kafka better > maybe on a code level. I am wondering if anyone can give me any advice. To &g

Re: [DISCUSS] Require KIPs to include "How to teach this section"

2024-11-04 Thread Colin McCabe
omplains. best, Colin On Fri, Nov 1, 2024, at 19:10, Matthias J. Sax wrote: > The KIP already has a section "Documentation Plan" > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=50859709#KIPTemplate-DocumentationPlan > > If it's not used properly, comm

Re: [VOTE] 3.9.0 RC5

2024-10-30 Thread Colin McCabe
chekafka-1826/ Hopefully these should now show up in Maven as expected. best, Colin On Tue, Oct 29, 2024, at 23:51, Luke Chen wrote: > Hi Colin, > > I was trying to test the RC, but I found the artifacts in Maven artifacts > to be voted upon are not up-to-date. > Not only the "L

Re: [VOTE] 3.9.0 RC5

2024-11-01 Thread Colin McCabe
/og8l24z8oj87x0zsoqddfxo54qsj4d1o I'll continue with the release process and the release announcement will follow in the next few days. best, Colin On Thu, Oct 31, 2024, at 21:03, Luke Chen wrote: > Hi Colin, > > Thanks for the update. > I've ran a sanity check as Mickael suggested for the mave

Re: [VOTE] 3.9.0 RC5

2024-11-01 Thread Colin McCabe
Argh. That should read "5 +1 votes". :) Thanks, everyone. Colin On Fri, Nov 1, 2024, at 14:58, Colin McCabe wrote: > Hi all, > > This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes. > > +1 votes > PMC Members: > * Justine Olshan > * Bill Bej

Re: [VOTE] 3.9.0 RC2

2024-10-28 Thread Colin McCabe
On Sun, Oct 27, 2024, at 01:44, Anton Agestam wrote: > Colin > > I have presented four reasons, I'll list them again below. Please let me > know which ones there isn't already enough information on the thread > already. > > - The behavior is new. Hi Anton, Thi

Re: [VOTE] KIP-866 ZooKeeper to KRaft Migration

2024-11-11 Thread Colin McCabe
Hi all, I was doing some JIRA cleanup and noticed someone commenting about the language in KIP-866 referencing an offline KRaft -> ZK conversion tool. Since we never ended up implementing this, I have removed it from the KIP. best, Colin On Wed, Nov 30, 2022, at 12:27, David Arthur wr

[VOTE] 3.9.0 RC5

2024-10-26 Thread Colin McCabe
gards, Colin

Re: [VOTE] 3.9.0 RC2

2024-10-26 Thread Colin McCabe
along with feature freeze, code freeze, and every other freeze.) best, Colin On Sat, Oct 26, 2024, at 09:45, Anton Agestam wrote: > Hi Colin, > >> The current behavior is older than 3.9 > > No it is not. The *Java implementation* is older than 3.9. But the behavior > is a su

Re: [VOTE] 3.9.0 RC5

2024-10-31 Thread Colin McCabe
gradlewAll script seems a bit confusing (or maybe that's just me?) best, Colin On Wed, Oct 30, 2024, at 11:02, David Arthur wrote: > Should there be Scala 2.12 artifacts in the Maven staging? > > I see these for 2.13: > * > https://repository.apache.org/content/groups/stag

Re: New release branch 3.9

2024-09-17 Thread Colin McCabe
Hi Chia-Ping Tsai, I've already pushed the RC0 to maven central, so we cannot include this now. (I am waiting on some other release phases to complete before I can announce the RC!) However, I suspect that there will be another RC, so don't abandon the PR just yet :) best, Coli

[VOTE] 3.9.0 RC0

2024-09-17 Thread Colin McCabe
ne (Native): https://github.com/apache/kafka/actions/runs/10914303515 Thanks to everyone who helped with this release candidate, either by contributing code, testing, or documentation. Regards, Colin

Re: [VOTE] 3.9.0 RC0

2024-09-18 Thread Colin McCabe
ation generation is slightly wonky I'll create a new RC in a day or two that will fix these issues. best, Colin On Wed, Sep 18, 2024, at 07:21, Jakub Scholz wrote: > One more thing I noticed - the documentation at > https://kafka.apache.org/39/documentation.html#brokerconfigs seems to be &

Re: New release branch 3.9

2024-09-21 Thread Colin McCabe
to accommodate this fix. I will take a closer look on Monday. best, Colin On Fri, Sep 20, 2024, at 03:39, Christo Lolov wrote: > Hello, > > I have filed https://issues.apache.org/jira/browse/KAFKA-17584 as a > blocker. While it has not been introduced by a KIP, I think this has >

Re: New release branch 3.9

2024-09-23 Thread Colin McCabe
unblocked, as discussed earlier (forgot to mention that earlier) best, Colin On Sat, Sep 21, 2024, at 21:32, Colin McCabe wrote: > Hi Christo, > > Thanks for the bug report! It's a bit tricky to decide if this is a > blocker or not since it apparently has been around for a long

Re: New release branch 3.9

2024-09-23 Thread Colin McCabe
Thanks Chia-Ping Tsai! Please do backport the test fix to the 3.9 branch if we can get it in soon. (I don't think it's a blocker though...) best, Colin On Sun, Sep 22, 2024, at 05:29, Chia-Ping Tsai wrote: > hi Colin > > The `reassign_partitions_test.py` is unstable in 3

Re: [DISCUSS] Apache Kafka 4.0.0 release

2024-09-24 Thread Colin McCabe
+1. Thanks, David. best, Colin On Mon, Sep 23, 2024, at 10:11, Chris Egerton wrote: > Thanks David! +1 > > On Mon, Sep 23, 2024, 13:07 José Armando García Sancio > wrote: > >> +1, thanks for volunteering David! >> >> On Mon, Sep 23, 2024 at 11:56 AM David Ar

Re: [DISCUSS] bug fix to allow dynamically change some log configs

2024-09-24 Thread Colin McCabe
g" and bad. Maybe we should have removed those synonyms in 4.0, but I guess that ship has sailed. Perhaps 5.0 :) best, Colin On Tue, Sep 24, 2024, at 02:52, Luke Chen wrote: > Hi all, > > KAFKA-17584 <https://issues.apache.org/jira/browse/KAFKA-17584> is about a > bug

Re: New release branch 3.9

2024-09-26 Thread Colin McCabe
Hi all, I wanted to give a quick update on the 3.9.0 release. We had several fixes in the last few days (KAFKA-17459, KAFKA-17584, etc.) and uncovered two new blocker bugs which now have PRs available (KAFKA-17608, KAFKA-17604). Once those are in, I'll create a new RC. thanks, all. Coli

<    9   10   11   12   13   14   15   16   17   18   >