Re: [DISCUSS] Apache Kafka 3.8.0 release

2024-06-14 Thread Colin McCabe
+. best, Colin On Fri, Jun 14, 2024, at 15:37, Matthias J. Sax wrote: > That's my understanding, and I would advocate strongly to keep the 4.0 > release schedule as planed originally. > > The 3.9 one should really be an additional "out of schedule" release not >

Re: [DISCUSS] Apache Kafka 3.9.0 release

2024-06-14 Thread Colin McCabe
e 3.8 JIRA backlog soon as well. I expect most of that stuff to get shifted to 3.9 unless it truly is a blocker. best, Colin On Fri, Jun 14, 2024, at 15:48, Matthias J. Sax wrote: > +1 > > Btw: I just looked into Jira, and we have 90 tickets with "fixed > version" set to 4.0.0.

Re: [DISCUSS] Apache Kafka 3.9.0 release

2024-06-17 Thread Colin McCabe
hould fit within the normal timeframes). That being said, if people want a shorter 4.0, I'm open to that, as long as we're confident we could actually do that :) best, Colin On Fri, Jun 14, 2024, at 20:56, Sophie Blee-Goldman wrote: > +1, thank you Colin > > Given the July f

Re: [VOTE] KIP-1042: Support for wildcard when creating new acls

2024-06-17 Thread Colin McCabe
s with the propsed changes, versus without them. best, Colin On Mon, Jun 17, 2024, at 01:26, Muralidhar Basani wrote: > Hi all, > > I would like to call a vote on KIP-1042 which extends creation of acls with > MATCH pattern type. > > KIP - > https://cwiki.apache.org/confluence/di

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-06-17 Thread Colin McCabe
be considering contributing that back to the community, so anything that makes that harder or less likely to happen seems like a negative. best, Colin On Mon, Jun 17, 2024, at 11:35, Diop, Assane wrote: > Hi Greg, > Thank you for your thoughtful response. > Our motivation is to enable ne

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-06-21 Thread Colin McCabe
- The server will simply leave out the features whose minimum supported value is 0 for clients that send v3 - ApiVersionsRequest v4 will be supported in AK 3.9 and above. AK 3.8 will ship with ApiVersionsRequest v3 just as today. thanks, Colin On Mon, Apr 15, 2024, at 11:01, Justine Olshan

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-06-21 Thread Colin McCabe
gh to send a v0, v1, or v2 ApiVersionsRequest anyway) best, Colin On Fri, Jun 21, 2024, at 16:46, Justine Olshan wrote: > Thanks Colin, > > This makes sense to me. Namely in the case where we perhaps don't want to > support version 0 anymore, we need the range to be able to not i

[DISCUSS] KIP-1014: Managing Unstable Features in Apache Kafka

2024-06-25 Thread Colin McCabe
res, MVs, or RPCs. Please take a look. best, Colin

Re: [DISCUSS] KIP-1014: Managing Unstable Features in Apache Kafka

2024-06-26 Thread Colin McCabe
Hi Justine, Since these configurations are (and have always been) internal, changing or removing them should be no problem. best, Colin On Tue, Jun 25, 2024, at 16:51, Justine Olshan wrote: > Hey Colin, > > I made a single configuration as part of KIP-10

Re: [DISCUSS] KIP-1014: Managing Unstable Features in Apache Kafka

2024-06-26 Thread Colin McCabe
On Wed, Jun 26, 2024, at 12:09, Jun Rao wrote: > Hi, Colin, > > Thanks for restarting the discussion. A few comments. > > 1. "An unstable RPC version can be changed at any time, until it becomes > stable." > > What's our recommendation to non-java develop

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-06-26 Thread Colin McCabe
Hi Justine, Yes, that was what I was thinking. best, Colin On Mon, Jun 24, 2024, at 11:11, Justine Olshan wrote: > My understanding is that the tools that don't rely on ApiVersions should > still return 0s when it is the correct value. I believe these commands do > not require thi

Re: [DISCUSS] Apache Kafka 3.9.0 release

2024-06-26 Thread Colin McCabe
Hi Mario and Nelson, Thanks for asking. Both of these KIPs can certainly go in 3.9 if we can hit the deadlines. If you need an extra day or two just ping me. (But I don't want to extend things for too long!) :) Do we feel that KIP-1040 can make feature freeze? best, Colin On Tue, J

Re: [DISCUSS] KIP-1014: Managing Unstable Features in Apache Kafka

2024-06-26 Thread Colin McCabe
On Wed, Jun 26, 2024, at 13:16, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. > > 1. https://kafka.apache.org/protocol.html#The_Messages_ConsumerGroupDescribe > lists ConsumerGroupDescribeRequest, whose latest version is unstable. > Hi Jun, I think that is a bug.

Re: [DISCUSS] KIP-1014: Managing Unstable Features in Apache Kafka

2024-06-26 Thread Colin McCabe
we allow non-developers to use KIP-1014 features, we'll get bogged down in compatibility discussions (no matter what the KIP says). best, Colin On Wed, Jun 26, 2024, at 14:49, Jun Rao wrote: > Hi, Colin, > > Thanks for the reply. > > 4. "A developer could modify the code

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-06-27 Thread Colin McCabe
tures enabled, to make sure it looks the way we want. best, Colin On Thu, Jun 27, 2024, at 14:01, Jun Rao wrote: > Hi, Colin, > > ApiVersionResponse includes both supported and finalized features. If we > only suppress features in the supported field, but not in the finalized >

Re: [DISCUSS] KIP-1014: Managing Unstable Features in Apache Kafka

2024-06-27 Thread Colin McCabe
le is that people will start to rely on it, and expect compatibility guarantees that aren't there. It would also be good to understand how many EA features will need or want radical metadata changes before going GA. We did try pretty hard to avoid this in cases like JBOD... best, Colin

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-07-01 Thread Colin McCabe
can translate requests to set the finalized level to 0, into requests to set it to 1. So maybe this solution is worth considering, although it's unfortunate to lose 0. I suppose we'd have to special case metadata.version being set to 1, since that was NOT equivalent to it being "

Re: [DISCUSS] Apache Kafka 3.9.0 release

2024-07-08 Thread Colin McCabe
Hi Federico, Thanks for the KIP. I've added it to the release plan. I hope we can get it in this week or next, since feature freeze is imminent. Colin On Sun, Jun 30, 2024, at 23:29, Federico Valeri wrote: > Hi Colin, is it possible to include this small KIP in the release plan? &

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-07-08 Thread Colin McCabe
Hi all, I've updated the approach in https://github.com/apache/kafka/pull/16421 so that we change the minVersion=0 to minVersion=1 in older ApiVersionsResponses. I hope we can get this in soon and unblock the features that are waiting for it! best, Colin On Wed, Jul 3, 2024, at 10:55, Ju

New release branch 3.9

2024-07-29 Thread Colin McCabe
to trunk. *Blockers (existing and new that we discover while testing the release) will be double-committed. *Please discuss with your reviewer whether your PR should go to trunk or to trunk+release so they can merge accordingly. *Please help us test the release! * best, Colin

Re: [kafka-clients] [ANNOUNCE] Apache Kafka 3.8.0

2024-07-29 Thread Colin McCabe
+1. Thanks, Josep! Colin On Mon, Jul 29, 2024, at 10:32, Chris Egerton wrote: > Thanks for running the release, Josep! > > > On Mon, Jul 29, 2024, 13:31 'Josep Prat' via kafka-clients > wrote: >> The Apache Kafka community is pleased to announce the rel

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-07-29 Thread Colin McCabe
Hi Jun, Just to close the loop on this... the KIP now mentions both ApiVersionResponse and BrokerRegistrationRequest. best, Colin On Mon, Jul 8, 2024, at 14:57, Jun Rao wrote: > Hi, Colin, > > Thanks for the update. Since the PR also introduces a new version of > BrokerRegistr

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
Hi all, KIP-853 is shipping in 3.9. If we have to delay 3.9 to accomplish this, we will, but that seems very unlikely at this point. We are mostly on schedule so far. Trunk is 4.0. best, Colin On Tue, Jul 30, 2024, at 15:35, Greg Harris wrote: > Hi Matthias, > > I agree with you.

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
On Tue, Jul 30, 2024, at 08:59, José Armando García Sancio wrote: > Thanks Colin. > > For KIP-853 (KRaft Controller Membership Changes), we still have the > following features that are in progress. > > 1. UpdateVoter RPC and request handling > <https://issues.apache.org

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
sure we > can them all into 4.0 release, but of course also don't want to work on > them prematurely (to avoid that we have to revert them after merging). > > > -Matthias Hi Matthias, If you have something you want in 4.0, please feel free to add it to trunk. Trunk is 4.0

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
ads on the mailing list about it. So it certainly would make no sense to ship 3.9 without KIP-853. If you have a specific feature you want in 3.9, please let me know and I will see if we can make it in. best, Colin > > In that case, a straightforward revert would be the way forward: >

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
Raft work > would only take a few more weeks, and that is also why we have a 3.9 > release branch already... > > If there is so much uncertainty about finishing KRaft work, why did we > cut a 3.9 branch now, and have a dedicate release plan for it? Colin did > propose the r

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
Hi Chia-Ping Tsai, If you can get them done this week then I think we can merge them in to 3.9. If not, then let's wait until 4.0, please. best, Colin On Tue, Jul 30, 2024, at 09:07, Chia-Ping Tsai wrote: > hi Colin, > > Could you please consider adding > https://issues.apach

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
ng at now. If you have a feature that you've been waiting for 4.0 for, then you are unblocked. Please feel free to add it to trunk. best, Colin On Tue, Jul 30, 2024, at 14:13, Igor Soarez wrote: > My understanding was that the reason for the shorter cycle > to the 3.9 release

Re: New release branch 3.9

2024-07-30 Thread Colin McCabe
Yeah, please go ahead. I know a lot of people are waiting for 4.0. best, Colin On Tue, Jul 30, 2024, at 16:05, Matthias J. Sax wrote: > Thanks for clarifying Colin. So my assumptions were actually correct. > > We have a lot of contributors waiting to pick-up 4.0 tickets, and I'll

Re: [DISCUSS] Graduation steps for Features

2024-07-31 Thread Colin McCabe
e a strong preference on terms. I think no matter how much standardization we do, there will always be a lot of discretion in these terms :) But maybe this helps clarify how they've been used in the past? best, Colin On Wed, Jul 31, 2024, at 02:03, Josep Prat wrote: > Hi Kafka devs, >

Re: New release branch 3.9

2024-08-01 Thread Colin McCabe
we'll be feature-complete on KIP-853. best, Colin On Thu, Aug 1, 2024, at 09:23, Mickael Maison wrote: > Hi, > > Considering KIP-853 does not look like it's near completion (20 out of > 49 tasks done in https://issues.apache.org/jira/browse/KAFKA-14094), > I'm not

Re: New release branch 3.9

2024-08-01 Thread Colin McCabe
Hi Chris, I don't see why this shouldn't go to 3.9. Would you call this a bugfix? best, Colin On Wed, Jul 31, 2024, at 13:54, Chris Egerton wrote: > Hi Colin, > > Would it be alright if I cherry-picked > https://github.com/apache/kafka/pull/16678 onto 3.9? This isn'

Re: New release branch 3.9

2024-08-01 Thread Colin McCabe
Hi Josep, I don't have a strong opinion on whether KAFKA-16524 should land in 3.9.1. Let's discuss that more on the mailing list when it's time for 3.9.1. best, Colin On Thu, Aug 1, 2024, at 12:07, Josep Prat wrote: > Hi Colin, > Will JIRAs like adding metrics ( > h

Re: New release branch 3.9

2024-08-01 Thread Colin McCabe
t the "refactor" ones :) As I said, we'll shoot for getting the remaining feature tasks for KIP-853 done in a few days. Probably by Monday. best, Colin On Thu, Aug 1, 2024, at 12:18, Colin McCabe wrote: > Hi Josep, > > I don't have a strong opinion on whether KAF

Re: [DISCUSS] Graduation steps for Features

2024-08-01 Thread Colin McCabe
t important thing. Unfortunately we >don't seem to be consistent on this; some EA features moved smoothly to GA >with no compatibility breaks, while others did drop compat. It could be useful >to have a term for features that for which upgrade is not supported. Perhaps >"experimenta

Re: [VOTE] KIP-1022 Formatting and Updating Features

2024-08-01 Thread Colin McCabe
Hmm, I thought I had added that already. I guess I missed it. Sorry for the confusion, and thanks for the update. best, Colin On Tue, Jul 30, 2024, at 15:06, Jun Rao wrote: > Thanks for updating the KIP, Justine. > > Jun > > On Tue, Jul 30, 2024 at 1:37 PM Justine Olshan > w

Re: New release branch 3.9

2024-08-08 Thread Colin McCabe
veryone who has worked on this release, and thanks for your patience. I do believe we will deliver on the promise of a much-shorter-than-usual release cycle, even with the 2 week delay. best, Colin

Re: New release branch 3.9

2024-08-09 Thread Colin McCabe
Hi Chia-Ping Tsai, Thank you! And thanks for your own contributions, especially the many code reviews (not only of my PRs, but everyone's). best, Colin On Thu, Aug 8, 2024, at 19:14, Chia-Ping Tsai wrote: >> - We have completed all the feature work for KIP-853: KRaft controller &g

Re: New release branch 3.9

2024-08-09 Thread Colin McCabe
us as Done KIP-996: Postponed to subsequent release KIP-994: Postponed to subsequent release KIP-997: Postponed to subsequent release KIP-966: Postponed to subsequent release KIP-956: Postponed to subsequent release KIP-950: Marked 3.9 status as Done best, Colin On Fri, Aug 9, 2024, at 14

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

2024-08-09 Thread Colin McCabe
broker's information in DescribeClusterResponse. Then you would probably add something like a list-brokers action to the kafka-cluster.sh command line. It makes more sense to put it there anyway, since that's where unregister is. best, Colin On the thread you linked, you pointed

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

2024-08-09 Thread Colin McCabe
On Fri, Aug 9, 2024, at 14:46, Colin McCabe wrote: > Hi Gantigmaa, > > I agree with you that we need a way to list fenced brokers. > > However, I don't think this KIP is the right way to do that. This KIP, > if I understand correctly, is about listing all Raft observer

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

2023-12-04 Thread Colin McCabe
Yes, this should be CLUSTER_ACTION on CLUSTER, to be consistent with our other internal RPCs. best, Colin On Mon, Dec 4, 2023, at 04:28, Viktor Somogyi-Vass wrote: > Hi Igor, > > I'm just reading through your KIP and noticed that the new protocol you > created doesn't s

Re: [DISCUSS] KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)

2023-12-26 Thread Colin McCabe
be a good idea? I'm not sure what the easiest way to build just one module is -- hopefully we don't have to go through maven or something. best, Colin On Fri, Dec 22, 2023, at 10:39, Ismael Juma wrote: > Hi all, > > I was watching the Java Highlights of 2023 from Nicolai Pa

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-28 Thread Colin McCabe
Congratulations, Divij! best, Colin On Thu, Dec 28, 2023, at 11:38, Bruno Cadonna wrote: > Congratulations Divij! Well deserved! > > Best, > Bruno > > On 12/27/23 12:45 PM, Luke Chen wrote: >> Hi, Everyone, >> >> Divij has been a Kafka committer since June,

Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release

2023-12-28 Thread Colin McCabe
ty mechanisms and new features will not be available in ZK mode, and give a link to our documentation about migration. We should probably also move the example configurations for kraft from config/kraft to config. And move the zk ones into config/zk. Or maybe even drop the ZK ones altogether, s

Re: [DISCUSS] Road to Kafka 4.0

2023-12-28 Thread Colin McCabe
Hi all, Let's continue this dicsussion on the "[DISCUSS] KIP-1012: The need for a Kafka 3.8.x release" email thread. Colin On Tue, Dec 26, 2023, at 12:50, José Armando García Sancio wrote: > Hi Divij, > > Thanks for the feedback. I agree that having a 3.8 release is

Re: Apache Kafka 3.7.0 Release

2023-12-28 Thread Colin McCabe
. Right now I'm leaning towards just making it GA since that's how most features work. It's kind of rare for us to do a multi-step rollout for new features. best, Colin On Wed, Dec 20, 2023, at 03:43, Mickael Maison wrote: > Hi, > > With the current timeline for 3.7, I t

Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release

2023-12-28 Thread Colin McCabe
On Thu, Dec 28, 2023, at 18:17, Justine Olshan wrote: > Hey Colin, > > Some folks were concerned about the lack of automatic unclean leader > election. I mentioned that KIP-966 would actually be better with its > aggressive recovery option. > I think folks were hoping for some

Re: [DISCUSS] Road to Kafka 4.0

2024-01-04 Thread Colin McCabe
3.8. Do we have to get KIP-719 into 3.8 so that we have a reasonable deprecation period? Also, if we do upgrade, I agree with Ismael that we should consider going to log4j3. Assuming they have a non-beta release by the time 4.0 is ready. best, Colin On Thu, Jan 4, 2024, at 03:08, Mickael Mais

Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release

2024-01-04 Thread Colin McCabe
a tentative schedule. https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.8.0 Please let me know if these dates make sense -- they're just proposals right now. best, Colin On Thu, Dec 28, 2023, at 20:14, Colin McCabe wrote: > On Thu, Dec 28, 2023, at 18:17, Justine Olsh

Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release

2024-01-05 Thread Colin McCabe
-966, or something else). Please revise this. best, Colin On Fri, Jan 5, 2024, at 02:50, Anton Agestam wrote: > +1 from me. > > Den fre 5 jan. 2024 kl 10:33 skrev Josep Prat : > >> Hi all, >> >> I'd like to start a vote on KIP-1012: >> >> https://cwiki.

Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release

2024-01-05 Thread Colin McCabe
t what we need to have in the last 3.x release. The dates are there to give people a sense of when that release will be. Also, if you have other stuff you want to get into 3.8, please submit PRs / KIPs and we'll follow the normal process as usual. Actually, this is something we should state

Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release

2024-01-05 Thread Colin McCabe
esides the required ones best, Colin On Fri, Jan 5, 2024, at 13:27, Colin McCabe wrote: > Hi, > > Thanks for calling the vote, Josep. > > I re-checked this, and saw something that we missed updating. One thing > we talked about earlier is that KIP-966 is actually not required for >

Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release

2024-01-05 Thread Colin McCabe
Hi Justine, If there are more things you think are needed in 3.8 for migration, let's discuss in the VOTE thread. best, Colin On Fri, Jan 5, 2024, at 09:23, Justine Olshan wrote: > While I agree we should have this release and should vote on it soon, is it > worth determining the

Re: Apache Kafka 3.7.0 Release

2024-01-05 Thread Colin McCabe
metadata is an internal gradle module. It is not used by clients. So I don't see why you would want to publish it (unless I'm misunderstanding something). best, Colin On Fri, Jan 5, 2024, at 10:05, Stanislav Kozlovski wrote: > Thanks for reporting the blockers, folks. Good job f

Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release

2024-01-08 Thread Colin McCabe
Thanks for the changes. +1 (binding) best, Colin On Sun, Jan 7, 2024, at 23:43, Josep Prat wrote: > Hi all, > > Thanks for your comments, > I reworded some parts of the KIP to express that: > - The KIP is to agree that we need at least one more minor version in the > 3.x ser

Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release

2024-01-08 Thread Colin McCabe
ne is very rough" to KIP-833 and it caued all kinds of confusion. So overall I'd prefer to leave the language about 4.0 unchanged. best, Colin > > Cheers, > > Chris > > On Mon, Jan 8, 2024 at 11:41 AM Greg Harris > wrote: > >> Thanks Josep for leading the

Re: Apache Kafka 3.7.0 Release

2024-01-08 Thread Colin McCabe
Hi Apporv, Please remove the metadata module from any artifacts published for clients. It is only used by the server. best, Colin On Sun, Jan 7, 2024, at 03:04, Apoorv Mittal wrote: > Hi Colin, > Thanks for the response. The only reason for asking the question of > publishing the me

Re: Apache Kafka 3.7.0 Release

2024-01-08 Thread Colin McCabe
Oops, hit send too soon. I see that #15127 was already merged. So we should no longer be publishing :metadata as part of the clients artifacts, right? thanks, Colin On Mon, Jan 8, 2024, at 11:42, Colin McCabe wrote: > Hi Apporv, > > Please remove the metadata module from any

Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release

2024-01-08 Thread Colin McCabe
On Fri, Jan 5, 2024, at 14:50, Greg Harris wrote: > Hi Colin, > > Thanks for the quick reply! > >> If we cannot get KIP-853 done in time for 3.8, then we'd move to have >> another 3.x release. > > This is the crux of my concern, and this is satisfactory. Thi

Re: [DISCUSS] KIP-1015: Limit number of ssl connections in brokers

2024-01-08 Thread Colin McCabe
In that case, it's hardly worth having a separate limit for SSL, since the number of plaintext connections is bounded, and low. But perhaps your use-case is different. Do you really think you'll have both a lot of PLAINTEXT and a lot of SSL connections? cheers, Colin On Sat, Jan 6,

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-08 Thread Colin McCabe
ably won't work, and you won't be able to upgrade. (It was unstable, we told you not to use it.) best, Colin On Fri, Jan 5, 2024, at 07:32, Proven Provenzano wrote: > Hey folks, > > I am starting a discussion thread for managing unstable metadata > versions > in Apac

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-08 Thread Colin McCabe
way, the contents of the 3.7 branch will change up until 3.7.0 is released. Once it gets released, it's never unreleased. We just move on to 3.7.1. Same thing here. best, Colin > > Thanks, > > Justine > > On Mon, Jan 8, 2024 at 12:44 PM Colin McCabe wrote: > >>

Re: Apache Kafka 3.7.0 Release

2024-01-08 Thread Colin McCabe
published, but maybe other private server-side gradle modules are getting published. This seems bad. Is there a reason to publish modules that are only used by the server on Sonatype? best, Colin On Mon, Jan 8, 2024, at 12:50, Ismael Juma wrote: > Hi Colin, > > I think you may have misu

Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release

2024-01-08 Thread Colin McCabe
the issue here. I also feel that 4 months should be enough (for anyone? :) ) But I'm always an optimist. best, Colin On Mon, Jan 8, 2024, at 13:03, Chris Egerton wrote: > Hi Colin, > > The idea isn't to hold up 4.0.0 any longer; in fact, it's the opposite. If > things s

Re: Apache Kafka 3.7.0 Release

2024-01-08 Thread Colin McCabe
Hi Ismael, I wasn't aware of that. If we are required to publish all modules, then this is working as intended. I am a bit curious if we've discussed why we need to publish the server modules to Sonatype. Is there a discussion about the pros and cons of this somewhere? regards, Col

Re: Apache Kafka 3.7.0 Release

2024-01-08 Thread Colin McCabe
On an unrelated note, I found a blocker bug related to upgrades from 3.6 (and earlier) to 3.7. The JIRA is here: https://issues.apache.org/jira/browse/KAFKA-16094 Fix here: https://github.com/apache/kafka/pull/15153 best, Colin On Mon, Jan 8, 2024, at 14:47, Colin McCabe wrote: >

Re: Kafka trunk test & build stability

2024-01-09 Thread Colin McCabe
issue. regards, Colin On Tue, Jan 2, 2024, at 07:00, Nick Telford wrote: > Addendum: I've opened a PR with what I believe are the changes necessary to > enable Remote Build Caching, if you choose to go that route: > https://github.com/apache/kafka/pull/15109 > > On Tue, 2 Jan 20

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-09 Thread Colin McCabe
s add more examples of what we can do (which is anything) :) best, Colin On Mon, Jan 8, 2024, at 14:18, Justine Olshan wrote: > Hey Colin, > > I had some offline discussions with Proven previously and it seems like he > said something different so I'm glad I brought it up here. &

Re: Apache Kafka 3.7.0 Release

2024-01-09 Thread Colin McCabe
KAFKA-16094 has been fixed and backported to 3.7. Colin On Mon, Jan 8, 2024, at 14:52, Colin McCabe wrote: > On an unrelated note, I found a blocker bug related to upgrades from > 3.6 (and earlier) to 3.7. > > The JIRA is here: > https://issues.apache.org/jira/browse/KAFK

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

2024-01-09 Thread Colin McCabe
le just format them all). 5. Removing a controller I think in this case, we can have an explicit command. This is similar to the broker case, where we have the "kafka-cluster.sh unregister" command. best, Colin On Mon, Jan 8, 2024, at 10:13, José Armando García Sancio wrote: > Hi all,

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-09 Thread Colin McCabe
example. Nobody would have said that it was production ready in 3.7 ... hence it belonged (and still belongs) in an unstable MV, until that changes (hopefully soon :) ) best, Colin > --Proven > > On Tue, Jan 9, 2024 at 3:26 PM Colin McCabe wrote: > >> Hi Justine, >> >>

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

2024-01-09 Thread Colin McCabe
re. Agreed. Voter changes should be extremely rare. Listing the full set makes debugging easier as well. > 4. Should ReplicaUuid in FetchRequest be a tagged field? It seems like a > lot of overhead for all consumer fetches. +1 best, Colin > > On Mon, Jan 8, 2024 at 10:13 AM José Arma

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-10 Thread Colin McCabe
On Wed, Jan 10, 2024, at 09:16, Justine Olshan wrote: > Hmm it seems like Colin and Proven are disagreeing with whether we can swap > unstable metadata versions. > >> When we reorder, we are always allocating a new MV and we are never > reusing an existing MV even if it

Re: Logging in Kafka

2024-01-10 Thread Colin McCabe
t happen. best, Colin On Wed, Jan 10, 2024, at 09:13, Ismael Juma wrote: > Hi Viktor, > > A logging library that requires Java 17 is a deal breaker since we need to > log from modules that will only require Java 11 in Apache Kafka 4.0. > > Ismael > > On Wed, Jan 10, 2024

Re: [VOTE] 3.7.0 RC2

2024-01-14 Thread Colin McCabe
ere isn't really unavailability since a ZK controller can be elected quickly after the kraft quorum is gone. >> Further, since we will have a 3.8 release - it is >> likely we will ultimately recommend users upgrade from that version given >> its aim is to have strategic K

Re: [VOTE] 3.7.0 RC2

2024-01-15 Thread Colin McCabe
Docs fix discussed in the thread is here: https://github.com/apache/kafka/pull/15193 best, Colin On Sun, Jan 14, 2024, at 23:56, Colin McCabe wrote: > Hi Stanislav, > > Thanks for making the first RC. The fact that it's titled RC2 is > messing with my mind a bit. I hope

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-15 Thread Colin McCabe
k we do want them to be able to go into trunk. If they have to go into a branch, then there is actually no need for any of this. Doing big features in long-lived branches is one genuine alternative to unstable MVs, I think. But it's not an alternative that works well with our current tooling

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

2024-01-29 Thread Colin McCabe
> request protocol that worked under IV4 doesn't work now. > Upgrades are not supported if you are running an unstable MV. It's intended only for testing. Colin > > Thanks, > > Jun > > On Fri, Jan 19, 2024 at 2:13 PM Artem Livshits > wrote: > >> Hi Col

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-07 Thread Colin McCabe
wngrade (if any) and do them carefully one at a time. The shotgun approach, where you upgrade everything at once, actually isn't what I think most users want to do in production. best, Colin On Fri, Mar 1, 2024, at 13:25, Jun Rao wrote: > Hi, Justine, > > Thanks for the reply. >

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-15 Thread Colin McCabe
27;t allow a command that moves a version in the wrong direction >(when upgrade actually downgrades a feature for example) >- allow a command to describe all the feature versions for a given >metadata version > > Note that Colin expressed some concern with having the --r

Re: [VOTE] 3.6.2 RC1

2024-03-22 Thread Colin McCabe
what the issue is here. best, Colin On Thu, Mar 21, 2024, at 19:32, Chia-Ping Tsai wrote: > hi Manikumar > >> Pls let me know after merging the PR. I will generate RC2 later today. > > Sure. We will complete it ASAP > > >> Manikumar 於 2024年3月22日 上午9:26 寫道: >&

Re: [VOTE] 3.6.2 RC1

2024-03-22 Thread Colin McCabe
I have created a PR to fix this here : https://github.com/apache/kafka/pull/15584 best, Colin On Fri, Mar 22, 2024, at 13:28, Colin McCabe wrote: > Sorry but I have to vote -1 > > I tried verifying that the migration quotas bug described in > https://issues.apache.org/jira/browse

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-26 Thread Colin McCabe
ld need to be run, if you like. But let them decide whether to pull the trigger on each upgrade or downgrade. This also avoids having to solve the thorny issue of how to have a single RPC do both upgrades and downgrades. best, Colin On Tue, Mar 26, 2024, at 14:59, Justine Olshan wrote: > Hi a

[ANNOUNCE] New committer: Igor Soarez

2024-04-24 Thread Colin McCabe
also contributed to discussing and reviewing many KIPs such as KIP-690, KIP-554, KIP-866, and KIP-938. Congratulations, Igor! Thanks, Colin (on behalf of the Apache Kafka PMC)

Re: [DISCUSS] KIP-1073 Return inactive observer nodes in DescribeQuorum response

2024-08-22 Thread Colin McCabe
n and hence will not > request fenced brokers. You need to describe what happens when the client requests to see fenced brokers, but the protocol doesn't support that. I would suggest UnsupportedVersionException. best, Colin On Mon, Aug 19, 2024, at 08:29, Gantigmaa Selenge wrote: &

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

2024-08-23 Thread Colin McCabe
ems... odd. :) If we get rid of the level 1 (which doesn't seem useful to me, as I've explained above), having "alpha", "beta", and "production" seems very easy to understand and friendly for our users. (Or, if you really absolutely need to have level 1, you c

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

2024-08-23 Thread Colin McCabe
ed multiple steps, I wonder if we need anything different than what we have now (just a "release version" field). best, Colin > > AS3: For KIPs which end up with some pieces being completed before others, > it would be simple to document the sub-features which were delivered > an

Re: Question about ResourcePatternFilter

2024-08-23 Thread Colin McCabe
Hi Claude, As I recall, the intention was that if PatternType is ANY, the name field would simply be ignored. Probably we should have thrown an exception if someone tried to create a ResourcePatternFilter with a non-null name and PatternType ANY. best, Colin On Tue, Aug 20, 2024, at 04:19

Re: New release branch 3.9

2024-08-23 Thread Colin McCabe
Thanks for the heads up, Josep. I saw both emails but I didn't make the connection until now. It does seem likely that we'll have to modify this (either for 3.9... or soon.) best, Colin On Tue, Aug 13, 2024, at 05:09, Josep Prat wrote: > Hi Colin, > > Just to make sur

Re: New release branch 3.9

2024-08-23 Thread Colin McCabe
I wanted to give a quick update on 3.9. Code freeze is coming up next Thursday on the 29th. As usual, after that point, only critical bug fixes will be accepted. Thanks again to everyone who has helped with this release. best, Colin On Fri, Aug 23, 2024, at 13:18, Colin McCabe wrote

Re: [VOTE] KIP-1042: Support for wildcard when creating new acls

2024-08-23 Thread Colin McCabe
he single case mentioned > in the KIP documentation. > > If there are no issues with this code, I will complete the documentation > and fix the checkstyle and then open a pull request. It's fine to open a pull request. You can link it on the KIP page. As always, we don't wa

Re: [DISCUSS] Apache Kafka 3.9.0 release

2024-08-23 Thread Colin McCabe
Hi Loic, I think I responded in another thread, but just to close the loop here, KIP-1033 is indeed in 3.9. regards, Colin On Mon, Jul 22, 2024, at 08:13, Loic Greffier wrote: > Hello, > > Can KIP-1033 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1033%3A+Add+K

Re: [DISCUSS] KIP-1081: Graduation Steps for Features

2024-08-26 Thread Colin McCabe
will react to "that feature is on level 2" with incomprehension. On the other hand, if you tell them that the feature is "alpha," they'll get what you're saying. Let's not add jargon that our users won't understand. best, Colin > Best, > Viktor &g

Re: [VOTE] KIP-1042: Support for wildcard when creating new acls

2024-08-29 Thread Colin McCabe
On Thu, Aug 29, 2024, at 01:34, Claude Warren, Jr wrote: > Colin, > > Thanks for your insightful comments. I came to the same conclusion. > > I do have 2 Jira tickets to simplify some of this. > > 1) KAFKA-17316 <https://issues.apache.org/jira/browse/KAFKA-17316>

Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change

2024-08-29 Thread Colin McCabe
forgetting to turn off unclean leader election afterwards, and it's faster. You should only set the configurations for unclean leader election if you want to leave it on forever. (i.e. you genuinely don't care about losing data, and just always want a leader.) best, Colin On Wed, Aug 28

Re: [DISCUSS] KIP-1086: Add ability to specify a custom produce request parser.

2024-08-29 Thread Colin McCabe
ever add server-side interceptors, we'll want to give access to records, not to internal data structures like Requests. This could involve a big performance hit, but perhaps people will be willing to pay it. best, Colin On Thu, Aug 29, 2024, at 09:36, Maxim Fortun wrote: > Oooh! A

Re: New release branch 3.9

2024-08-29 Thread Colin McCabe
e not blockers for 3.9. In a few cases, I asked a question about whether the JIRA was a blocker or not. If all goes according to plan, we should now have two more weeks of stabilization before the release. Thanks, everyone. Colin On Fri, Aug 23, 2024, at 13:30, Colin McCabe wrote: > I wanted t

Re: [DISCUSS] KIP-1086: Add ability to specify a custom produce request parser.

2024-08-29 Thread Colin McCabe
uture while maintaining the compatibility promised in this KIP. So this would be a very significant constraint on development, which we probably wouldn't adopt unless there was no other way forward. best, Colin On Thu, Aug 29, 2024, at 11:27, Maxim Fortun wrote: > Hi Colin, > T

<    4   5   6   7   8   9   10   11   12   13   >