Hi Vijay,
The current progress is tracked under the ticket
https://issues.apache.org/jira/browse/KAFKA-9119 as sub-tasks. The first
milestone would be moving off all admin request handling out of ZK and let
them to talk to brokers.
Guozhang
On Wed, Mar 11, 2020 at 8:23 AM vijay gupta wrote:
>
Hi,
What’s the latest status on this JIRA?
Regards,
Vijay
On 2019/11/06 15:07:03 Viktor Somogyi-Vass wrote:
> Hi Elmahdi,
>
> I've added the JIRA to the KIP (and also below) where you can track the
> progress (but more subtask will come as the current ones don't represent
> the full work to be
Hi Elmahdi,
I've added the JIRA to the KIP (and also below) where you can track the
progress (but more subtask will come as the current ones don't represent
the full work to be done).
https://issues.apache.org/jira/browse/KAFKA-9119
Viktor
On Tue, Nov 5, 2019 at 5:15 PM Elmahdi FRID wrote:
> H
Hello Folks any status abbout this kip and it's possible to test this use case ?
On 2019/08/01 21:04:46, "Colin McCabe" wrote:
> Hi all,
>
> I've written a KIP about removing ZooKeeper from Kafka. Please take a look
> and let me know what you think:
>
> https://cwiki.apache.org/confluence/di
Thanks, Colin. That all makes sense, especially the part about the onerous
testing requirements associated with supporting both Zookeeper and the new
metadata quorum simultaneously. Given that, I now buy into the idea that
the transition to the new metadata quorum becomes the main path forward
on
On Mon, Sep 2, 2019, at 07:51, Ron Dagostino wrote:
> Hi Colin. It is not unusual for customers to wait before upgrading —
> to avoid so-called “point-zero” releases — to avoid as many of the
> inevitable bugs that ride along with new functionality as possible.
> Removal of Zookeeper is going
Hi Colin. It is not unusual for customers to wait before upgrading — to avoid
so-called “point-zero” releases — to avoid as many of the inevitable bugs that
ride along with new functionality as possible. Removal of Zookeeper is going
feel especially risky to these customers, and arguably it is
Hi Ryanne,
Good point. I added a section titled "future work" with information about the
follow-on KIPs that we discussed here.
best,
Colin
On Fri, Aug 23, 2019, at 13:15, Ryanne Dolan wrote:
> Thanks Colin, sgtm. Please make this clear in the KIP -- otherwise it is
> hard to nail down what w
Thanks Colin, sgtm. Please make this clear in the KIP -- otherwise it is
hard to nail down what we are voting for.
Ryanne
On Fri, Aug 23, 2019, 12:58 PM Colin McCabe wrote:
> On Fri, Aug 23, 2019, at 06:24, Ryanne Dolan wrote:
> > Colin, can you outline what specifically would be in scope for
On Fri, Aug 23, 2019, at 06:24, Ryanne Dolan wrote:
> Colin, can you outline what specifically would be in scope for this KIP vs
> deferred to the follow-on KIPs you've mentioned? Maybe a Future Work
> section? Is the idea to get to the bridge release with this KIP, and then
> go from there?
>
> R
Colin, can you outline what specifically would be in scope for this KIP vs
deferred to the follow-on KIPs you've mentioned? Maybe a Future Work
section? Is the idea to get to the bridge release with this KIP, and then
go from there?
Ryanne
On Thu, Aug 22, 2019, 11:58 AM Colin McCabe wrote:
> On
On Wed, Aug 21, 2019, at 19:48, Ron Dagostino wrote:
> Thanks, Colin. The changes you made to the KIP related to the bridge
> release help make it clearer. I still have some confusion about the phrase
> "The rolling upgrade from the bridge release will take several steps."
> This made me think yo
Thanks, Colin. The changes you made to the KIP related to the bridge
release help make it clearer. I still have some confusion about the phrase
"The rolling upgrade from the bridge release will take several steps."
This made me think you are talking about moving from the bridge release to
some ot
Hi Ryanne,
Apache Ratis looks like a very interesting project, but I don't think this is
the right use-case for it. At its heart, Apache Kafka is a system for managing
logs. We should avoid adding a dependency on an external system to manage the
logs of Kafka itself, since that is one of Kafk
Colin, have you considered leveraging Apache Ratis (incubating)?
Ryanne
On Wed, Aug 21, 2019, 1:28 PM Colin McCabe wrote:
> On Wed, Aug 21, 2019, at 06:38, Eno Thereska wrote:
> > Hi Colin,
> >
> > Nice KIP! For such a big change it would be good to add a pointer or
> > two to related work that
On Tue, Aug 20, 2019, at 07:19, Ron Dagostino wrote:
> Hi Colin. The diagram up at the top confused me -- specifically, the lines
> connecting the controller/active-controller to the brokers. I had assumed
> the arrows on those lines represented the direction of data flow, but that
> is not the c
On Wed, Aug 21, 2019, at 04:22, Ron Dagostino wrote:
> Hi Colin. I like the concept of a "bridge release" for migrating off of
> Zookeeper, but I worry that it may become a bottleneck if people hesitate
> to replace Zookeeper -- they would be unable to adopt newer versions of
> Kafka until taking
On Wed, Aug 21, 2019, at 06:38, Eno Thereska wrote:
> Hi Colin,
>
> Nice KIP! For such a big change it would be good to add a pointer or
> two to related work that provides some sort of soft proof that the
> approach taken makes sense. Also such work often builds on other work
> and it might be us
Hi Colin,
Nice KIP! For such a big change it would be good to add a pointer or
two to related work that provides some sort of soft proof that the
approach taken makes sense. Also such work often builds on other work
and it might be useful to trace its roots. May I recommend adding a
pointer to "Ta
Hi Colin. I like the concept of a "bridge release" for migrating off of
Zookeeper, but I worry that it may become a bottleneck if people hesitate
to replace Zookeeper -- they would be unable to adopt newer versions of
Kafka until taking (what feels to them like) a giant leap. As an example,
assum
Hi Colin. The diagram up at the top confused me -- specifically, the lines
connecting the controller/active-controller to the brokers. I had assumed
the arrows on those lines represented the direction of data flow, but that
is not the case; the arrows actually identify the target of the action, a
Hi all,
The KIP has been out for a while, so I'm thinking about calling a vote some
time this week.
best,
Colin
On Mon, Aug 19, 2019, at 15:52, Colin McCabe wrote:
> On Mon, Aug 19, 2019, at 12:52, David Arthur wrote:
> > Thanks for the KIP, Colin. This looks great!
> >
> > I really like the i
On Mon, Aug 19, 2019, at 12:52, David Arthur wrote:
> Thanks for the KIP, Colin. This looks great!
>
> I really like the idea of separating the Controller and Broker JVMs.
>
> As you alluded to above, it might be nice to have a separate
> broker-registration API to avoid overloading the metadata
Thanks for the KIP, Colin. This looks great!
I really like the idea of separating the Controller and Broker JVMs.
As you alluded to above, it might be nice to have a separate
broker-registration API to avoid overloading the metadata fetch API.
When a broker gets a metadata delta, will it be a se
Hi Mickael,
Thanks for taking a look.
I don't think we want to support that kind of multi-tenancy at the controller
level. If the cluster is small enough that we want to pack the controller(s)
with something else, we could run them alongside the brokers, or possibly
inside three of the broker
Thank Colin for kickstarting this initiative.
Just one question.
- A nice feature of Zookeeper is the ability to use chroots and have
several Kafka clusters use the same Zookeeper ensemble. Is this
something we should keep?
Thanks
On Mon, Aug 5, 2019 at 7:44 PM Colin McCabe wrote:
>
> On Mon, A
On Mon, Aug 5, 2019, at 10:02, Tom Bentley wrote:
> Hi Colin,
>
> Thanks for the KIP.
>
> Currently ZooKeeper provides a convenient notification mechanism for
> knowing that broker and topic configuration has changed. While KIP-500 does
> suggest that incremental metadata update is expected to co
On Mon, Aug 5, 2019, at 07:49, Viktor Somogyi-Vass wrote:
> Hey Colin,
>
> I think this is a long-awaited KIP, thanks for driving it. I'm excited to
> see this in Kafka once. I collected my questions (and I accept the "TBD"
> answer as they might be a bit deep for this high level :) ).
> 1.) Are t
Hi Colin,
Thanks for the KIP.
Currently ZooKeeper provides a convenient notification mechanism for
knowing that broker and topic configuration has changed. While KIP-500 does
suggest that incremental metadata update is expected to come to clients
eventually, that would seem to imply that for some
Hey Colin,
I think this is a long-awaited KIP, thanks for driving it. I'm excited to
see this in Kafka once. I collected my questions (and I accept the "TBD"
answer as they might be a bit deep for this high level :) ).
1.) Are there any specific reasons for the Controller just periodically
persist
Thanks for explaining Ismael! Breaking down into follow-up KIPs sounds like
a good idea.
On Sat, Aug 3, 2019 at 10:14 AM Ismael Juma wrote:
> Hi Boyang,
>
> Yes, there will be several KIPs that will discuss the items you describe in
> detail. Colin, it may be helpful to make this clear in the KI
Hi Boyang,
Yes, there will be several KIPs that will discuss the items you describe in
detail. Colin, it may be helpful to make this clear in the KIP 500
description.
Ismael
On Sat, Aug 3, 2019 at 9:32 AM Boyang Chen
wrote:
> Thanks Colin for initiating this important effort!
>
> One question
Thanks Colin for initiating this important effort!
One question I have is whether we have a session discussing the controller
failover in the new architecture? I know we are using Raft protocol to
failover, yet it's still valuable to discuss the steps new cluster is going
to take to reach the stab
On Fri, Aug 2, 2019, at 16:33, Jose Armando Garcia Sancio wrote:
> Thanks Colin for the detail KIP. I have a few comments and questions.
>
> In the KIP's Motivation and Overview you mentioned the LeaderAndIsr and
> UpdateMetadata RPC. For example, "updates which the controller pushes, such
> as Le
Hi Ryanne,
Good idea. I added some of this discussion to the KIP-- in particular, more
about controller failover.
cheers,
Colin
On Fri, Aug 2, 2019, at 13:28, Ryanne Dolan wrote:
> Thanks Colin, that helps. Can we add some of this to the KIP?
>
> Ryanne
>
> On Fri, Aug 2, 2019 at 12:23 PM Co
Thanks Colin for the detail KIP. I have a few comments and questions.
In the KIP's Motivation and Overview you mentioned the LeaderAndIsr and
UpdateMetadata RPC. For example, "updates which the controller pushes, such
as LeaderAndIsr and UpdateMetadata messages". Is your thinking that we will
use
Thanks Colin, that helps. Can we add some of this to the KIP?
Ryanne
On Fri, Aug 2, 2019 at 12:23 PM Colin McCabe wrote:
> On Fri, Aug 2, 2019, at 07:50, Ryanne Dolan wrote:
> > Thanks Colin, interesting KIP.
> >
> > I'm concerned that the KIP does not actually address its stated
> > motivation
On Fri, Aug 2, 2019, at 07:50, Ryanne Dolan wrote:
> Thanks Colin, interesting KIP.
>
> I'm concerned that the KIP does not actually address its stated
> motivations. In particular, "Simpler Deployment and Configuration" are not
> really achieved, given that: 1) the proposal still requires quorums
Thanks Colin, interesting KIP.
I'm concerned that the KIP does not actually address its stated
motivations. In particular, "Simpler Deployment and Configuration" are not
really achieved, given that: 1) the proposal still requires quorums (now of
controllers, instead of ZK nodes), with the same res
Hi Harsha,
Thanks for the heads up. This should be fixed-- give it another try.
best,
Colin
On Thu, Aug 1, 2019, at 14:15, Harsha wrote:
> Hi Colin,
> Looks like KIP is missing the images , links are broken.
> Thanks,
> Harsha
>
> On Thu, Aug 1, 2019, at 2:05 PM, Colin McCabe wro
Hi Colin,
Looks like KIP is missing the images , links are broken.
Thanks,
Harsha
On Thu, Aug 1, 2019, at 2:05 PM, Colin McCabe wrote:
> Hi all,
>
> I've written a KIP about removing ZooKeeper from Kafka. Please take a
> look and let me know what you think:
>
> https://cwiki.apach
Hi all,
I've written a KIP about removing ZooKeeper from Kafka. Please take a look and
let me know what you think:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
cheers,
Colin
42 matches
Mail list logo
