Thanks, Jason. Good feedback
1. I was mostly referring to the fact that the ReplicaManager uses a
background thread to send the ZK notification and it really has no
visibility as to whether the ZK operation succeeded or not. We'll most
likely want to continue using a background thread for batching
Hi David,
This looks good. I just have a few comments:
1. I'm not sure it's totally fair to describe the current notification
mechanism as "best-effort." At least it guarantees that the controller will
eventually see the event. In any case, I think we might want a stronger
contract going forward.
I've updated the KIP with the feedback from this discussion
https://cwiki.apache.org/confluence/display/KAFKA/KIP-589+Add+API+to+update+Replica+state+in+Controller.
I'll send out the vote thread shortly.
Thanks again,
David
On Tue, May 5, 2020 at 10:34 AM Tom Bentley wrote:
> Hi Colin,
>
> Yeah
Hi Colin,
Yeah, that makes sense, thanks. I was thinking, longer term, that there are
other benefits to having the log dir information available to the
controller. For example it would allow the possibility for CREATE_TOPIC
requests to include the intended log dir for each replica. But that's
obvi
Hi Tom,
As you said, the controller doesn't know about log directories, although
individual brokers do. So the brokers do currently have to enumerate all the
partitions that need to be removed to the controllers explicitly. So this KIP
isn't changing anything in that regard.
The current flow
Hi David,
> In the rejecting the alternative of having an RPC for log dir failures
> you say
>
> I guess what I really mean here is that I wanted to avoid exposing the
> notion of a log dir to the controller. I can update the description to
> reflect this.
>
Ah, I think I see now. While each brok
Jose/Colin/Tom, thanks for the feedback!
> Partition level errors
This was an oversight on my part, I meant to include these in the response
RPC. I'll update that.
> INVALID_REQUEST
I'll update this text description, that was a copy/paste left over
> I think we should mention that the controll
Hi David,
Thanks for the KIP!
In the rejecting the alternative of having an RPC for log dir failures you
say:
It was also rejected to prevent "leaking" the notion of a log dir to the
> public API.
>
I'm not quite sure I follow that argument, since we already have RPCs for
changing replica log d
Hi David,
Thanks for the KIP!
I think the ReplicaStateEventResponse should have a separate error code for
each partition.
Currently it just has one error code for the whole request/response, if I'm
reading this right. I think Jose made a similar point as well. We should plan
for scenarios
Hi David,
Thanks for the KIP.
> ReplicaStateEventResponse => ErrorCode [Topic [PartitionId]]
>ErrorCode => Int32
>Topic => String
>PartitionId => Int32
> ...
> Partition-level errors:
Based on my understanding of the response, it doesn't look like the
controller has a way of encoding
Hey everyone,
I'd like to start the discussion for KIP-589, part of the KIP-500 effort
https://cwiki.apache.org/confluence/display/KAFKA/KIP-589+Add+API+to+update+Replica+state+in+Controller
This is a proposal to use a new RPC instead of ZooKeeper for notifying the
controller of an offline repli
11 matches
Mail list logo
