Hi,
I see that Kafka uses zookeeper for group membership and leader election.
Curious to know if Kafka developers ever discussed to move away from
Zookeeper and use Gossip based protocols. Is there any specific advantage
of using Zookeeper over Gossip based implementation or vice versa?
Thanks,
U
I see that, when a new topic is created, two metadata records, a
TopicRecord (just the name and id of the topic) and a PartitionRecord (more
like LeaderAndIsr, with leader id and replica ids for the partition) are
created.
While creating the topic, log entries for both the records need to be
commit
eTopic ||
||
||
On Fri, Jul 10, 2020 at 1:30 AM Colin McCabe wrote:
> On Thu, Jul 9, 2020, at 04:37, Unmesh Joshi wrote:
> > I see that, when a new topic is created, two metadata records, a
> > TopicRecord (just the name and id of the topic)
> Hi Unmesh,
>
> Yes, once the last stable offset advanced, we would consider the topic
> creation to be done, and then we could return success to the client.
>
> best,
> Colin
>
> On Thu, Jul 9, 2020, at 19:44, Unmesh Joshi wrote:
> > It still needs HighWaterMark / LastS
Hi,
In the FetchRequest Handling, how to make sure we handle scenarios where
the leader might have been disconnected from the cluster, but doesn't know
yet?
As discussed in the Raft Thesis section 6.4, the linearizable semantics of
read requests is implemented in LogCabin by sending heartbeat to f
mesh
On Mon, Jul 27, 2020 at 6:23 AM Boyang Chen
wrote:
> Thanks for the questions Unmesh!
>
> On Sun, Jul 26, 2020 at 6:18 AM Unmesh Joshi
> wrote:
>
> > Hi,
> >
> > In the FetchRequest Handling, how to make sure we handle scenarios where
> > the leader migh
, that way, telling
client in time that it was disconnected from the quorum, and not keep on
sending state metadata to clients.
Thanks,
Unmesh
On Mon, Jul 27, 2020 at 9:31 AM Unmesh Joshi wrote:
> >>Could you clarify on this question? Which part of the raft group doesn't
> >&g
a stop-the-world approach to
> > > > > snapshotting, which is why we need the controller micromanaging
> > > snapshots
> > > > > on all followers. Alternatives include fuzzy snapshots which can be
> > > done
> > > > > concurrently. I
e advantage of creating a separate `controller.id`
> > > instead of
> > > > > just using `broker.id`?
> > > > > 3. It sounds like you are imagining a stop-the-world approach to
> > > > > snapshotting, which is why we need the controller micromanaging
>
Hi,
The LeaseStartTimeMs is expected to be the broker's
'System.currentTimeMillis()' at the point of the request. The active
controller will add its lease period to this in order to compute
the LeaseEndTimeMs.
I think this is a bit confusing. Monotonic clock on the active controller
shoul
simplekafka/kip500/Kip631ControllerTest.scala
I hope it will be helpful in driving some discussions and doing quick
experimentation outside of the Kafka codebase.
Thanks,
Unmesh
On Tue, Aug 4, 2020 at 7:52 AM Unmesh Joshi wrote:
> Hi,
>
> >>>>The LeaseStartTimeM
>>Hi Unmesh,
>>Thanks, I'll take a look.
Thanks. I will be adding more to the prototype and will be happy to help
and collaborate.
Thanks,
Unmesh
On Tue, Aug 11, 2020 at 12:28 AM Colin McCabe wrote:
> Hi Jose,
>
> That'a s good point that I hadn't considered. It's probably worth having
> a sep
McCabe wrote:
> Hi all,
>
> I'm thinking of calling a vote on KIP-631 on Monday. Let me know if
> there's any more comments I should address before I start the vote.
>
> cheers,
> Colin
>
> On Tue, Aug 11, 2020, at 05:39, Unmesh Joshi wrote:
> > >>Hi U
tml
)
Thanks,
Unmesh
On Sat, Aug 29, 2020 at 10:49 AM Colin McCabe wrote:
> On Fri, Aug 28, 2020, at 19:36, Unmesh Joshi wrote:
> > Hi Colin,
> >
> > There were a few of questions I had..
>
> Hi Unmesh,
>
> Thanks for the response.
>
> >
> >
troller. I
> think
> > > we should do that in a follow-on KIP, though, since this one is already
> > > pretty big.
> > >
> > > >
> > > > 4. Because we expose Raft log to all the brokers, any de-duplication
> of
> > > the
> > > > br
e good to add some details on how
leases are maintained in case of active controller (raft leader) failure.
e.g. Attached a few diagrams considering leases are maintained with log
entries.
On Mon, Aug 31, 2020 at 6:28 PM Unmesh Joshi wrote:
> >>The reason for including LeaseStartTimeMs
wrote:
> > Colin wrote:
> > > The reason for including LeaseStartTimeMs in the request is to ensure
> > > that the time required to communicate with the controller gets
> included in
> > > the lease time. Since requests can potentially be delayed in the
> network
&g
;
> >
> > Thanks,
> > Unmesh
> >
> > On Sat, Sep 5, 2020 at 1:28 AM Colin McCabe wrote:
> >
> > > > Colin wrote:
> > > > > The reason for including LeaseStartTimeMs in the request is to
> ensure
> > > > > that th
Thanks for the KIP.
+1 (non-binding)
On Tue, Sep 15, 2020 at 12:23 AM Colin McCabe wrote:
> Hi all,
>
> I'd like to call a vote on KIP-631: the quorum-based Kafka Controller.
> The KIP is here:
>
> https://cwiki.apache.org/confluence/x/4RV4CQ
>
> The DISCUSS thread is here:
>
>
> https://lists.
Hi,
I came across this issue which has discussion about capping
maxParallelForks while running tests.
https://issues.apache.org/jira/browse/KAFKA-2613
Is this still the case?
Thanks,
Unmesh
elForks` can be set via gradle argument:
> https://github.com/apache/kafka/blob/trunk/build.gradle#L78
>
> And in Jenkins, it looks like we default to 2.
> https://github.com/apache/kafka/blob/trunk/Jenkinsfile#L40
>
> Thank you.
> Luke
>
> On Fri, Apr 15, 2022 at 1:24 AM U
Hi Boyang,
Thanks for the KIP..
I think there are two aspects of linearizable read implementations in Raft.
1. Providing linearizable reads from the leader
Even read requests from the leader might not return the latest committed
results if the leader is partitioned. The leader needs to make su
Went through the changes since the last discussion thread, and it's looking
in good shape. Thanks!.
+ 1 (non-binding)
On Wed, Dec 16, 2020 at 4:34 PM Tom Bentley wrote:
> Thanks for the KIP Colin, it does a great job of clearly explaining some
> pretty complex changes.
>
> +1 (non-binding)
>
> T
23 matches
Mail list logo
