Hi Calvin,
Regarding partition reassignment, I have two comments.
I notice the KIP says "The AlterPartitionReassignments will not change the ELR"
however, when a reassignment completes (or reverts) any replicas removed from
the replica set would be removed from the ELR. Sounds obvious but I fig
Hi all,
As part of my work on formally verifying different parts of Apache Kafka
and working on KIP-966 I have built up a lot of knowledge about how the
replication protocol works. Currently it is mostly documented across
various KIPs and in the code itself. I have written a complete protocol
desc
ld be a great contribution to the project. Having it there would
> allow
> > > the community to maintain it while changes to the protocol are made.
> That
> > > would also pave the way for having other specs in the future (e.g. new
> > > consumer group protocol
I would like to see more explicit discussion of topic retention and share
groups. There are a few options here from simple to more sophisticated. There
are also topic-level and share-group level options.
The simple thing would be to ensure that the SPSO of each share group is
bounded by the Log
I think it is a great technique and I've used local invariants when doing
system modelling in Jepsen Maelstrom which has no global view of state for
checking global invariants. Sometimes the kind of assertions you want could
be too costly for inclusion in a production system so the idea of gating
t
If you want to understand how the replication protocol works, how it can be
configured for consistency, how it can be configured for availability then
I have written up a more formal description of the protocol and written
TLA+ specifications. These should answer most of your questions and if not,
Hi Jose,
I have a question about how voters and observers, which are far behind the
leader, catch-up when there are multiple reconfiguration commands in the
log between their position and the end of the log.
Here are some example situations that need clarification:
Example 1
Imagine a cluster of
The current configs are hard to use for the Kafka user and a little inflexible
so I am pleased to see the discussion.
Ultimately we want flexibility. We don't want to force users to understand the
underlying implementation/protocol and we want the producer to handle high or
low throughput effic
Hi Jose,
Thanks for getting this started. Some comments:
- Regarding the removal of voters, when a leader appends a
RemoveVoterRecord to its log, it immediately switches to the new
configuration. There are two cases here:
1. The voter being removed is the leader itself. The KIP documents that
+1 for me too. Once the KIP-853 is agreed I will make any necessary changes
and submit a PR to the apache/kafka repo.
Jack
On Tue, Jul 26, 2022 at 10:10 PM Ismael Juma wrote:
> I'm +1 for inclusion in the main repo and I was going to suggest the same
> in the KIP-853 discussion. The original au
Hi Jose,
It's looking good!
> I think that when a replica has caught up is an implementation detail
> and we can have this detailed discussion in Jira or the PR. What do
> you think?
Yes, that sounds fine. For what it's worth, making the leader take the
decision of when an observer is caught-up
Jack Vanlightly created KAFKA-16281:
---
Summary: Probable IllegalState possible with KIP-966
Key: KAFKA-16281
URL: https://issues.apache.org/jira/browse/KAFKA-16281
Project: Kafka
Issue Type
12 matches
Mail list logo
