Thanks for participating in the discussion, Yang & Chesnay. LeaderElection
interface extension gave me a headache as well. I added it initially
because I thought it would be of more value. But essentially, it doesn't
help but make the code harder to understand (as your questions rightfully
point out). I agree that the FLIP is good enough without this extension. I
moved it into the Rejected Alternatives section of the FLIP and would
propose going ahead without it.
I will answer your questions about the LeaderElection extension, anyway:
BTW, if the *LeaderElectionService#register(return LeaderElection)* and
*LeaderElectionService#onGrantLeadership* are guarded by a same lock, then
we could ensure that the leaderElection in *LeaderContender* is always
non-null when it tries to confirm the leadership. And then we do not need
the
*LeaderContender#initializeLeaderElection*. Right?
No, we still would need LeaderContender#initializeLeaderElection because
the LeaderElectionService needs to be capable of setting the LeaderElection
within the LeaderContender before triggering the process for granting the
leadership. This all needs to happen within the
LeaderElectionService#register(LeaderContender). It's indepent of the lock.
With the extension, how does the leader contender get access to the
LeaderElection? I would've assumed that LEService returns a LeaderElection
when register is called, but according to the diagram this method doesn't
return anything. Is that what initiateLeaderElection is doing?
Correct. My initial plan was to make
LeaderElectionService#register(LeaderContender) return the LeaderElection
instance. That method could have been called within the LeaderContender.
But this approach has the flaw that LeaderContender would be in charge
within this control flow where, actually, we would want
LeaderElectionService to be still in charge to trigger the process for
granting the leadership. This required the
LeaderContender.initializeLeaderElection(LeaderElection) method to be added
to enable the LeaderElectionService to do the initialization. I added a
comment to the corresponding class diagram to make this clearer.
The DefaultLeaderElection will rely on package-private methods of the
DLEService to handle confirm/hasLeadership calls?
Correct. I added the missing package-private methods to the class diagram
in the FLIP to clear things up.
On Wed, Jan 18, 2023 at 11:47 AM Chesnay Schepler <ches...@apache.org>
wrote:
There are a lot of good things in this, and until the Extension bit I'm
fully on board.
With the extension, how does the leader contender get access to the
LeaderElection? I would've assumed that LEService returns a
LeaderElection when register is called, but according to the diagram
this method doesn't return anything. Is that what initiateLeaderElection
is doing?
The DefaultLeaderElection will rely on package-private methods of the
DLEService to handle confirm/hasLeadership calls?
I don't fully understand why LContender#initializeLeaderElection is
required.
On 05/01/2023 14:49, Matthias Pohl wrote:
Hi everyone,
I brought up FLINK-26522 [1] in the mailing list discussion about
consolidating the HighAvailabilityServices interfaces [2], previously.
There, it was concluded that the community still wants the ability to
have
per-component leader election and, therefore, keep the
HighAvailabilityServices interface as is. I went back to work on
FLINK-26522 [1] to figure out how we can simplify the current codebase
keeping the decision in mind.
I wanted to handle FLINK-26522 [1] as a follow-up cleanup task of
FLINK-24038 [3]. But while working on it, I realized that even
FLINK-24038
[3] shouldn't have been handled without a FLIP. The per-process leader
election which was introduced in FLINK-24038 [3] changed the ownership
of
certain components. This is actually a change that should have been
discussed in the mailing list and deserved a FLIP. To overcome this
shortcoming of FLINK-24038 [3], I decided to prepare FLIP-285 [4] to
provide proper documentation of what happened in FLINK-24038 and what
will
be manifested with resolving its follow-up FLINK-26522 [1].
Conceptually, this FLIP proposes moving away from Flink's support for
single-contender LeaderElectionServices and introducing multi-contender
support by disconnecting the HA-backend leader election lifecycle from
the
LeaderContender's lifecycle. This allows us to provide LeaderElection
per
component (as it was requested in [2]) but also enables us to utilize a
single leader election for multiple components/contenders as well
without
the complexity of the code that was introduced by FLINK-24038 [3].
I'm looking forward to your comments.
Matthias
[1] https://issues.apache.org/jira/browse/FLINK-26522
[2] https://lists.apache.org/thread/9oy2ml9s3j1v6r77h31sndhc3gw57cfm
[3] https://issues.apache.org/jira/browse/FLINK-24038
[4]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-285%3A+Refactoring+LeaderElection+to+make+Flink+support+multi-component+leader+election+out-of-the-box
