public abstract class KetchLeader
extends java.lang.Object
A leader instance starts up in
KetchLeader.State.CANDIDATE
and tries
to begin a new term by sending an
ElectionRound
to all replicas. Its
term starts if a majority of replicas have accepted this leader instance for
the term.
Once elected by a majority the instance enters
KetchLeader.State.LEADER
and runs
proposals offered to queueProposal(Proposal)
. This continues until
the leader is timed out for inactivity, or is deposed by a competing leader
gaining its own majority.
Once timed out or deposed this KetchLeader
instance should be
discarded, and a new instance takes over.
Each leader instance coordinates a group of
KetchReplica
s. Replica instances are
owned by the leader instance and must be discarded when the leader is
discarded.
In Ketch all push requests are issued through the leader. The steps are as
follows (see KetchPreReceive
for an
example):
Proposal
with the
ReceiveCommand
s that represent the push.
queueProposal(Proposal)
on the leader instance.
Proposal.await()
.
Proposal.getCommands()
, looking at
Command.getResult()
.
The leader gains consensus by first pushing the needed objects and a
RefTree
representing the
desired target repository state to the refs/txn/accepted
branch on
each of the replicas. Once a majority has succeeded, the leader commits the
state by either pushing the refs/txn/accepted
value to
refs/txn/committed
(for Ketch-aware replicas) or by pushing updates
to refs/heads/master
, etc. for stock Git replicas.
Internally, the actual transport to replicas is performed on background
threads via the KetchSystem
's
executor service. For performance, the
KetchLeader
,
KetchReplica
and
Proposal
objects share some state,
and may invoke each other's methods on different threads. This access is
protected by the leader's lock
object. Care must be taken to prevent
concurrent access by correctly obtaining the leader's lock.
Modifier and Type | Class and Description |
---|---|
static class |
KetchLeader.State
Current state of the leader instance.
|
Modifier and Type | Field and Description |
---|---|
private LogIndex |
committedIndex
Leader knows this (and all prior) states are committed.
|
private KetchReplica[] |
followers |
private LogIndex |
headIndex
End of the leader's log.
|
private boolean |
idle
Is the leader idle with no work pending? If
true there is no work
for the leader (normal state). |
(package private) java.util.concurrent.locks.Lock |
lock
Lock protecting all data within this leader instance.
|
private static org.slf4j.Logger |
log |
private java.util.List<Proposal> |
queued
Pending proposals accepted into the queue in FIFO order.
|
private RefTree |
refTree
State of the repository's RefTree after applying all entries in
queued . |
(package private) boolean |
roundHoldsReferenceToRefTree
If
true refTree must be duplicated before queuing the
next proposal. |
private Round |
runningRound
Current round the leader is preparing and waiting for a vote on.
|
private LocalReplica |
self |
private KetchLeader.State |
state |
private KetchSystem |
system |
private long |
term
Term of this leader, once elected.
|
private KetchReplica[] |
voters
Leader's knowledge of replicas for this repository.
|
Modifier | Constructor and Description |
---|---|
protected |
KetchLeader(KetchSystem system)
Construct a leader for a Ketch instance.
|
Modifier and Type | Method and Description |
---|---|
private void |
commitAsync(KetchReplica caller) |
private static LocalReplica |
findLocal(java.util.Collection<KetchReplica> voters) |
(package private) LogIndex |
getCommitted() |
(package private) LogIndex |
getHead() |
(package private) KetchSystem |
getSystem() |
(package private) long |
getTerm() |
private void |
initialize() |
(package private) boolean |
isIdle() |
private ProposalRound |
newProposalRound() |
(package private) void |
nextRound()
Schedule the next round; invoked while
lock is held. |
private void |
notifySuccess(Round round) |
(package private) void |
onReplicaUpdate(KetchReplica replica)
Asynchronous signal from a replica after completion.
|
protected abstract Repository |
openRepository()
Get an instance of the repository for use by a leader thread.
|
void |
queueProposal(Proposal proposal)
Queue a reference update proposal for consensus.
|
(package private) void |
runAsync(Round round) |
private void |
runLeader() |
private void |
scheduleLeader() |
void |
setReplicas(java.util.Collection<KetchReplica> replicas)
Configure the replicas used by this Ketch instance.
|
void |
shutdown()
Gracefully shutdown this leader and cancel outstanding operations.
|
LeaderSnapshot |
snapshot()
Snapshot this leader
|
java.lang.String |
toString() |
private static java.util.Collection<java.lang.Integer> |
validVoterCounts() |
private static final org.slf4j.Logger log
private final KetchSystem system
private KetchReplica[] voters
private KetchReplica[] followers
private LocalReplica self
final java.util.concurrent.locks.Lock lock
This lock extends into the KetchReplica
instances used by the
leader. They share the same lock instance to simplify concurrency.
private KetchLeader.State state
private long term
private final java.util.List<Proposal> queued
These proposals were preflighted and do not contain any conflicts with
each other and their expectations matched the leader's local view of the
agreed upon refs/txn/accepted
tree.
private RefTree refTree
queued
. New proposals must be consistent with this tree to be
appended to the end of queued
.
Must be deep-copied with RefTree.copy()
if
roundHoldsReferenceToRefTree
is true
.
volatile boolean roundHoldsReferenceToRefTree
true
refTree
must be duplicated before queuing the
next proposal. The refTree
was passed into the constructor of a
ProposalRound
, and that external reference to the RefTree
object is held by the proposal until it materializes the tree object in
the object store. This field is set true
when the proposal begins
execution and set false
once tree objects are persisted in the
local repository's object store or refTree
is replaced with a
copy to isolate it from any running rounds.
If proposals arrive less frequently than the RefTree
is written
out to the repository the roundHoldsReferenceToRefTree
behavior
avoids duplicating refTree
, reducing both time and memory used.
However if proposals arrive more frequently refTree
must be
duplicated to prevent newly queued proposals from corrupting the
runningRound
.
private LogIndex headIndex
private LogIndex committedIndex
private boolean idle
true
there is no work
for the leader (normal state). This field is false
when the
leader thread is scheduled for execution, or while runningRound
defines a round in progress.private Round runningRound
protected KetchLeader(KetchSystem system)
system
- Ketch system configuration the leader must adhere to.KetchSystem getSystem()
public void setReplicas(java.util.Collection<KetchReplica> replicas)
Replicas should be configured once at creation before any proposals are executed. Once elections happen, reconfiguration is a complicated concept that is not currently supported.
replicas
- members participating with the same repository.private static java.util.Collection<java.lang.Integer> validVoterCounts()
private static LocalReplica findLocal(java.util.Collection<KetchReplica> voters)
protected abstract Repository openRepository() throws java.io.IOException
The caller will close the repository.
java.io.IOException
- cannot reopen the repository for the leader.public void queueProposal(Proposal proposal) throws java.lang.InterruptedException, java.io.IOException
This method does not wait for consensus to be reached. The proposal is checked to look for risks of conflicts, and then submitted into the queue for distribution as soon as possible.
Callers must use Proposal.await()
to see if the proposal is done.
proposal
- the proposed reference updates to queue for consideration.
Once execution is complete the individual reference result
fields will be populated with the outcome.java.lang.InterruptedException
- current thread was interrupted. The proposal may have been
aborted if it was not yet queued for execution.java.io.IOException
- unrecoverable error preventing proposals from being attempted
by this leader.private void initialize() throws java.io.IOException
java.io.IOException
private void scheduleLeader()
private void runLeader()
private ProposalRound newProposalRound()
long getTerm()
LogIndex getHead()
LogIndex getCommitted()
boolean isIdle()
void runAsync(Round round)
void onReplicaUpdate(KetchReplica replica)
Must be called while lock
is held by the replica.
replica
- replica posting a completion event.private void notifySuccess(Round round)
private void commitAsync(KetchReplica caller)
void nextRound()
lock
is held.public LeaderSnapshot snapshot()
public void shutdown()
public java.lang.String toString()
toString
in class java.lang.Object