public abstract class KetchReplica
extends java.lang.Object
LocalReplica
or RemoteGitReplica
.
Replicas can be either a stock Git replica, or a Ketch-aware replica.
A stock Git replica has no special knowledge of Ketch and simply stores
objects and references. Ketch communicates with the stock Git replica using
the Git push wire protocol. The
KetchLeader
commits an agreed upon
state by pushing all references to the Git replica, for example
"refs/heads/master"
is pushed during commit. Stock Git replicas use
KetchReplica.CommitMethod.ALL_REFS
to
record the final state.
Ketch-aware replicas understand the RefTree
sent during the proposal
and during commit are able to update their own reference space to match the
state represented by the RefTree
. Ketch-aware replicas typically use
a RefTreeDatabase
and
KetchReplica.CommitMethod.TXN_COMMITTED
to record the final state.
KetchReplica instances are tightly coupled with a single
KetchLeader
. Some state may be
accessed by the leader thread and uses the leader's own
KetchLeader.lock
to protect shared
data.
Modifier and Type | Class and Description |
---|---|
static class |
KetchReplica.CommitMethod
How this replica wants to receive Ketch commit operations.
|
static class |
KetchReplica.CommitSpeed
Delay before committing to a replica.
|
static class |
KetchReplica.Participation
Participation of a replica in establishing consensus.
|
static class |
KetchReplica.State
Current state of a replica.
|
(package private) static class |
KetchReplica.WeakRetryPush
Weakly holds a retrying replica, allowing it to garbage collect.
|
Modifier and Type | Field and Description |
---|---|
private KetchReplica.CommitMethod |
commitMethod |
private KetchReplica.CommitSpeed |
commitSpeed |
private java.lang.String |
error |
private long |
lastRetryMillis |
private KetchLeader |
leader |
(package private) static org.slf4j.Logger |
log |
private long |
maxRetryMillis |
private long |
minRetryMillis |
private KetchReplica.Participation |
participation |
private static byte[] |
PEEL |
private java.util.List<ReplicaPushRequest> |
queued |
private java.lang.String |
replicaName |
private long |
retryAtMillis |
private java.util.concurrent.Future<?> |
retryFuture
Scheduled retry due to communication failure.
|
private java.util.Map<java.lang.String,ReceiveCommand> |
running |
private java.util.Map<ObjectId,java.util.List<ReceiveCommand>> |
staged |
private KetchReplica.State |
state
What is happening with this replica.
|
private ObjectId |
txnAccepted
Value known for
"refs/txn/accepted" . |
private ObjectId |
txnCommitted
Value known for
"refs/txn/committed" . |
private java.util.Map<java.lang.String,ReceiveCommand> |
waiting |
Modifier | Constructor and Description |
---|---|
protected |
KetchReplica(KetchLeader leader,
java.lang.String name,
ReplicaConfig cfg)
Configure a replica representation.
|
Modifier and Type | Method and Description |
---|---|
(package private) void |
afterPush(Repository repo,
ReplicaPushRequest req)
Callback from
ReplicaPushRequest upon success or failure. |
protected abstract void |
blockingFetch(Repository repo,
ReplicaFetchRequest req)
Fetch objects from the remote using the calling thread.
|
(package private) boolean |
canDelete(Ref ref) |
private static ReceiveCommand |
copy(ReceiveCommand c) |
private boolean |
defer(ReplicaPushRequest req) |
private static void |
delete(java.util.List<ReceiveCommand> cmds,
java.util.List<ReceiveCommand> createCmds) |
protected java.lang.String |
describeForLog()
Get description of this replica for error/debug logging purposes.
|
private void |
doRetryPush() |
private static boolean |
equals(ObjectId a,
LogIndex b) |
KetchReplica.CommitMethod |
getCommitMethod()
Get how Ketch will commit to the repository.
|
KetchReplica.CommitSpeed |
getCommitSpeed()
Get when Ketch will commit to the repository.
|
(package private) static ObjectId |
getId(Ref ref) |
KetchLeader |
getLeader()
Get leader instance this replica follows.
|
java.lang.String |
getName()
Get unique-ish name for debugging.
|
KetchReplica.Participation |
getParticipation()
Get how the replica participates in this Ketch system.
|
KetchSystem |
getSystem()
Get system configuration.
|
(package private) ObjectId |
getTxnAccepted() |
(package private) boolean |
hasAccepted(LogIndex id) |
(package private) void |
initialize(java.util.Map<java.lang.String,Ref> refs)
Update the leader's view of the replica after a poll.
|
protected java.util.Collection<ReceiveCommand> |
prepareCommit(Repository git,
java.util.Map<java.lang.String,Ref> current,
ObjectId committed)
Build a list of commands to commit
KetchReplica.CommitMethod.ALL_REFS . |
private void |
prepareTxnCommitted(java.util.List<ReceiveCommand> cmds,
ObjectId committed) |
private void |
pushAsync(ReplicaPushRequest req) |
(package private) void |
pushCommitAsync(LogIndex committed) |
(package private) void |
pushTxnAcceptedAsync(Round round)
Schedule a proposal round with the replica.
|
private static ObjectId |
readId(ReplicaPushRequest req,
ReceiveCommand cmd) |
private void |
removeStaged(java.util.List<ReceiveCommand> cmds,
ObjectId committed) |
private void |
retryLater(ReplicaPushRequest req) |
private void |
runNextPushRequest()
Determine the next push for this replica (if any) and start it.
|
(package private) boolean |
shouldPushUnbatchedCommit(LogIndex committed,
boolean leaderIdle) |
protected void |
shutdown()
Called by leader to perform graceful shutdown.
|
(package private) ReplicaSnapshot |
snapshot() |
protected abstract void |
startPush(ReplicaPushRequest req)
Begin executing a single push.
|
private void |
updateView(ReplicaPushRequest req,
ObjectId acceptId,
ReceiveCommand commitCmd) |
private boolean |
waitingForRetry() |
static final org.slf4j.Logger log
private static final byte[] PEEL
private final KetchLeader leader
private final java.lang.String replicaName
private final KetchReplica.Participation participation
private final KetchReplica.CommitMethod commitMethod
private final KetchReplica.CommitSpeed commitSpeed
private final long minRetryMillis
private final long maxRetryMillis
private final java.util.Map<ObjectId,java.util.List<ReceiveCommand>> staged
private final java.util.Map<java.lang.String,ReceiveCommand> running
private final java.util.Map<java.lang.String,ReceiveCommand> waiting
private final java.util.List<ReplicaPushRequest> queued
private ObjectId txnAccepted
"refs/txn/accepted"
.
Raft literature refers to this as matchIndex
.
private ObjectId txnCommitted
"refs/txn/committed"
.
Raft literature refers to this as commitIndex
. In traditional
Raft this is a state variable inside the follower implementation, but
Ketch keeps it in the leader.
private KetchReplica.State state
private java.lang.String error
private java.util.concurrent.Future<?> retryFuture
private long lastRetryMillis
private long retryAtMillis
protected KetchReplica(KetchLeader leader, java.lang.String name, ReplicaConfig cfg)
leader
- instance this replica follows.name
- unique-ish name identifying this replica for debugging.cfg
- how Ketch should treat the replica.public KetchSystem getSystem()
public KetchLeader getLeader()
public java.lang.String getName()
protected java.lang.String describeForLog()
public KetchReplica.Participation getParticipation()
public KetchReplica.CommitMethod getCommitMethod()
public KetchReplica.CommitSpeed getCommitSpeed()
protected void shutdown()
Default implementation cancels any scheduled retry. Subclasses may add
additional logic before or after calling super.shutdown()
.
Called with KetchLeader.lock
held
by caller.
ReplicaSnapshot snapshot()
void initialize(java.util.Map<java.lang.String,Ref> refs)
Called with KetchLeader.lock
held by caller.
refs
- map of refs from the replica.ObjectId getTxnAccepted()
boolean hasAccepted(LogIndex id)
void pushTxnAcceptedAsync(Round round)
Called with KetchLeader.lock
held by caller.
round
- current round being run by the leader.private static ReceiveCommand copy(ReceiveCommand c)
boolean shouldPushUnbatchedCommit(LogIndex committed, boolean leaderIdle)
void pushCommitAsync(LogIndex committed)
private void prepareTxnCommitted(java.util.List<ReceiveCommand> cmds, ObjectId committed)
private void removeStaged(java.util.List<ReceiveCommand> cmds, ObjectId committed)
private static void delete(java.util.List<ReceiveCommand> cmds, java.util.List<ReceiveCommand> createCmds)
private void runNextPushRequest()
If the replica has successfully accepted the committed state of the
leader, this method will push all references to the replica using the
configured KetchReplica.CommitMethod
.
If the replica is KetchReplica.State.LAGGING
this method will begin catch up
by sending a more recent refs/txn/accepted
.
Must be invoked with KetchLeader.lock
held by caller.
private void pushAsync(ReplicaPushRequest req)
private boolean defer(ReplicaPushRequest req)
private boolean waitingForRetry()
private void retryLater(ReplicaPushRequest req)
private void doRetryPush()
protected abstract void startPush(ReplicaPushRequest req)
This method must move processing onto another thread. Called with
KetchLeader.lock
held by caller.
req
- the request to send to the replica.void afterPush(@Nullable Repository repo, ReplicaPushRequest req)
ReplicaPushRequest
upon success or failure.
Acquires the KetchLeader.lock
and updates the leader's internal
knowledge about this replica to reflect what has been learned during a
push to the replica. In some cases of divergence this method may take
some time to determine how the replica has diverged; to reduce contention
this is evaluated before acquiring the leader lock.
repo
- local repository instance used by the push thread.req
- push request just attempted.private void updateView(ReplicaPushRequest req, @Nullable ObjectId acceptId, ReceiveCommand commitCmd)
@Nullable private static ObjectId readId(ReplicaPushRequest req, @Nullable ReceiveCommand cmd)
protected abstract void blockingFetch(Repository repo, ReplicaFetchRequest req) throws java.io.IOException
Called without KetchLeader.lock
.
repo
- local repository to fetch objects into.req
- the request to fetch from a replica.java.io.IOException
- communication with the replica was not possible.protected java.util.Collection<ReceiveCommand> prepareCommit(Repository git, java.util.Map<java.lang.String,Ref> current, ObjectId committed) throws java.io.IOException
KetchReplica.CommitMethod.ALL_REFS
.git
- local leader repository to read committed state from.current
- all known references in the replica's repository. Typically
this comes from a push advertisement.committed
- state being pushed to refs/txn/committed
.java.io.IOException
- cannot read the committed state.boolean canDelete(Ref ref)