Scenarios

Each scenario is one hand-built situation showing one thing — a clean election, two candidates deadlocking each other, a network cut in half. Open one and press play.

Each scenario is (config, seed, actions, flags) and one phenomenon it exists to show. They replay identically, and you can take control at any point.

  • A clean election

    Network: Lossy — Raft's normal operating condition

    clean-election

    Five nodes on a network that loses some messages. A leader is elected, then entries replicate and commit.

    Phenomenon

    A leader is elected from a standing start and client entries commit, under a network that loses messages — the ordinary case, which is worth seeing before anything breaks.

    nodes:
    5
    seed:
    4
    drop:
    8.0%
    actions:
    3
  • Split vote

    Network: Calm — nothing gets lost

    split-vote

    Near-uniform election timeouts make several nodes campaign at once. There is no tiebreak — the timeout jitter is what resolves it.

    Phenomenon

    With the randomized election timeout narrowed almost to nothing, candidates collide and split the vote. Raft has no tiebreak rule: the split is resolved only because the next round of timeouts differs. This is what the randomization is for.

    nodes:
    5
    seed:
    3
    drop:
    0.0%
    actions:
    1
  • A leader stranded in the minority

    Network: Calm — nothing gets lost

    partition-stranded-leader

    A leader is cut off on the minority side. It still believes it leads, but cannot commit anything; the majority side elects a new leader in a higher term.

    Phenomenon

    A leader partitioned into a minority still believes it leads, and still accepts client entries — but it can never commit them, because commitment needs a majority. The majority side elects a new leader in a higher term, and when the partition heals the stale leader learns of that term and steps down. Ablating the step-down rule leaves two leaders in one term.

    nodes:
    5
    seed:
    13
    drop:
    0.0%
    actions:
    5

    Protects: Election Safety · §5.1

  • Log divergence, and its repair

    Network: Calm — nothing gets lost

    log-divergence-repair

    Both sides of a partition accept different entries at the same index. After the heal, the new leader walks nextIndex back until the logs agree, then overwrites the divergent tail.

    Phenomenon

    The old leader, stranded with one follower, appends entries that can never commit. The majority elects a new leader that appends different entries at the same indices. When the partition heals, the AppendEntries consistency check fails, the leader walks nextIndex backwards until the logs agree, and overwrites the divergent tail. This is the signature view: rows that fail to line up, and then line up again.

    nodes:
    5
    seed:
    8
    drop:
    0.0%
    actions:
    8
  • A leader crashes mid-replication

    Network: Calm — nothing gets lost

    leader-crash-mid-replication

    A leader accepts an entry and crashes before a majority has stored it. The next leader decides that entry’s fate.

    Phenomenon

    A leader accepts an entry and crashes before a majority stores it. The entry is neither committed nor discarded: whether it survives depends entirely on whether the next leader happens to hold it, which the election restriction decides.

    nodes:
    5
    seed:
    4
    drop:
    0.0%
    actions:
    7
  • A candidate whose log has fallen behind

    Network: Calm — nothing gets lost

    election-restriction-overwrite

    One node is isolated while the cluster commits several entries. Its term climbs, its log does not. The election restriction is what keeps it from ever winning.

    Phenomenon

    An isolated node campaigns over and over, so its term climbs far above everyone else while its log stays short. When the partition heals its high term forces a new election — and the election restriction is the only thing stopping it from winning with a log that is missing committed entries. Turn the restriction off and those entries are lost.

    nodes:
    5
    seed:
    3
    drop:
    0.0%
    actions:
    6

    Protects: Leader Completeness · §5.4.1

  • Figure 8

    Network: Calm — nothing gets lost

    figure-8

    The scenario from the paper, played live. An entry from an older term is stored on a majority — and can still be overwritten.

    Phenomenon

    The paper's Figure 8, played out in the simulator. It opens at panel (a): S1 leads term 2 and has replicated index 2 to S2 only. S1 crashes, S5 wins term 3 and writes a different entry at index 2, S1 returns and wins term 4 and pushes its index 2 onto a majority — and that entry is still not safe. Turn off the current-term commit rule and it is declared committed, then overwritten. The exact panel-by-panel reproduction lives in tests/figure8.

    nodes:
    5
    seed:
    1
    drop:
    0.0%
    actions:
    20

    Protects: State Machine Safety · §5.4.2

  • Voting twice after a restart

    Network: Calm — nothing gets lost

    double-vote-restart

    A node casts a vote, crashes, and comes back. Because votedFor is persistent, it refuses to vote again in the same term.

    Phenomenon

    A follower votes, crashes and restarts. Because votedFor is persistent state, it remembers and refuses to vote a second time in the same term. Make votedFor volatile and the same node hands a second candidate the majority it needs, producing two leaders in one term.

    nodes:
    3
    seed:
    11
    drop:
    0.0%
    actions:
    5

    Protects: Election Safety · §5.2

  • Changing the cluster’s members

    Network: Calm — nothing gets lost

    membership-change

    The cluster {0,1,2} becomes {2,3,4}. The change passes through the joint configuration C-old,new; without it each side could elect its own leader in the same term.

    Phenomenon

    A cluster cannot switch directly from C-old to C-new, because there is no instant at which every server switches together — for a while some believe C-old and others C-new. Here {0,1} is a majority of {0,1,2} and {3,4} is a majority of {2,3,4}, and the two are disjoint: that is Figure 10. With joint consensus the transitional C-old,new demands majorities of *both* sets at once, so when the partition cuts the cluster neither side can win — it stalls, which is correct. Turn the rule off and both sides elect a leader in the same term.

    nodes:
    5
    seed:
    15
    drop:
    0.0%
    actions:
    5

    Protects: Election Safety · §6

  • A snapshot for a follower left behind

    Network: Calm — nothing gets lost

    log-compaction

    One follower is cut off while the cluster keeps committing. The leader has already discarded the entries it needs, so AppendEntries cannot catch it up — a snapshot is sent instead.

    Phenomenon

    Each server snapshots independently once it has applied enough entries, discarding what lies below — up to lastApplied and never further. When the partition heals, node 4 is so far behind that its nextIndex falls inside the range the leader has already discarded. AppendEntries cannot help it at that point: the entries it needs are gone. The leader sends InstallSnapshot instead, and node 4 replaces its state machine wholesale. In the ledger, rows below a snapshot point become hatched rather than blank, because those entries are not lost — they have been folded into the state.

    nodes:
    5
    seed:
    3
    drop:
    0.0%
    actions:
    9
  • The consistency check, switched off

    Network: Calm — nothing gets lost

    log-matching-break

    The leader probes a follower at an index that has already diverged. With the consistency check on it is rejected and repairs; with it off it is accepted, and two logs agree at one (index, term) on top of different prefixes.

    Phenomenon

    The AppendEntries consistency check only bites when nextIndex lands exactly on a divergent index. Here node 4 holds a term-3 entry at index 3 while the term-4 leader holds a term-2 entry there, and the leader's first probe falls on precisely that index. With the check on, node 4 rejects, the leader walks back, and the divergent tail is overwritten. With the check off, node 4 accepts, the leader believes their logs agree, and the next entry is appended on top of a different prefix — so both now hold (index 4, term 4) over contents that differ. Worse, the leader counts node 4 as a replica when committing.

    nodes:
    5
    seed:
    1
    drop:
    0.0%
    actions:
    2

    Protects: Log Matching · §5.3

  • Campaigning without incrementing the term

    Network: Calm — nothing gets lost

    double-candidacy

    Node 1 has already voted for node 0 in term 1. It campaigns again — with the term incremented that is a new ballot; without it, it overwrites its own vote in the same term.

    Phenomenon

    Incrementing the term on candidacy is what makes a campaign a new ballot. Node 0 leads term 1 on its own vote and node 1's; node 2 never received its RequestVote, so node 2's votedFor is still empty. Once node 0 is cut off, node 1 campaigns. With the rule on it moves to term 2, and the two leaders sit in different terms — not a violation. Without it, node 1 campaigns inside term 1, overwrites its own vote for node 0, and node 2 — which has not voted — hands it a second majority in the very same term.

    nodes:
    3
    seed:
    1
    drop:
    0.0%
    actions:
    2

    Protects: Election Safety · §5.2