How to read the picture

Each shape is one computer. The little boxes between them are messages in flight — messages take time to arrive, and some never arrive at all.

• Leader

  In charge. The only one that accepts new records and distributes them.

• Follower

  An ordinary member. Waits to hear from the leader and copies its records.

• Candidate

  Currently standing for election and asking the others for their votes.

• Down

  Currently down. Sends nothing and receives nothing.

Every computer’s list of records, side by side, aligned on row number. While the columns match, everyone agrees. A gap or a different value means their lists have diverged — and you can watch the leader repair it.

The messages in flight

• RVAsking for a vote — a candidate wants to be elected.
• RV✓Vote granted.
• RV✗Vote refused.
• HBA leader’s periodic “I am still in charge”.
• AE·nThe leader is sending n new records to be copied.
• AE✓Records accepted; the two lists agree.
• AE✗Rejected — the preceding record does not match. The leader backs up a row.
• ISA snapshot, sent because the receiver has fallen too far behind.