Diagnostic code

ZC-E0215

the store went backwards or forked

Kind

error

Blame

external

Exit code

3

What it means

The server remembers the highest revision it has seen and the chain hash of that revision. The store now answers with a lower revision, or with a different hash for the revision the server remembers, so writes the store acknowledged are gone or the history has diverged. This is what a failover to a replica that was behind, a restore from an old backup, or a store pointed at by mistake looks like. Acting on it would hand out placements and fences that contradict ones already given.

The server stops: controllers do not step and every store call fails with this code until an operator looks. It never recovers by itself. Look at the store (which database, which replica, which backup). If the rolled-back or forked store is the one to keep, run zcube store accept-rollback --confirm, which makes the server remember the store as it is now. The old and new revision and hash are shown in the reply of that command; a decision record that keeps them is not written yet.

An agent raises it too: it remembers the highest revision and hash the server's answers carried, and refuses an answer from behind them. After accept-rollback on the server, remove the file remembered in the agent's state directory and restart the agent.

Offline

The same text, without a network:

zcube explain E0215