How Zigcube is built

Store: a versioned key-value interface

The same semantics as etcd, on databases people already run. Every committed transaction gets one global revision, and the revision is assigned inside the transaction that writes the data.

• Operations: get, list (at a revision, paged, reversible), txn (compare on mod_revision, then puts and deletes), watch (ordered, gap-free, an explicit error once compacted), leases (grant, keepalive, revoke, list), and compact.
• Postgres: each writer's first statement is UPDATE store_meta SET revision = revision + 1 RETURNING revision. That serialises writers, so revision order is commit order and a rollback never burns a number. LISTEN/NOTIFY wakes watchers, a bounded poll backs it up, and the reader checks the log has no hole. The schema is plain heap; OrioleDB is optional and has not been run.
• SQLite: WAL mode, with BEGIN IMMEDIATE as the single writer. For development and single-node use.
• Memory: the reference implementation, used by the tests and the console.
• One conformance suite runs on all three backends with a fresh seed on every run.

store.zig · store_memory.zig · store_sqlite.zig · store_postgres.zig · store_conformance.zig · docs/store-design.md

The core: controllers as steps

A controller is reconcile(ctx) → Wakeup. It reads, does what is due, writes each action with its decision record in one transaction, and returns the time or key prefixes that should wake it next. It never blocks.

• Level-triggered and idempotent. A converged system writes nothing, so a lost event costs nothing.
• Partial progress is normal. A transaction touches at most four replicas. A failed compare writes nothing, and the next step plans again.
• No hidden state. Whatever cannot finish now, such as a stop's grace period or a held replica, is written to the store.
• WASM-clean. No threads, sockets, files, libc, std.Io, real clock or randomness. CI compiles it to wasm32-freestanding and runs it in Node.

core.zig · controller.zig · replica_controller.zig · agent.zig · apply.zig · why.zig · retention.zig · docs/core.md

zcube-server

The shell around the core: an accept loop, one task per connection, a driver thread that runs the controllers when their Wakeup falls due, timers for lease expiry and compaction, and a watchdog over the connection table.

• API (docs/api.md): reads under /v1; POST /v1/apply, with ?dry_run=1 to preview; DELETE /v1/services/{name}; agent endpoints under /v1/agent/nodes/{node}/.
• Every request has an ID, returned in X-Request-Id: the client's own if it sends one (the CLI does, with a random part), otherwise r-<start>-<counter>, unique across restarts.
• A failing store is reported once as a diagnostic, not once per step, with backoff.
• Writes are unauthenticated: listen on a private address.

server.zig · server_drive.zig · server_http.zig · server_listen.zig · api.zig · agent_api.zig

zcube-agent and FakeRuntime

One single-threaded agent per node. It reconciles the node's desired replicas against what the runtime lists, and reports what it observes. The Runtime interface is the contract with Zigbox: a small serialisable Handle, start, stop with grace, inspect, list, and an event stream. There is no blocking wait.

• One supervisor per container (the owner's recorded exception): zcube-agent supervise is the command's parent, runs in its own session, and records the exit, so the agent can restart without losing anything.
• Re-adoption is list: a restarted agent reads the state directory and never starts a duplicate.
• A container whose supervisor vanished is lost (ZC-E0509). Its process group is killed once.
• Images come from fixtures/images.zon. Any other digest fails with the placeholder ZBX-E0410. Limits are recorded but not enforced.

zcube_agent.zig · agent_run.zig · agent_remote.zig · runtime.zig · fake_runtime.zig · fake_supervisor.zig · docs/agent.md

zcube, the command line

• apply shows a diff and asks first. --yes skips the question, and --dry-run only shows the diff.
• get service|replicas, delete service, and explain <code> (works offline).
• why <kind> <name> follows decision records back to an observed fact or a desired key.
• bundle --last writes a redacted .tar.gz of evidence that never leaves the machine.
• Output is plain, --verbose or --output json. The exit code comes from the blame of the first diagnostic.

cli.zig · cli_*.zig · bundle*.zig · docs/cli.md

Diagnostics, shared with Zigbox

Every failure is a coded diagnostic with a blame, a cause and a fix, never a stack trace or a bare Zig error name. Zigcube's ZC- registry plugs into Zigbox's diagnostic, render and explain modules as a second product. Zigbox is a pinned package.

• Blame decides the exit code: 1 user, 2 environment, 3 external, 70 bug.
• Golden tests cover every code in all three output modes. A no-stack-trace test covers every failure path, and the redactor is fuzzed.
• An error page per code is generated from the registry and checked in CI.

diag.zig · src/diag/ · cli_golden.zig · errors_site.zig · DIAGNOSTICS.md

Web console

Plain HTML, CSS and ES modules, embedded in zcube-server. There is no build step and it makes no request to any other host. It lists once, then follows the store's watch, the same way an agent does, and shows the connection state at all times.

With ?sim=c0 it runs the real core, compiled to WASM, over a memory store and a simulated runtime in the browser. That module has exactly one import, env.zc_panic.

web/ · console_wasm.zig · docs/console.md

How it is checked

• demo-c0 is the milestone demo. Real server, agent and CLI processes, on Postgres and on SQLite, assert every step and acceptance criterion: kill, scale, an unknown image, server and agent restarts, the store going down, and a bundle.
• sim-c0 runs the same scenario natively and as WASM on virtual time and compares the reports. It is a simulation, not the demo.
• agent-smoke is a host check of the server and agent as real processes.
• Also: test, tidy, test-store on each backend, wasm-check, wasm-test and console-wasm. The WASM jobs run under each V8 tier separately, because Node 22 crashes when tiers mix.

What gets built

• Node registration on a lease. /nodes/<node> is bound to the agent's lease and carries its declared CPU and memory; while it exists, the agent holds tenure. zcube get nodes lists the nodes.
• A scheduler that filters on resource fit, then scores for spread, and writes the placement as /assigned/<node>/<replica> with its reasons in the decision. Agents watch only their own prefix.
• A node controller. On lease expiry it marks the node lost in /node-status/<node>, waits a short grace, and marks its replicas lost; the replica controller replaces them on the survivors, with new ids. Nothing moves back when the node returns.
• Fencing, with a stated window. Exclusive ownership of a replica is guaranteed by the store; exclusive execution only outside a stated window, and not on a wedged node. The agent stops its own workloads a margin before its lease can expire, timed from when it sent its last successful keepalive. Writes from an agent that lost tenure are refused by the store. demo-c1 tests a partition: the agent stays alive but cannot reach the server.
• Complete or truncated. zcube why says whether its chain reached a terminal, retention never evicts a live subject's newest decision of each kind or its chain, and decisions stay immutable: repetition is counted in a separate summary that is never a cause.
• demo-c1 (three agents, six replicas, one agent killed, zcube why node node3) and sim-c1, which checks the invariants under partitions, clock skew and kills that fail.

Decided before the brief

• One writer role per key prefix: the API, the replica controller, the scheduler, the node controller, each node's agent. The scheduler no longer writes into /replicas.
• The read-set rule: a controller's transaction compares every key whose value justified the action, so a stale plan writes nothing.
• Only the scheduler assigns and releases, and it updates the allocation record in the same transaction. A retired replica or a lost node is a fact the scheduler reacts to.
• The full placement compare set. A placement compares the replica, the absence of another assignment, the node's tenure and capacity, its readiness, its allocation record, and the absence of an open audit finding. A node lost between planning and commit gets nothing.
• No phantoms in capacity. A decision over a set compares one authoritative aggregate that every change to the set updates. For capacity that is /allocations/<node>, owned by the scheduler and changed in the same transaction as every placement and release, so no placement overcommits a node.
• Audited, never silently repaired. An auditor rebuilds each node's allocation from its assignments at one revision. A mismatch is recorded as a finding and reported as a bug, and the node gets no new placements. Repair is an operator's request, carried out by the scheduler as a decision of its own.
• Admissible observations: a controller acts on an observation only under the node's current tenure, the replica's current placement, and a ready node. Older ones may explain, never justify.
• Decisions are immutable and carry their evidence. A decision another decision cites never changes afterwards. Each one embeds what it read from the key it cites, so an explanation survives that key changing or being compacted. Repeated actions are counted in a separate summary.
• A lease expiry is not proof of a stop. Stateless replicas accept the stated window. A stateful replica (C7) moves only with a fencing proof: the node confirms, an external fence succeeds, or an operator asserts it, and durable writes check the placement epoch.
• Still to fix in the brief: what happens to a replica that fits no node (held as pending, with a decision naming the shortfall), the final fencing constants, a random part in server-assigned request IDs, and the watch fan-out benchmark.