Deploy

Cluster operations

Nexir uses explicit Raft membership commands. Mutating operations must run on the current leader and require server.enable_admin_commands=true.

Initialize a cluster

INIT node_id grpc_addr redis_addr

Run INIT exactly once on one fresh node. It creates a one-voter cluster, mints a durable cluster UUID, and requires the argument node ID to match the configured node ID.

Add and promote a node

CLUSTER ADD 2 http://10.0.0.12:50051 10.0.0.12:6379 TIMEOUT 300000
CLUSTER PROMOTE 2

ADD performs candidate identity and address checks, adds a learner, and waits for catch-up. If catch-up times out, the learner stays in membership; inspect it, then promote later or remove it. Promote only after CLUSTER MEMBERS shows it caught up.

View cluster state

CLUSTER INFO
CLUSTER MEMBERS
CLUSTER IDENTITY
CLUSTER STATUS

• INFO reports state, role, leader, UUID, term, indexes, voters, and joint membership.
• MEMBERS reports IDs, roles, advertised addresses, leader, matched indexes, and lag.
• IDENTITY reports the local configured ID, cluster UUID, and initialized flag.
• STATUS returns serialized Raft metrics for diagnostics.

Change a node role

CLUSTER PROMOTE node_id
CLUSTER DEMOTE node_id

DEMOTE keeps the node as a learner. Nexir refuses to demote the current leader or last voter. Transfer leadership first when necessary.

Remove a node

CLUSTER REMOVE node_id

Removal completely removes a learner or voter. It does not stop or fence the process. After success, stop the removed node and archive or destroy its old path. A process left running can still serve stale LOCAL reads.

Transfer leadership

CLUSTER TRANSFER-LEADER 2
# or select the most caught-up replacement automatically
CLUSTER STEPDOWN

The target must be a caught-up voter. During the election window, clients may receive MOVED or temporary retryable errors. Use planned transfer before leader maintenance or removal.

Update advertised addresses

1. Stop the node.
2. Restart with the same node ID and storage path, using the new bind addresses.
3. Run CLUSTER UPDATE-ADDR node_id grpc_addr redis_addr through the leader.
4. Verify CLUSTER MEMBERS and CLUSTER INFO.

Nexir verifies that the process at the new peer address identifies as the same node in the same cluster.

Failure and quorum

A three-voter cluster can lose one voter and retain a majority. Loss of a majority makes writes and linearizable reads unavailable. Restore quorum if possible; online membership operations cannot commit without it.

Permanent majority loss

Nexir provides an offline, destructive-boundary recovery path that converts one chosen survivor into a new one-voter cluster identity:

nexir --accept-terms recover inspect --storage-path /var/lib/nexir-node1
nexir --accept-terms recover force-new-cluster --storage-path /var/lib/nexir-node1
nexir --accept-terms recover force-new-cluster --commit --confirm-node-id 1 \
  --storage-path /var/lib/nexir-node1

Stop and isolate every old node first. Inspect every reachable survivor and choose the highest last_log_id index, breaking ties with last_applied. Dry run before committing. The recovered cluster keeps everything present in the chosen survivor's log and applied state; acknowledged writes that existed only on dead nodes are irrecoverably lost. Re-add only wiped old nodes.

Client behavior during failover

Expect MOVED 0 <leader>, CLUSTERDOWN no known leader, or quorum errors. Every supported write touches one key and is idempotent when replayed with the same arguments, so a retry after an uncertain response is safe — except that a retried SET ... EX or EXPIRE restarts its deadline. Read back PTTL when the exact deadline matters.