Add a Second Node for Redundancy Without Double-Signing

Add a Second Node for Redundancy Without Double-Signing

Every validator operator eventually faces the same question: what happens when my one server goes down? The obvious answer, run a second node, is also the most dangerous answer if it is done naively. Two validator clients with the same keys, both signing, is exactly what slashing punishes. Redundancy done right is not two validators. It is one validator, with a tested way to move its signing job to a standby machine, and a hard guarantee that both machines never sign at the same time.

The distinction to hold in your head is between duplicating data and duplicating signing. Execution and consensus nodes, the parts that sync chain data and track the head of the chain, are safe to duplicate. You can and should run more than one of them, in different places, so a single failure never blinds you. The validator client, the part that holds the keys and signs attestations and blocks, must have exactly one active instance at any moment. Everything in this post follows from that one rule.

The active-passive pattern

The standard safe pattern is active-passive. The primary machine runs the full stack: execution client, consensus client, and validator client. The standby machine runs an execution client and a consensus client, stays fully synced, and keeps its validator client stopped, with its connection to the signer disabled. When the primary fails, an operator, or a tested automation, confirms the primary validator client is stopped, and only then starts the validator client on the standby. The standby is ready in minutes because it was already synced. It is safe because it was not signing while the primary was alive.

Some chains offer protocol-level redundancy instead of operator-managed failover. MultiversX, for example, documents hot-standby nodes with a redundancy level: the main node signs, standbys sync but do not sign, and a standby takes over only after a set number of missed rounds by the main node. If your chain offers a feature like this, use it, and follow its exact rules, including never assigning the same redundancy level to two nodes. If your chain does not, the active-passive pattern above is the safe default, and you must build the stop-first discipline yourself.

Stop first, then start: the rule that prevents slashing

Safe failover has a strict order, and the order matters more than the speed. Stop the validator client on the old machine and confirm the process has actually exited, not just been told to stop. Check the process list, not just the service status. Then, and only then, start the validator client on the new machine with the same keys. This is the same rule as a safe migration, and it is the rule most slashing incidents break: an operator starts the new server before stopping the old one, or restores a backup onto a second machine while the first is still running, and both sign for a few minutes. Those few minutes are enough, as our earlier post on running multiple validators on one server explains in detail.

Automated failover deserves extra suspicion. Automation that can activate two clients at once is worse than manual failover, because it can double-sign while nobody is watching. If you automate, automate the stop first: the automation must prove the primary is stopped, through a health check that fails closed, before it starts the standby. A health check that assumes the primary is dead because it cannot reach it is not proof. Network partitions are exactly when both sides believe they are the survivor. Design for that case, and test it on testnet, not in production at 3am.

Slashing protection: the second line of defence

Validator clients keep a slashing-protection database that records what they have already signed, and refuse to sign a conflicting attestation or block for the same slot. This database is your second line of defence when the stop-first rule fails, but only if it is shared or replicated between the primary and standby signer instances. A standby with an empty or stale protection database cannot know what the primary already signed, and may happily sign a conflicting message. Before any failover, verify that the protection data is current on the machine about to sign, and never delete or reset it to fix a sync problem without understanding what you are discarding.

Key safety belongs in the same conversation. Our validator keys guide covers separation of signing and withdrawal keys and offline backups, and our key rotation guide covers changing keys without downtime panic. For larger setups, consider a remote signer or a key-management layer that refuses conflicting signatures at the signing layer itself, so that even a mistaken double-start cannot produce a slashable signature. That layer is cheaper than a slashing event, every time.

Beacon node redundancy is different, and safer

Not all redundancy carries slashing risk. Some validator clients support connecting to multiple beacon nodes, and this is redundancy you should use freely. If your primary beacon node lags or goes down, the validator client can read chain data from a second beacon node hosted elsewhere, without any change to who is signing. Run beacon nodes in different zones or with different providers, monitor their sync lag independently, and alert when the gap between them grows. This protects you against the most common outage, a single node falling behind, without touching the signing path at all.

The same logic applies to RPC endpoints for your applications. Our RPC failover guide shows how a primary plus fallback endpoint, with health checks on chain-tip lag, keeps apps serving during a node outage. RPC failover is safe to automate aggressively, because serving reads twice is harmless. Signing twice is not. Keep those two failover designs in separate mental boxes, with separate automation rules.

Distributed validators: a third option worth knowing

Distributed Validator Technology, or DVT, offers a different answer to the redundancy problem. Instead of one key on one machine, the validator key is split into shares held by a cluster of operators, and a threshold of them must agree on what to sign before a valid signature is produced. The private key is never reconstructed in one place, and the cluster keeps signing as long as enough operators are online. Projects in this space include Obol and SSV, and their documentation is the place to start if this model fits your setup. DVT adds operational complexity and its own failure modes, so test it on testnet and understand its threshold maths before committing real stake to it.

The failover drill checklist

A redundancy setup that has never been tested is a hope, not a plan. Run this drill on testnet at least once a quarter. One, confirm the standby is synced and within a small lag of the chain tip. Two, stop the primary validator client and verify the process exited. Three, start the validator client on the standby and confirm it attests correctly. Four, fail back the same way, stop first, then start. Five, check the slashing-protection database survived both moves intact. Six, write down what was slow or confusing, and fix that, not just the failure. The drill that finds a gap on testnet is the drill that saves a validator on mainnet.

If you would rather not run this drill alone, DevOpsBy builds and tests failover setups for validator teams, including monitoring that pages a human before users notice. Start with a free infra audit, or read our managed node operations page for what ongoing coverage looks like.

Disclaimer: This post is educational only, not financial advice. Figures are listing or project data as of 2026-11-05 and may change - always confirm version numbers, requirements, and dates in the official docs before acting on them.

Related Articles

DevOps & Infrastructure Services

Need hands-on help? We build and manage the infrastructure behind the concepts covered in this article.

DevOpsBy Assistant

How can we help?

Select a topic to get started: