A validator without a tested backup is one disk failure away from losing everything that cannot be re-downloaded. Chain data can be re-synced, given time and bandwidth, but validator keys cannot be recreated, and a configuration that took weeks to tune cannot be rebuilt from memory at 3am during an outage. Backups are the difference between an incident that costs an hour and an incident that costs the validator. This post explains what to back up, how often, where to store it, and how to prove a restore works before you need it.
The short answer: back up keys on creation and on every change, back up configuration on every change, and snapshot node data on a schedule that matches how long a re-sync would take. Store it all encrypted, off the server, and somewhere a failure of that server cannot reach. And test the restore regularly, because a backup that has never been restored is a guess. Our upgrade guide depends on these habits, and our post on securing validator keys covers the key-handling side in more depth.
What is worth backing up
Four things matter. First, validator keys: the signing key that attests and proposes, and the withdrawal credentials that control the funds. These cannot be replaced, so they get the strongest protection. Second, configuration files: client settings, startup scripts, firewall rules, and any tuning you applied. Third, node data snapshots: a point-in-time copy of the chain database that lets you restore quickly instead of re-syncing from genesis. Fourth, the runbook: the written steps for starting, stopping, upgrading, and restoring the node, because the person restoring may not be the person who set it up.
Treat keys and data differently. Keys are small, change rarely, and must never be lost, so back them up immediately on creation and keep several encrypted copies in separate places. Node data is large, changes constantly, and can be rebuilt by syncing, so snapshot it on a schedule and accept that a restore will always be slightly behind the chain tip.
What is NOT worth backing up
Not everything on the server deserves backup space. The full chain database is the obvious example: on a large chain it can be hundreds of gigabytes or more, and backing it up daily is slow and expensive when a snapshot plus a re-sync gets you to the same place. Back up a snapshot on a schedule instead, and let the chain itself be the source of truth for the rest. Logs are another: keep recent logs for troubleshooting, but do not archive them as if they were data, and never back up temporary files, caches, or package downloads.
The test that decides whether something belongs in a backup is simple: if the server died right now, would you need this exact copy to get running again, or could you rebuild it? Configuration and keys pass the test. A full copy of chain data that you can re-download does not, unless your re-sync time is so long that a snapshot is genuinely cheaper than the downtime.
How often to back up each piece
Frequency follows change, not the calendar. Keys are backed up on creation and whenever they change, which should be rare, and the backup is verified immediately. Configuration is backed up on every change, which is easy to automate with a version control commit or a file copy to the backup store. Node data snapshots follow a schedule based on re-sync cost: daily for a chain that takes a day to sync, weekly for a small chain that re-syncs in an hour, and always before an upgrade.
A backup schedule template
- Keys: on creation and on change, verified immediately, stored encrypted in at least two separate places.
- Configuration: on every change, plus a weekly copy even if nothing changed, so a silent edit is still caught.
- Node data snapshot: daily or weekly depending on re-sync cost, and always before an upgrade.
- Restore test: monthly for data snapshots, and after every change to the backup process itself.
- Runbook: updated whenever a procedure changes, and reviewed in full once a quarter.
Adjust the template to your chain and your hardware, but do not adjust it to zero. A schedule that exists on paper and never runs is worse than no schedule, because it creates confidence without protection. Automate the scheduled copies where you can, and put the restore test on a calendar with a named owner.
Where to store backups
A backup stored on the same disk as the node is not a backup, it is a second copy waiting for the same failure. Follow the simple version of the 3-2-1 rule: three copies of anything that matters, on two different kinds of storage, with one copy off site. For a validator that usually means the live keys on the server, an encrypted copy on separate storage in a different location, and an encrypted offline copy for keys.
Keep the backup store separate from the server in every sense that matters: different disk, different machine, different account, and ideally a different provider or region. A billing problem, a compromised account, or a data centre outage should not be able to take both the node and its backups at once.
Encrypt everything, and test the decrypt
Every backup that leaves the server must be encrypted, and keys must be encrypted even at rest on the server. Use a standard tool and a passphrase or key that is stored separately from the backup itself, and write down where that passphrase lives and who can access it. An encrypted backup whose passphrase is lost is the same as no backup.
Test the decrypt as part of every restore test, not as a separate exercise. A restore test that skips decryption proves nothing about the step most likely to fail under pressure. If the decrypt needs a hardware device, a passphrase, or a second person, include that dependency in the test.
Restore test, step by step
A restore test uses a separate machine, or a staging node, so a failed test never touches production. The steps are the same every time, and they should be written in the runbook, not improvised.
- Fetch the backup. Download the snapshot and the configuration from the backup store, using the same credentials and path the runbook describes.
- Decrypt and verify. Decrypt the files, check their size and timestamp, and confirm the snapshot is the one you intended to restore.
- Restore and start. Put the data and config in place, start the node with the same startup command production uses, and watch the logs.
- Confirm sync and duties. Check the block height against an explorer, confirm peers are connected, and confirm the validator attests or produces blocks. A node that starts but never attests has failed the test.
- Record the result. Write down how long the restore took, what went wrong, and what needs fixing. A test with no record is a test that never happened.
Time the restore, because the restore time is the number that decides how bad an outage will be. If a restore takes six hours, your incident plan must assume six hours of downtime, not the thirty minutes the backup step takes. If the time is too long, the fix is usually a more frequent snapshot schedule or faster storage.
Common backup mistakes
The most common mistake is the untested backup: a job that runs every night, reports success, and has never been restored. The second is keys on the same server as the node, often in the same folder as the config, so one disk failure takes both. The third is an unencrypted cloud backup, which turns a storage breach into a key theft. The fourth is no versioning: a single backup file overwritten by a corrupted copy leaves no good restore point. Keep several generations of snapshots and configs.
The last mistake is treating backups as a setup task instead of an operations task. A backup plan set up once and never reviewed will quietly stop matching reality: the chain grows, the client changes its data layout, the passphrase holder leaves, the storage bill lapses. Review the plan once a quarter and run the restore test monthly.
A validator that can be rebuilt from a tested backup is a validator that can survive its worst day. Back up the keys, back up the config, snapshot the data, store it all encrypted and off the server, and prove the restore works every month. If your team runs validators and wants backups, monitoring, and restore drills handled as a service, see our node operations and node setup pages, or start with a free infra audit and we will review your backup plan along with the rest of the stack.
Disclaimer: This post is educational only, not financial advice. Figures are listing or project data as of 2026-11-07 and may change - always confirm version numbers, requirements, and dates in the official docs before acting on them.
