I run restic backups for the WordPress sites I operate and had documented what they retained. I had never restored one. Every drill in the playbook meant to change that was still marked Never run yet, so my confidence was asserted, not verified.
TL;DR
Documented backups are not proven backups. I built a catalog of 17 backup-confidence drills, set a cadence scaled to each drill’s cost, and made every restore target scratch rather than live production. The drills that cannot run yet are marked as such, not treated as green.
The backup job was not the proof
The configuration was documented; a working restore was not. Verification had to span six axes: snapshot freshness, repository integrity, restore from scratch, full-disaster recovery, posture audits, and credential recovery off the virtual private server (VPS). Each drill costs runtime, so cadence scales to importance.
A backup job that reports success looks like proof. It is not. A backup is a claim until a restore proves it, and that test cannot run over production: restoring over a live path is the failure the drill exists to prevent. The safe unit is the drill that exercises the backup without touching production. What belongs in a backup set is a separate rule .
The decision: a catalog of 17 drills, with a cadence scaled to importance
I organized backup confidence into a catalog of 17 drills, BCK-001 to BCK-017, each carrying the same metadata contract and split into six sections, one per axis.
The cadence follows importance: a snapshot count check daily, a 5% integrity check weekly, the full checks and restores monthly, and the disaster-recovery drills quarterly. A schedule table carries each drill’s run state, updated after every run, with anomalies appended to the incidents changelog.
I rejected trusting the documented restic retention without drills, which asserts confidence rather than verifying it. A scheduler would eventually run the drill layer unattended , and the off-VPS credential axis is a dependency of its own .
The discipline that makes a restore drill safe
Every single-target and full-VPS restore drill uses a scratch target only and destroys it afterwards. Files restore to /tmp/restore-test, never over live /etc, /var/www, or /home; a MySQL dump replays only into restore_drill_scratch; and the full-VPS drill restores to a temporary VPS without touching the live one or its DNS. Secret-bearing files such as wp-config.php and the agent’s .env file are checked with sudo test -s and stat only, never printed. Cleanup removes the scratch directory and drops the scratch database.
Restoring over a live path was the alternative: it turns the drill into the failure it prevents. A restored backup is also an input to a gate of its own, re-validated before it serves .
The honesty rule on recovery targets
Recovery targets mean nothing until measured. I state the recovery point objective (RPO) as inherent to the daily cadence, at most 24 hours. Every recovery time objective (RTO) is marked TO VALIDATE: a single file in 5 minutes or less, a WordPress site in 30 minutes or less, a full VPS in 4 hours or less. The audit drills record observed values only and flag a threshold rather than invent one.
A number no drill has measured is a promise the design has not earned, so I rejected stating them as fact.
What broke, and what the design guards against
Nothing broke. Every record behind this piece reports the same result, and I will not dress that into an incident it is not. The honest friction is the catalog’s own status. Every drill was still marked Never run yet: what I had was a plan, not a proven recovery, and the RTO figures stay unvalidated until the drills run.
The scratch guardrail keeps a drill from becoming the failure it prevents, and the TO VALIDATE markers keep an unmeasured target from passing as measured. A drill marked Never run yet is not a green light.
The outcome, and what it rests on
Backup confidence is now exercised on a fixed cadence across the six axes. Every drill that cannot run yet is marked Never run yet or TO VALIDATE rather than treated as green, and the restore drills prove file, database, and VPS recoverability with no exposure of live paths or secrets.
I state this at the playbook’s level, not as a measured result. All three records are narrated from the playbook, not read off a run, so I claim the design and its status, never a measured restore time or a success rate. A promise like that also rests on a tested restore, the discipline behind four WordPress sites moved to one VPS with nothing lost.
The operator behind this stack, and its trade-offs, is on the about page. If this is the shape of your problem, let’s talk.