How to Schedule and Rotate Backups Without Losing Old Versions

A backup rotation schedule is the rule that decides how many restore points you keep and how long each one survives before it is deleted or overwritten. Get it right and you can recover from a mistake made ten minutes ago or a corrupted file discovered six months later; get it wrong and you either run out of disk or discover that every copy you hold was taken after the damage was already done.

The frequency question and the retention question are separate. Frequency controls your recovery point objective, meaning how much data you are willing to lose. Retention controls how far back you can reach when nobody noticed the problem at the time. Most people conflate the two and end up with a single daily job that keeps the last seven copies, which is fine until a bad deploy or a silent data corruption has been replicated into all seven.

Choosing a backup frequency that matches your change rate

Look at how often your data actually changes, not how nervous you feel. A database that takes writes all day needs a different cadence from a static site whose content changes when someone publishes. The usual pattern for a busy system is a nightly full backup plus incremental or differential runs during the day, because a full dump every hour is mostly wasted I/O on a dataset that only moved a few megabytes.

For filesystem work, rsync with hard links is the classic mechanism. Each run creates a new directory that looks like a complete tree, but unchanged files are hard links to the previous run, so the space cost is only the deltas. The command shape looks like this:

rsync -a --delete --link-dest=/backups/current /srv/data/ /backups/2024-06-01/
ln -sfn /backups/2024-06-01 /backups/current

The --link-dest flag is the whole trick. Files identical to the reference tree are linked rather than copied. Note the ordering: you point current at the new tree only after the transfer succeeds, otherwise a failed run can poison the reference. For databases, a physical copy of the data directory while the server is running is not a backup, it is a future corruption report. Use the engine's own dump tool so the output is consistent.

Retention policy and the backup rotation schedule

Retention is where the schedule earns its name. A flat policy of "keep the last N" is easy to implement and easy to reason about, but it gives you dense coverage of the recent past and nothing else. The standard answer is a tiered or generational scheme, often described as grandfather, father, son: keep many recent points, fewer medium-term points, and a small number of long-term ones.

A practical shape is hourly for a day, daily for a month, monthly for a year, and yearly beyond that. You do not need a scheduler that understands those words. You need a naming convention that encodes the timestamp and a pruning pass that keeps the newest entry in each bucket. Storing the timestamp in the directory name makes the retention script a pure text problem.

2024-06-01T02:00:00Z   keep (newest daily)
2024-06-01T03:00:00Z   drop (hourly bucket, older)
2024-05-01T02:00:00Z   keep (newest monthly)
2023-06-01T02:00:00Z   keep (newest yearly)

Write the pruning logic so that it is idempotent and so that it never removes the most recent successful backup, whatever the bucket rules say. A common failure is a retention script that runs before the new backup finishes and deletes the only good copy. Guard it with a marker file written on success, and have the pruner refuse to act if that marker is missing or older than expected.

Immutable and offsite copies

Retention on the same host is not retention against ransomware, accidental deletion by an administrator, or a failed disk. If the credentials that write the backups can also delete them, the attacker or the tired operator wins. The fix is a second copy that the primary system cannot modify, whether that is object storage with a write-once retention window, a separate server that pulls rather than receives, or physical media rotated offsite.

Pull-based replication is worth the extra setup. The backup server initiates the connection, so the production host never holds credentials that grant delete rights on the archive. A pull job can be as simple as an rsync or restic run on a timer, and the archive side enforces retention independently.

Verifying that the schedule actually works

A backup you have not restored is a hypothesis. Automate a check that does more than confirm the file exists and has a plausible size. For archives, list the contents and fail the job if the listing errors. For databases, restore into a scratch instance and run a query against a known table. The check should write its result somewhere you will see it, and a failed check should page someone rather than wait in a log.

Watch the storage curve. If your retention policy is working, usage should plateau rather than climb forever. A steadily rising total usually means the pruner is not matching the filenames the writer produces, often because of a timezone suffix or a changed date format. Log both the created and deleted counts on every run so the two numbers can be compared at a glance.

Start by writing down your recovery point objective and your recovery time objective, then pick a frequency that meets the first and a retention tier that covers your realistic worst case, which is usually a problem discovered weeks after it began. Implement the writer and the pruner as separate jobs, test a restore end to end, and only then trust the schedule. Revisit it when your data volume or change rate shifts, because a policy tuned for a quiet site will quietly stop protecting a busy one.