backuper

Pull-based backup agent for a fleet of servers. A master machine pulls configs and consistent database dumps from nodes over SSH into dated hardlink snapshots, mirrors them to any rclone-supported cloud, prunes retention (GFS) and restores anything from a single file to a whole node — also from the cloud.

Nodes need nothing except sqlite3 and SSH key access; everything else is installed automatically.

Install

On the master machine:

curl -fsSL https://git.arabianq.ru/arabianq/backuper/raw/branch/main/install.sh | bash

or manually:

git clone https://git.arabianq.ru/arabianq/backuper.git /opt/backuper && cd /opt/backuper
cp backup.conf.example backup.conf
cp examples/node.conf.example conf.d/$(hostname -s).conf
./bin/backuper check        # installs missing packages
rclone config               # create the remote named in backup.conf
./bin/backuper run --dry && ./bin/backuper run

How it works

node1 (ssh) ─┐
             ├─> staging/<node>/ ─> snapshots/<node>/<date>/ ─> rclone → cloud
master ──────┘   (per-run build)     (hardlink deduplication)       (GFS retention)

1. Staging — node tasks are pulled in on every run: rsync tasks first, then sqlite/pgdump dumps that overwrite raw copies at the same paths. Databases in a snapshot are always restorable by a plain copy.
2. Snapshot — rsync -a --link-dest against the previous date: a full browsable tree that costs only the changes.
3. Cloud — rclone copy of the new dated directory; later runs are deltas.
4. Retention — local by KEEP_LOCAL_DAYS, cloud by GFS (see below).
5. Webhook + exit code at the end.

backup.conf

Option	Default	Description
BACKUP_ROOT	/var/backups/servers	local snapshots, staging, restore downloads
LOCAL_ENABLED / CLOUD_ENABLED	true / true	where to store; both false is forbidden
KEEP_LOCAL_DAYS	14	local retention (days)
RCLONE_REMOTE / RCLONE_PATH	gdrive / backups/servers	upload target
KEEP_REMOTE_DAYS / WEEKLY / MONTHLY	30 / 8 / 6	cloud GFS: keep N recent days, N Mondays, N 1st-of-month
CLEANUP_REMOTE	true	prune old cloud snapshots
RCLONE_TRANSFERS / RCLONE_CHECKERS	8 / 4	rclone parallelism
RCLONE_COPY_LINKS	false	dereference symlinks on upload (--copy-links)
CLOUD_ARCHIVE	false	upload as one tar.zst per node/run instead of the raw tree (fast for many small files)
AUTO_INSTALL_PKGS	true	install missing packages, also on nodes
SSH_OPTS / RSYNC_TIMEOUT	sane defaults	ssh/rsync options
WEBHOOK_URL / WEBHOOK_ON / WEBHOOK_HEADERS	empty / error	run summary notifications
GLOBAL_RSYNC_EXCLUDES	.venv … *-wal *-shm	excludes for all dir/file tasks

Nodes: conf.d/<name>.conf

NODE_NAME="node1"
NODE_SSH="root@node1.example"   # empty = the master itself
NODE_STRATEGY="snapshots"       # snapshots | sync
NODE_CLOUD_ARCHIVE=""           # "" = inherit CLOUD_ARCHIVE, true, false
PRE_RESTORE_CMD=""              # hooks run on the node around a restore
POST_RESTORE_CMD=""
TASKS=( … )

STRATEGY=sync mirrors one tree without history (rclone sync to the cloud) — for huge nodes where dated snapshots are impractical.

NODE_STRATEGY=sync mirrors a single tree without history (rclone sync to the cloud) — for huge nodes where dated snapshots are impractical. With CLOUD_ARCHIVE=true (or per-node NODE_CLOUD_ARCHIVE=true) the cloud copy becomes one <date>.tar.zst under archives/<node>/ instead of the raw tree — a single API call per run, much faster for many-small-file nodes (gitea etc.). restore, status and rotate handle archives transparently.

Tasks

"type|field|field|…" — paths inside the snapshot are written without the leading /.

Type	Format	Purpose
dir	dir|<dir>|<dest>|[exclude=a,b,c]	recursive rsync
file	file|<file>|<dest>	single file
compose	compose|<dir>|<dest>	only *.yaml, *.yml, .env, recursively
sqlite	sqlite|<db>|<dest>	consistent sqlite3 .backup, no downtime
pgdump	pgdump|<container>|<pg user>|<db>|<dest.sql>	pg_dump via docker exec

dir/file/compose run first, dumps overwrite raw copies at the same paths. *-wal/*-shm are always excluded. Files that are not SQLite (e.g. Zoraxy's sys.db) must be captured via dir/file.

Commands

backuper run [node…] [--dry]     back up all or selected nodes
backuper restore                 interactive restore wizard
backuper restore <node> <date> <path> <target>
backuper status                  snapshots local/cloud, quota, disk usage
backuper list                    nodes and their tasks
backuper check                   packages, ssh, rclone, config validation
backuper rotate [node…]          prune per retention policy

Options: --dry, --yes, --push (restore in place), --hooks/--no-hooks.

Restore

backuper restore walks you through: node → source (local/cloud) → date → whole node or a file/directory search → destination (original location or a safe folder). Node hooks run around the restore.

Scripted:

backuper restore node1 latest opt/app/app.db /tmp/restore/   # to a folder
backuper restore node1 2026-09-24 data/app original --yes    # in place
backuper restore node1 latest "" /tmp/full-node1             # whole node

Snapshots missing locally are downloaded from the cloud. --push/original restores onto the node over SSH.

Retention

Local: everything older than KEEP_LOCAL_DAYS is deleted (the newest is always kept). Cloud: everything newer than KEEP_REMOTE_DAYS is kept, plus the last KEEP_REMOTE_WEEKLY Mondays and KEEP_REMOTE_MONTHLY 1st-of-month snapshots.

Webhook

A JSON summary is POSTed after every run (works with ntfy, Apprise API, anything):

{"source":"backuper","host":"master","status":"ok","duration_sec":143,
 "nodes_ok":["node1"],"nodes_failed":[],"errors":[]}

Pair it with a dead man's switch (Healthchecks.io) — a webhook cannot alert about a run that never started.

systemd

install.sh sets this up; manually:

cp systemd/* /etc/systemd/system/ && systemctl daemon-reload
systemctl enable --now backuper.timer          # daily at 04:00 + jitter
systemctl start backuper@node1                 # a single node

Gotchas

• Not everything that looks like a database is SQLite (e.g. Zoraxy's sys.db) — capture such files via dir/file.
• RCLONE_COPY_LINKS=true dereferences symlinks on upload; a symlink pointing at a big tree pulls it into the cloud. Keep it false unless you need symlink targets (e.g. letsencrypt/live/*).
• The first cloud upload uploads everything; Google Drive may stall on the last objects — later runs are deltas and take minutes.
• backup.conf and conf.d/*.conf are gitignored on purpose: live addresses and secrets.