Since Migration2025, sourceware runs on a collection of VMs on a small number of bare-metal servers. This allows multiple backup strategies:

• "none" :-), meaning, the crown jewels of the sourceware forges are the git repos. These repos already exist duplicated on hundreds of developer workstations and major services. These can serve as the germs of recovery from a total disaster.

• public rsync. Mailing list archives and some other resources are actively mirrored via our rsync services. Ditto partial, but useful for disaster recovery.

• local git-backed config files. Cron jobs on sourceware git-control a variety of config files and directories for informal configuration management. See root crontab(1).

• remote system-wide rsync. Frank operates a periodic far-off-site rsync of the sourceware.org primary server filesystems, including all of the above, plus bugzilla databases etc.

• VM host LVM snapshots. We have started rolling out periodic automated LVM snapshots of the locally hosted VMs that make up the bulk of sourceware servers. The host machines are not directly accessible from the internet. See /root/VM-BACKUP.sh, a simple little shell script, and usual LVM commands.

• future: cross-host VM LVM snapshots. Once we have all our RDU2 and RDU3 hardware together again in RDU3, we can warm-backup VM LVM snapshots from the primary server to another one that's beefy enough to run sourceware. That should allow near-instant switchover in the case of severe hardware failure.