Joergs LinuxPages: Backup

"Nobody cares if you do backups. All that counts is your ability to restore the data."

This page contains material about backup - from simple hints to complete scripts, and links to further references. Please note that a few of these scripts date many years back; they may no longer be actual.

Backup on CDs

Backup on multisession CD-R

This is discussed in-depth on my CD Writing page.

Backup on CD-RW and DVD-RW

This is discussed in-depth on my CD Writing page.

System Backup

Backup to
a(ny) network drive

The script backup2net.sh allows the backup of large amounts of data to another backup location. It isn't complicated at all - the only requirement is that the "other backup location" is mounted on the system where you want to perform the backup. Thus, you can use any USB disk drive, or a NAS server somewhere on your network!

The script uses rsync according to a description at this blog entry where I added the framework (logging and error checking) of backup2tape.sh. The resulting script has the following features:

can write data to any folder on the system (network or local)
uses rsync for efficient data transport
allows to specify include & exclude patterns
supports snapshots
supports multiple configurations via command line
backup report is mailed to administrator

Download the backup2net.sh script (updated 2011-01-04).

Backup to
a tape drive

A while ago the amount of data that should be included in a daily backup outgrew the volume of a CD-RW. At a time when I did not yet have a DVD writer but access to a tape drive, I wrote the script backup2tape.sh. Some features:

writes data to tape drive & verifies after write
allows to specify include & exclude patterns
supports full & differential backup
supports multiple configurations via command line
preserves access rights (ACL)
backup report is mailed to administrator

The script relies on Jörg Schilling's star. I am using this script with success for several years now to perform regular backups on several Linux machines, either manually or via cron, with DDS2 and DLT tapes. Different configuration files can be used as needed.

Download the backup2tape.sh script (updated 2010-12-28).

Backup to remote tape server

One of the wonderful things with Linux is the built-in "allround" networking. This makes it very easy to use almost any device that is attached to a remote computer, such as the display, scanner, or other devices. If you want to use a remote tape drive (say, located on machine "tapehost") to backup your local computer, use something along the following line:

tar cvf - /path/to/file | ssh tapehost 'buffer -s 32k -p 75 -m 10m > /dev/nst0'

Tape access usually requires root rights, so you may need to enable superuser access by adding the option -h to the entry for rshd in /etc/inetd.conf:

shell   stream   tcp   nowait   root    /usr/sbin/tcpd   in.rshd -Lh

You can use a similar procedure to clone an existing system to another one (where the remote system, here called targetPC, should be booted from floppy or CD, so that no access to the harddisk is required for system operation during the cloning process):

(cd / && tar cpf - .) | ssh targetPC '(cd / && tar xpf -)'

... or the other way round, i.e. you fetch the system from targetPC:

 ssh sourcePC '(cd / && tar cpf - .)' | (cd / && tar xpf -)'

Specific Backup and Recovery Tasks

Backing up
a whole partition

Everybody knows that MS Windows sometimes "crashes", and in rare cases you need to re-install the whole operating system. Now, reinstalling Windows requires many reboots to upgrade to the latest drivers for motherboard, graphics, sound, modem and the like. If you cannot do without Windows, I recommend to make a backup of the complete, fresh installation - not only of the files, but really of the whole partition. This is easily achieved using Linux (you may use any floppy-based Linux to do this):

dd if=/dev/hda1 | gzip -c > /mnt/nfs/part_win95.gz

... assuming hda1 is your windows partition, and /mnt/nfs is a free partition (local or remote) that you can write your data to. Alternatively, do the same thing over ssh:

dd if=/dev/hda1 | ssh login@remote 'gzip -c > /path/to/part_win95.gz'

The process may take a while, and the resulting file may be quite large (more than half of the partition size) even if the actually installed data are only 300 MB - this is due to the archival of the "raw" partiton data. Axel Buergers pointed out that this problem can be largely reduced by defragmenting and then "filling up" the Windows partition with a huge NULL file, something like dd if=/dev/zero of=/mnt/dos/eraseme bs=reallybignumber, which will abort once the partition is full. The file /mnt/dos/eraseme can then be deleted, and the "clean" partition archived as above. This "cleaning" process can reduce the size of the final partition image by 50%.

To unpack such a file to the original location:

gunzip -c /mnt/nfs/part_win95.gz  | dd of=/dev/hdx

Source: Patagonia CloneSys at the ETH Zürich.
See also: Partimage.org, a utility to save partitions.

Backup MySQL DB

Among others, I maintain a Linux system that runs some MySQL databases. To save the database files in regular intervals, the script below is called every night via crontab. It archives the files to a backup drive, in our case a NFS-mounted drive on a physically different computer. - Please adjust the mount points to your installation before launching. Indications for setting up the cron job are in the file.

Download the save_db.sh script.

Archive /home

Similar to the above, this is a simple script to archive the /home directory tree to a remote computer. It is called as a cron job. - Please adjust the mount points to your installation before launching. Indications for setting up the cron job are in the file.

Download the save_home.sh script.

Tools and Hints

Backup/Synchronise Firefox bookmarks

Firefox is a great web browser, but it has one downside: Mozilla products use a random folder name for user profiles. This causes a problem if you want to use the same bookmarks name on multiple computers, e.g. by synchronisation with unison.

The workaround is easy: Simply rename your profile folder in ~/.mozilla/firefox to whatever fits your needs - e.g. default.JHa. Then, edit ~/.mozilla/firefox/profiles.ini to reflect this change, and you're done.

Source: Micah Carrick's blog.

netcat

If you do need to copy data over the network in a very simple way - without scp or ssh, without ftp, even without telnet ... there is still netcat, the swiss knife of networking (on some systems the binary is called nc). I sometimes use a floppy-based Linux, such as tomsrtbt, to copy data between two computers, e.g. to backup the whole harddisk of a laptop.

An example how to use tar over netcat:

On the "sender" side, launch tar and pipe its output through netcat, specifying the destination IP address, "listen"-mode (-l) and an arbitrary port number, e.g. 5555:

tar cvf - . | netcat -l -p 5555

On the "receiving" computer, just give the sender's IP address (or hostname) and the same port number, and pipe the output through tar to unpack:

netcat 192.168.xxx.yyy 5555 | tar xvf -

... and the data will be transferred. The same works with almost any other command, e.g. dd instead of tar.

A Quote

A comment to the importance of Backup was also published in the Heise Newsticker in 2002-06:

Einen nachdenklichen Akzent setzte Douglas O'Shaugnessy vom Support-Service der Firma Legato, der mit 18 Spezialisten nach dem Einsturz der Türme des World Trade Centers vor Ort arbeitete. Shaugnessy [...] berichtete von der häufig vergeblichen Suche nach Recovery-Plänen und brauchbaren Inventar-Verzeichnissen. Dies ließ die rein technische Arbeit der Datensicherer zu einer Mischung aus Puzzle und Detektivspiel werden. Mangels aussagekräftiger Kennzeichnung mussten seine Spezialisten 20.000 Bänder auf der Suche nach den neuesten Backups durchforsten und mehr als einmal einen viel früheren Versionsstand der Sicherungen einspielen, damit die Firmen wieder arbeiten konnten.

Shaugnessy präsentierte eine Übersicht, nach der die Firmen den größten Schaden hatten, deren Mitarbeiter im World Trade Center vor allem mit Laptops arbeiteten. Rund 30 Prozent der Firmendaten seien verloren, weil die Mitarbeiter ihre mobilen Rechner nur unregelmäßig im zentralen Firmen-Backup sicherten. Shaugnessys Vortrag endete mit einer dringlich klingenden Bitte: "Dokumentieren Sie Ihren Backup-Prozess. Dokumentieren Sie Ihre Recovery-Maßnahmen. Dokumentieren Sie Ihr Tape-System. Drucken Sie diese Informationen mehrfach aus und bewahren Sie diese an anderen, sicheren Orten."

Jörgs LinuxPages: Backup