Every Linux distribution provides a range of utilities that you can use to make backups of your files. Here is the how I get the job done with crontab and a shell script using tar andfind

I wanted to secure all the data-files on my system on a regular basis. Regular for me implies on an automated basis. I've tried doing a manual backup every week or so but that caused too much hassle, so effectively I stopped making backups….
Further, I wanted to have an up-to-date backup of the most essential configuration settings on my system. This would help me in case I accidentally lost some datafiles or setting on my system. In case of losing everything I would need to reinstall my Linux distribution (plus extra installed software) and restore data files and settings. I decided that a full backup of the whole system wouldn't be worth the effort (and resources!).

Say “backup” and most Unix people think “tapedrive”. However, nowadays harddrives come that cheap that I chose to add an extra harddrive to my AMD 400 machine. This cheap option has the advantage that a harddrive can be mounted automatically, no need for manually inserting tapes. A disadvantage is that the backup resides in the same physical unit as the very data it is supposed to secure. However, since I do have a CD-writer om my local network I still have the option to copy a backup to a CD once in a while.
My main HD is 6Mb. The backup HD has 10Mb.

After adding the drive to my machine I wrote a little shell script (for bash) that basically does the following:

• it mounts my backupdrive
• it checks the date
• every sunday it makes a full backup of some datafiles and some configuration settings, older incremental backups are removed. other days it backups files that have been accessed the last day
• it dumps all the contents of a mysql database to the backup drive and zips the file
• it unmounts the backup drive

This script (I've stored it in /root/scripts) is called every night at 3:01 AM by cron. The crontab file looks like:

1 3 * * * /root/scripts/daily_backup

Add this line using crontab -e when root.

Here's the actual code:

#!/bin/bash
#
# creates backups of essential files
#
DATA="/home /root /usr/local/httpd"
CONFIG="/etc /var/lib /var/named"
LIST="/tmp/backlist_$$.txt"
#
mount /mnt/backup
set $(date)
#
if test "$1" = "Sun" ; then
        # weekly a full backup of all data and config. settings:
        #
        tar cfz "/mnt/backup/data/data_full_$6-$2-$3.tgz" $DATA
        rm -f /mnt/backup/data/data_diff*
        #
        tar cfz "/mnt/backup/config/config_full_$6-$2-$3.tgz" $CONFIG
        rm -f /mnt/backup/config/config_diff*
else
        # incremental backup:
        #
        find $DATA -depth -type f \( -ctime -1 -o -mtime -1 \) -print > $LIST
        tar cfzT "/mnt/backup/data/data_diff_$6-$2-$3.tgz" "$LIST"
        rm -f "$LIST"
        #
        find $CONFIG -depth -type f  \( -ctime -1 -o -mtime -1 \) -print > $LIST
        tar cfzT "/mnt/backup/config/config_diff_$6-$2-$3.tgz" "$LIST"
        rm -f "$LIST"
fi
#
# create sql dump of databases:
mysqldump -u root --password=mypass --opt mydb > "/mnt/backup/database/mydb_$6-$2-$3.sql"
gzip "/mnt/backup/database/mydb_$6-$2-$3.sql"
#
umount /mnt/backup

data files:
All my data files are in /root, /home or /usr/local/httpd.

settings:
I chose to backup all the setting in /etc (where most essential settings are stored), /var/named (nameserver settings) and /var/lib (not sure about the importance of this one…). I might need to add more to the list but I still far from being a Unix-guru . All suggestions are welcome!

tar versus cpio
The first version of this script used cpio to create backups iso tar. However, I found the cpio format not very handy for restoring single files so I chang ed it to tar. A disadvantage of using tar is that you can't (as far as I know) simply pipe the output of a find to it.
Using a construct like tar cvfz archive.tgz `find /home -ctime -1 -depth -print` caused errors for files that contained a space “ ” character. This problem was solved by wring the output of find to a file first (and using tar with the -T option).

How To Do A Full Backup
Using The tar Command

It's a good idea to do a full backup of the hard-drive after the initial installation as well as when you finally get your server set up the way you want. Having a snapshot of your system right after the initial installation gives you something to revert back to should you want to reconfigure your server without starting from scratch. Linux has many backup utilities but the old standard is still the favorite of admins because of the flexibility offered by its myriad of options. 

tar commands can become quite complex. It's easier to enter the command in a text file and make it a shell script. We also need to create a directory to hold the backup file. We'll use a separate directory so we can exclude that directory from the backup (we don't want tar trying to backup a file it's in the process of creating). Enter the following commands:

Then use the nano editor to create our shell script file with the command:

nano fullserver.sh

and enter the following command into it (and don't miss that period at the end of the command):

tar -cvpf /backups/fullbackup.tar –directory=/ –exclude=proc
–exclude=sys –exclude=dev/pts –exclude=backups .

The above command is on multiple lines for readability. Enter everything on the same line when entering the command in the editor. Once entered, exit the editor (Ctrl-X) saving the file. 

The c option creates the backup file. 

The v option gives a more verbose output while the command is running. This option can also be safely eliminated. 

The p option preserves the file and directory permissions. 

The f option needs to go last because it allows you to specify the name and location of the backup file which follows next in the command (in our case this is the/backups/fullbackup.tar file). 

The –directory option tells tar to switch to the root of the file system before starting the backup. 

We –exclude certain directories from the backup because the contents of the directories are dynamically created by the OS. We also want to exclude the directory that contains are backup file. 

Many tar references on the Web will give an exclude example as:

–exclude=/proc

However, this will often result in excluded directories still being backed up because there should not be the leading / (slash) character in front of the directory names. If you were to use the above example, tar would back up the proc directory which would result in errors saying that files had changed since they were read. 

In addition to holding your backup file, the /backups directory now holds your script file also. You have to make your script executable before you can run it. To do that and run it enter the following two commands:

To restore your “tar ball” you need to copy it to the top-most point in the file tree that it backed up. In our case the –directory option told tar to start backing up from the root of the file system so that's where we would want to copy the tar ball to. Then simply replace the c option with the x parameter line so:

tar -xvpf /fullbackup.tar

Having a backup file is nice but not of much use if the hard-drive itself fails. If your server is running the wu-ftpd FTP server daemon, you should FTP the tar ball off of your server so that it is stored on a different system. 

Note that with the use of the –exclude statements when doing the backup, the tar backup file (tar ball) isn't something you could use to do a “bare metal” restore (because the excluded directories would be missing). However, it does work very well if you do an initial vanilla install of the OS and then un-tar your tar ball on top of that. 

Saving Space With Compression 

You can compress your backup files to save storage space. Because a lot of Linux files are text files the space savings can be significant. Enough, perhaps, to be able to fit your archive file on a CD or DVD. 

To enable the compression you have to add the z switch to both the tar-ing and untar-ing commands. You also have to change the file name to indicate the compression. Our tar-ing command becomes:

tar -zcvpf /backups/fullbackup.tar.gz –directory=/ –exclude=proc
–exclude=sys –exclude=dev/pts –exclude=backups .

Likewise, our untar-ing command becomes:

tar -zxvpf /fullbackup.tar.gz

I just wanted to share the two scripts I have put together to manage my system backup needs. They have been built from a variety of sources and tailored to my individual needs, obviously YMMV.

The first one does a backup of my /home partition to an SD card in my AspireOne, ignoring media files and a few others:

“#!/bin/bash”

“# Backup /home to SD card”

“logfile=~/rsync_log.txt”

“if mountpoint -q /home/marc/backup”

“ ”“then”

“ ”“echo “$(date) Backup Drive is mounted.” > $logfile”

“ ”“echo “$(date) Starting script, about to run rysnc” > $logfile”

“ ”“/usr/bin/rsync -a –delete –exclude-from /home/marc/exclude /home/marc /home/marc/backup >> $logfile 2>&1”

“ ”“echo “$(date) returned from rsync call” >> $logfile ”

“else”

“ ”“echo “$(date) Backup Drive not mounted,mounting now.” > $logfile”

“ ”“mount /dev/mmcblk0p1 /home/marc/backup”

“ ”“echo “$(date) Backup Drive is mounted.” > $logfile”

“ ”“echo “$(date) Starting script, about to run rysnc” > $logfile”

“ ”“/usr/bin/rsync -a –delete –exclude-from /home/marc/exclude /home/marc /home/marc/backup >> $logfile 2>&1”

“ ”“echo “$(date) returned from rsync call” >> $logfile ”

“fi]”

Its not elegant and can probably be vastly improved and simplified as I am very new at this but it seems to work for me.

The second one does a backup of /, ignoring /home and a few others:

“#!/bin/bash”

“# Backup / to SD card with rsync”

“logfile=~/tar_rsync_log.txt”

“if mountpoint -q /home/marc/backup”

“ ”“then”

“ ”“echo “$(date) Backup Drive is mounted.” > $logfile”

“ ”“echo “$(date) Starting script, about to run tar/rysnc” > $logfile”

“ ”“#/usr/bin/rsync -a –delete –exclude-from /home/marc/tar_exclude / /storage/backup >> $logfile 2>&1”

“ ”“#/bin/tar cvzpf /home/marc/backup/root_backup.tgz /storage/backup”

“ ”“/bin/tar cvzpf /home/marc/backup/root_backup.tgz –exclude=home –exclude=media –exclude=proc –exclude=locale –exclude=bin/tar –exclude=storage –exlcude=root/.local/share/Trash / >> $logfile 2>&1”

“ ”“echo “$(date) returned from tar/rsync call” >> $logfile ”

“else”

“ ”“echo “$(date) Backup Drive not mounted,mounting now.” > $logfile”

“ ”“mount /dev/mmcblk0p1 /home/marc/backup”

“ ”“echo “$(date) Backup Drive is mounted.” > $logfile”

“ ”“echo “$(date) Starting script, about to run tar/rysnc” > $logfile”

“ ”“/usr/bin/rsync -a –delete –exclude-from /home/marc/tar_exclude / /storage/backup >> $logfile 2>&1”

“ ”“/bin/tar cvzpf /home/marc/backup/root_backup.tgz /storage/backup”

“ ”“echo “$(date) returned from tar/rsync call” >> $logfile ”

“fi”

It is equally as messy as the first, with the added joy of the log file not working, not that that is a big problem for me as it's primary function, the backup, works fine.
I call them both from root's crontab, the first one on the hour and the second one once a week. This way even if my hard drive melts all I need to do is eject the SD card and I can reinstall on a new hard drive and get my customisations and data back sharpish.

Any comments are very welcome, I need to get better at all this and criticism is the only way!

Last edited by badger (2011-06-15 08:52:08)