The backup and file recovery system
Introduction
Linux system backups are performed using the "backup manager" software. Documentation etc., is available on the WWW.
Backups are performed daily at 0115 using the shell script /var/archives/scripts/dobackup. This creates backup archives. The archives are created in the local system directory defined by backup manager's BM_REPOSITORY_ROOT configuration parameter.
Backed up archives need to kept securely. The company operates a Microsoft Windows server specifically for such purposes. The programme 'smbclient' is used to transfer the archives to the secure server which is known as adminserver.
The operation of the backup system is controlled by the configuration file /etc/backup-manager.conf. A listing of this file on 13/8/2008 is shown below.
export BM_REPOSITORY_ROOT="/var/archives/files" export BM_REPOSITORY_USER="root" export BM_REPOSITORY_GROUP="root" export BM_ARCHIVE_PURGEDUPS="true" export BM_ARCHIVE_TTL="10" export BM_ARCHIVE_METHOD="tarball-incremental" export BM_TARBALL_NAMEFORMAT="long" export BM_TARBALL_FILETYPE="tar.bz2" export BM_REPOSITORY_SECURE="true" export BM_ARCHIVE_PREFIX="linux" export BM_TARBALL_DIRECTORIES="/etc /usr/utils /srv /var/log /usr/home ↵ /useradmin /usr/moodledata /usr/share /usr/local /usr/bin ↵ /usr/sbin /usr/include /usr/lib /lib" export BM_TARBALLINC_MASTERDATETYPE="weekly" export BM_TARBALLINC_MASTERDATEVALUE="6"
For a full discussion see the on line documentation
Note The symbol ↵ is used in the script listings to indicate that the next line of text is really part of the current line.
The dobackup script
Here is a listing of the dobackup script.
# # The following creates backup files in accordance # with configuration information in /etc/backup-manager.conf # START=`date` /usr/sbin/backup-manager DONEARCHIVES=`date` LOG=/var/archives/logs/daily SLOG=/var/archives/logs/sambalog SUMMARY=/tmp/$$.sum DATES=`date +%Y%m%d` cd /var/archives/files cp /dev/null $SUMMARY cp /dev/null $LOG # # Generate logging information # The awk programme 'asum.awk' pretty prints # the summary information # for i in *$DATES.tar.bz2 do NAME=`echo $i| sed 's/linux//' | sed 's/-/\//g' | cut "-d." -f1` FSIZE=`ls -l $i | sed 's/ */ /g' | cut "-d " -f5` NFILES=`tar tfj $i | wc -l` echo $NAME $FSIZE $NFILES >> $SUMMARY done echo "Files Bytes Directory" >> $LOG echo "===== ===== =========" >> $LOG awk -f /var/archives/scripts/asum.awk < $SUMMARY >> $LOG echo >> $LOG # # Now to move archives to backup server # /usr/local/samba/bin/smbclient //adminserver/caddy Mumble -I 192.168.0.101 ↵ -U ldap-linux << EOS > $SLOG 2>&1 cd linux_backup prompt no mput *$DATES.tar.bz2 mput *.md5 mput *.txt EOS # # Check whether copying worked. If so local # copies are removed. # if [ $? -ne 0 ] then echo " *****************************************" >> $LOG echo " *Moving archives to backup server failed*" >> $LOG echo " * Will try again tomorrow *" >> $LOG echo " *****************************************" >> $LOG else echo " Archives moved to backup server" >> $LOG rm *$DATES.tar.bz2 fi DONE=`date` echo " Backups started at " $START >> $LOG echo " Archives prepared " $DONEARCHIVES >> $LOG echo " Transfer to remote " $DONE >> $LOG echo `date` >> /var/logs/backup cat $LOG >> /var/logs/backup cat $SLOG >> /var/logs/backup rm $SUMMARY
As well as creating the back up archives and copying them to the backup server, this script also creates logging information that is included in the daily monitoring output. "Mumble" is the Active Directory password for the user "ldap-linux".
smbclient is normally used interactively, here it takes its input from within the script. The inputs are shown in a distinctive colour in the listing above.
The recover script
The recover script is provided as a simple way to recover files from the backup system. It operates interactively and requires the user (who has to be "root") to specifiy the required file (which might be a directory, in which case every file in the directory is recovered), the number of archives to look at and whether the oldest or most recent found version of the file is to be recovered.
The script is designed primarily to recover specific files that have been accidentally or deliberately deleted from a working file system.
The script invokes a separate script called "getarchive" that actually fetches a particular archive file to the current directory. You can write your own script to use whatever method is appropriate to recover such files depending on how you have stored the files remotely (or locally).
Here's a listing of the recover script.
#
# backup-manager file recovery script.
# ===================================
#
# This is intended purely as a guide.
#
# In the interests of generality, a separate script "getarchive"
# has to be written that recover an archive file from wherever
# it has been saved - possibly via FTP, SMB or whatever.
#
# 'getarchive' script should take the archive file name as a
# command line argument, recover the archived file to the
# current directory and return exit code 0 for success and 1
# for failure.
#
# Edit the following 3 lines to reflect local set-up.
# RECDIR Directory that will hold recovered files and directories
# CONF The backup manager configuration file
# SDIR Directory holding this script (and 'getarchive')
#
RECDIR=/var/archives/recovery
CONF=/etc/backup-manager.conf
SDIR=/var/archives/scripts
#
#
read -p "Name of file/directory to recover " TORECOVER
INIT=`echo $TORECOVER | cut -c1`
if [ $INIT != "/" ]
then
echo "Full path name (relative to root) must be given"
exit
fi
read -p "How many days back " DAYS
read -p "Do you want oldest version of $TORECOVER [y/n] " OLDEST
if [ $OLDEST = 'y' ]
then
OLDEST=1
else
OLDEST=0
fi
echo "Will attempt to recover $TORECOVER from backups into $RECDIR"
echo "Will check archives up to $DAYS days ago [" `date --date=-${DAYS}days +%d/%m/%Y` "]"
if [ $OLDEST -eq 1 ]
then
echo "Will extract oldest available version"
else
echo "Will extract youngest available version"
fi
echo "Checking hierarchy"
#
# Checks that the initial path name of the file to recover matches one of the
# directories actually archived. BACKUPDIR is name of one of the directories
# nominated in backup-manager's configuration file.
#
# From it's name we can determine the archive file name.
#
DLIST=`grep BM_TARBALL_DIRECTORIES $CONF | cut -d\" -f2 `
PFX=`grep BM_ARCHIVE_PREFIX $CONF | cut -d\" -f2 `
cp /dev/null /tmp/$$blist
for i in $DLIST
do
echo $i >> /tmp/$$blist
done
BNAME="/"
MESG=""
BACKUPOK=0
WLIST=`echo $TORECOVER | sed "s/\// /g"`
for j in $WLIST
do
echo Checking $j
if [ -L $BNAME ]
then
#
# Small kludge to sort out symbolic links
#
LNAME=`ls -l $BNAME | sed 's/ */ /g' | cut "-d " -f10`
BNAME=$LNAME
fi
if grep "^$BNAME$" /tmp/$$blist > /dev/null
then
BACKUPOK=1
BACKUPDIR=$BNAME
break
else
if [ -z "$MESG" ]
then
MESG="$BNAME not currently backed up"
fi
fi
if [ $BNAME = "/" ]
then
BNAME=/$j
else
BNAME=$BNAME/$j
fi
done
if [ $BACKUPOK -eq 0 ]
then
echo "Cannot recover $TORECOVER - $MESG"
exit
else
echo "Will attempt recovery from $BACKUPDIR"
fi
#
# Construct "root" of archive name (date will be appended later) and also file name as
# it will appear in archive (no initial "/")
#
AROOT=`echo $BACKUPDIR | sed "s/\//-/g"`
AROOT=$PFX$AROOT
RECNAME=`echo $TORECOVER | cut -c2-`
cd $RECDIR
RECOVERED_OK=0
j=0
while [ $j -lt $DAYS ]
do
if [ $OLDEST -eq 1 ]
then
k=`expr $DAYS - $j`
else
k=$j
fi
j=`expr $j + 1`
#
# BFILE is actual archive name. First check whether there is a local copy - possibly
# from previous recovery operations. Local copies of archives should be cleared from
# time to time.
#
DSTR=`date --date=-${k}days +%Y%m%d`
BFILE=$AROOT.$DSTR.tar.bz2
if [ ! -f $BFILE ]
then
echo recovering $BFILE from back up server
/bin/sh $SDIR/getarchive $BFILE
if [ $? -ne 0 ]
then
echo could not recover $BFILE
continue
fi
else
echo $BFILE already copied from back up server
fi
echo checking archive $BFILE
#
# tar does the actual extraction. It is the user's responsibility to move the recovered
# file(s) to the correct place.
#
tar xfj $BFILE $RECNAME 2>/dev/null
if [ $? -eq 0 ]
then
echo File recovered to $RECDIR/$RECNAME
RECOVERED_OK=1
break
fi
done
if [ $RECOVERED_OK -eq 0 ]
then
echo Could not find $TORECOVER in backups
fi
rm /tmp/$$*
The getarchive script that we use is shown below. This reflects our use of 'smbclient' to fetch files from a Microsoft Windows server and includes various local settings.
# # Script for retrieving file from remote archive. # Command line argument is name of file to retrieve. # If file is retrievable it is retrieved to the current # directory. # # Return code is 0 for good operation # TLOG=/tmp/$$.slog if [ $# -ne 1 ] then exit 1 fi /usr/local/samba/bin/smbclient //adminserver/Linux Mumble -I 192.168.0.101 -U ldap-linux <<EOS > $TLOG 2>&1 prompt no get $1 EOS SMBEX=$? if [ $SMBEX -eq 0 ] then grep -l "NOT_FOUND" $TLOG > /dev/null if [ $? -eq 1 ] then RV=0 else RV=1 fi else RV=$SMBEX fi rm -f $TLOG exit $RV
And here is a log of a typical dialogue.
linux:/var/archives/scripts # ./recover
Name of file/directory to recover /srv/www/icons/mrtg-m.png
How many days back 10
Do you want oldest version of /srv/www/icons/mrtg-m.png [y/n] n
Will attempt to recover /srv/www/icons/mrtg-m.png from backups into ↵
/var/archives/recovery
Will check archives up to 10 days ago [ 03/08/2008 ]
Will extract youngest available version
Checking hierarchy
Checking srv
Checking www
Will attempt recovery from /srv
recovering linux-srv.20080813.tar.bz2 from back up server
Domain=[PTWOL] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager]
getting file \linux_backup\linux-srv.20080813.tar.bz2 of size 43998 ↵
as linux-srv.20080813.tar.bz2 (499.6 kb/s) (average 499.6 kb/s)
checking archive linux-srv.20080813.tar.bz2
recovering linux-srv.20080812.tar.bz2 from back up server
Domain=[PTWOL] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager]
getting file \linux_backup\linux-srv.20080812.tar.bz2 of size 45182 ↵
as linux-srv.20080812.tar.bz2 (735.4 kb/s) (average 735.4 kb/s)
checking archive linux-srv.20080812.tar.bz2
recovering linux-srv.20080811.tar.bz2 from back up server
Domain=[PTWOL] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager]
getting file \linux_backup\linux-srv.20080811.tar.bz2 of size 43890 ↵
as linux-srv.20080811.tar.bz2 (779.3 kb/s) (average 779.3 kb/s)
checking archive linux-srv.20080811.tar.bz2
recovering linux-srv.20080810.tar.bz2 from back up server
Domain=[PTWOL] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager]
getting file \linux_backup\linux-srv.20080810.tar.bz2 of size 43976 ↵
as linux-srv.20080810.tar.bz2 (456.9 kb/s) (average 456.9 kb/s)
checking archive linux-srv.20080810.tar.bz2
recovering linux-srv.20080809.tar.bz2 from back up server
Domain=[PTWOL] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager]
getting file \linux_backup\linux-srv.20080809.tar.bz2 of size 145090754 ↵
as linux-srv.20080809.tar.bz2 (993.6 kb/s) (average 993.6 kb/s)
checking archive linux-srv.20080809.tar.bz2
File recovered to /var/archives/recovery/srv/www/icons/mrtg-m.png
User input is indicated in distinct colour. Note that the script fetched 4 archives before finding one that contained the required file. This happens because the backup system does incremental archives and had to work back 5 days before finding a full back up. Note also that the recovered file has not been recovered to its correct position in the file system. This is left to the administrator who might wish to check that this is really what the user wants before possibly overwriting other files.