Multiplicity
Multiplicity is a very basic frontend to rdiff-backup
. Its goal is to
provide easy management of multiple independant backup tasks.
Overview
rdiff-backup
uses rsync
(incremental file tree copies to save bandwidth), hard links (to save backup space), ssh
(to transmit data) and a very handy incremental machinery (to allow us to go back in time through our backups).
multiplicity
is a script aimed at easing the management of backup jobs. It is a basic frontend to rdiff-backup
. It consists of the main scipt and a directory filled with task files. Each task file defines a backup source, a backup target and various other parameters.
Task files
A task file must be in the configuration directory (may be set on the command line and defaults to /etc/multiplicity
). It should be named with a .task
suffix (read dot task). The following configuration values are accepted in a task file :
run_this
: This globally enables or disables the task file. Any value other thanyes
disables the task file.max_age
: delete files deleted more than this value ago and file history older than this value. This parameter is directly passed tordiff-backup
.source_root
: the root directory of the files you want to backup.target_root
: the root directory where you want to backup your files.include_file
: a full path (or relative to the configuration directory) to a file that lists what files to include in the backup. As the default is to backup all files, this is only usefull if the files you list here would have been excluded regarding theexclude_file
configuration. See next dot.exclude_file
: files to exclude from the backup.pre
: execute this in thebash
shell before executing the backup task. This must fit on one line.post
: execute this after the backup task. This must fit on one line.
Example task file /etc/multiplicity/example.task
:
run_this no
max_age 185D
source_root /
target_root /backup
exclude_file local.exclude
pre mount -o remount,rw /backup
post mount -o remount,ro /backup
Carefull, what separates configuration keys from configuration values is a real tabulation \t
. I should probably fix this, one day.
Example exclude file /etc/multiplicity/local.exclude
:
/var
/proc
/bin
*.log
Global pre and post scripts
There is also the possibility of running global pre and post scripts, i.e scripts that run before and after the task batch. Those are to put in the configuration directory under the names of pre.sh
and post.sh
.
Example pre.sh
:
1 2 3 4 5 6 |
|
Usage
multiplicity
accepts the following command-line options :
-cconfdir
: Specifyconfdir
as the configuration directory, i. e. where to look for task files.-v
: Be verbose. Default is really quiet.-h
: Display a (very) short help summary.
Download
You can browse multiplicity's source online or grab a snapshot archive on the same page.
You can get the source using git
:
$ git clone http://sousmonlit.zincube.net/~niol/repositories.git/multiplicity
Installation (short)
multiplicity
may run in any directory, but it was made to be run in the following way :
- Create the directory
/etc/multiplicity
and createtask
files. You can re-use the example from the distribution. - Drop the
multiplicity
script in acron
directory if applicable, for example on Debian, I dropped it in/etc/cron.daily
to make it run once a day. After a couple of days, you may want to putVERBOSE
to a zero value to stop getting cron report emails.
Installation (my setup)
This section describes my setup. The goal is to backup from ripley
to backup1
and backup2
.
Password-less SSH logins
The first thing to do is to setup password-less SSH logins for the root
user to the hosts you want to backup to (or the hosts you want to backup from, but I'll describe here how to use remote backup hosts to backup to). First create a key pair for this purpose.
# cd
# mkdir .ssh
# chmod 700 .ssh
# ssh-keygen -t rsa -f ~/.ssh/id_rsa
(no passphrase)
If, like me, you have aliases for your hosts and special ports, you may want to use a .ssh/config
file :
Host backup1
User backupuser
Port 400
HostName backupfool.foo1.org
Host backup2
User backupuser
Port 400
HostName backupguy.foo2.org
Now, copy your IDs to the remote hosts (repeat for backup2
) :
# ssh-copy-id -i ~/.ssh/id_rsa.pub backupuser@backup1
Password:
Now you should be able to login with the local root
user on localhost
:
# ssh backup1
(no password prompted)
backupuser@backupfool.foo1.org $
Multiplicity configuration
So, here is the plan : I want to backup several selected directories to a dedicated /backup
partition and I want to backup only a part of those selected directories to the remote hosts, especially avoiding to send to the remote hosts my passwords and my emails.
I run a mySQL server on ripley
. Thus, I use /etc/multiplicity/pre.sh
to dump the database contents to a file :
1 2 |
|
Then I described /etc/multiplicity/local.include
which is a list of the things on ripley
that I want to backup :
/etc
- **tmp
/home/niol
/root
- /var/fileserv/enac/album**tn
/var/fileserv/enac/album/**jpg
/var/backups
/var/spool/minimalist
/var/www
- **
Basically, I want to backup /etc
, no files that end with tmp
, my personal directory, stuff on my ftp server except thumbnails directories, /var/backups
, my mailing list manager config, my web directory and nothing else. Check the backup-manager
manual page for more information on file inclusions and exclusions (globbing filelists).
But when backing up to a remot host, I want to omit some files (the contents of the ftp server which is too big and the ones that contain clear text passwords) which I described in /etc/multiplicity/remote.exclude
:
/var/fileserv/
**id_rsa
**id_dsa
**identity
**gpg*
**.bash_history
**.fetchmailrc
**.gnubiffrc
/etc/ssh/ssh_host*key
/etc/multiplicity/pre.sh
/home/niol/fromannalee/home/niol/.gaim
/home/niol/fromannalee/home/niol/.mozilla
/var/www/sousmonlit/sql.conf.php
(I really don't like clear-text-stored passwords…)
This was the most difficult part bescause you need to know what is on your computer. But now, the only remaining thing is to describe multiplicity tasks.
/etc/multiplicity/local.task
(the local backup):
run_this yes
max_age 185D
source_root /
target_root /backup/local
include_file local.include
pre mount -o remount,rw /backup
post mount -o remount,ro /backup
/etc/multiplicity/backup1.task
(the order of exclude_file
and include_file
is important):
run_this yes
max_age 10D
source_root /
target_root backup1::/home/backupuser/fromripley/
exclude_file remote.exclude
include_file local.include
/etc/multiplicity/backup2.task
(this is the second friend I send my backups to, just in case):
run_this yes
max_age 10D
source_root /
target_root backup2::/home/backupuser/fromripley/
exclude_file remote.exclude
include_file local.include
This should be self explainatory and this should show what is multiplicity
about.
Running the script
The first run should be a bit long :
# /etc/cron.daily/multiplicity -v
From now on, you can check on the remote hosts which files have been backed up :
# ssh backup1
backupuser@backupfool.foo1.org $ ls fromripley
etc home rdiff-backup-data root var
You also may check what increments are available :
backupuser@backupfool.foo1.org $ rdiff-backup -l fromripley
Found 9 increments:
increments.2005-12-27T06:30:54+01:00.dir Tue Dec 27 06:30:54 2005
increments.2005-12-28T06:31:53+01:00.dir Wed Dec 28 06:31:53 2005
increments.2005-12-29T06:31:03+01:00.dir Thu Dec 29 06:31:03 2005
increments.2005-12-30T06:32:27+01:00.dir Fri Dec 30 06:32:27 2005
increments.2005-12-31T06:33:26+01:00.dir Sat Dec 31 06:33:26 2005
increments.2006-01-01T06:32:10+01:00.dir Sun Jan 1 06:32:10 2006
increments.2006-01-02T06:31:25+01:00.dir Mon Jan 2 06:31:25 2006
increments.2006-01-03T06:30:41+01:00.dir Tue Jan 3 06:30:41 2006
increments.2006-01-04T06:31:11+01:00.dir Wed Jan 4 06:31:11 2006
Current mirror: Thu Jan 5 06:31:08 2006
Check the rdiff-backup
beginners guide to know how to recover an old file…