This is backup framework for creating flexible backup scripts.
- Module for upload backups to Amazon Glacier and remove old Glacier backups.
- MySQL module support creating backup with mysqldump and InnoBackupEx.
- LVM module for snapshots.
- Full/Diff backups of files backups with rsync.
- Full/Diff backups of btrfs with btrfs send/receive.
- Full/Diff backups of any binary files or block devices with diff-dd.
- Full/Diff backups of Libvirt VM's.
- Full/Diff backups of Libvirt VM's with external snapshots (works with qcow, qcow2, and raw file block devices).
- Full/Diff backups of S3 buckets (beta, can work with Amazon S3, Ceph and other S3 storage)
- Predefined jobs for popular cases.
- Install following dependency with your OS package manager:
For CentOS:
http://www.percona.com/downloads/percona-release/redhat/0.1-4/percona-release-0.1-4.noarch.rpm (Percona repo)
libvirt-python
python-pip
rsync
lvm2
p7zip
btrfs-progs
pigz
Percona-Server-client-57
percona-xtrabackup-24
For Debian/Ubuntu:
https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb (Percona repo)
python-libvirt
python-setuptools
python-pip
rsync
lvm2
p7zip
btrfs-progs
pigz
Percona-Server-client-57
percona-xtrabackup-24
- Install python package from pip:
pip install ebt
Configuration files stored in /etc/ebt/ directory.
File /etc/ebt/ebt.conf contain logging configuration.
File /etc/ebt/plans.py contains your backups jobs. Few examples of plans.py:
import ebt_jobs.btrfs
def btrfs_root_full():
backup = ebt_jobs.btrfs.BTRFSBackupFull(source='/', snap_dir='/.snap', dest_dir='/mnt/backup', day_exp=None, store_last=1)
backup.start()
def btrfs_root_diff():
backup = ebt_jobs.btrfs.BTRFSBackupDiff(source='/', snap_dir='/.snap', dest_dir='/mnt/backup', day_exp=None, store_last=5)
backup.start()This example of using predefined job ebt_jobs.btrfs.BTRFSBackupDiff. You can see all jobs in ebt_jobs/ directory.
Next example. You can inheritance your own class from base class:
import ebt_jobs.btrfs
import ebt_system
class BTRFSBackupFullWithRsync(ebt_jobs.btrfs.BTRFSBackupFull):
def _post_backup(self):
ebt_system.popen(command="rsync /usr/bin/rsync {0} remote:/mnt/backup/archive/".format(self.dest), shell=True)
class BTRFSBackupDiffWithRsync(ebt_jobs.btrfs.BTRFSBackupDiff, BTRFSBackupFullWithRsync):
pass
def btrfs_root_full():
backup = BTRFSBackupFullWithRsync(source='/', snap_dir='/.snap', dest_dir='/mnt/backup', day_exp=None, store_last=1)
backup.start()
def btrfs_root_diff():
backup = BTRFSBackupDiffWithRsync(source='/', snap_dir='/.snap', dest_dir='/mnt/backup', day_exp=None, store_last=5)
backup.start()You can write your own jobs from primitives that can be found in ebt_* dirs.
root » ebt --help
usage: ebt [-h] [-j JOBS [JOBS ...]] [-p PLAN] [-c CONFIG] [-v]
optional arguments:
-h, --help show this help message and exit
-j JOBS [JOBS ...], --jobs JOBS [JOBS ...]
List of jobs to run
-p PLAN, --plan PLAN Custom path to plans file
-c CONFIG, --config CONFIG
Custom path to config file
-v, --version Display program version and exit
Where -j or --jobs is the functions names from your plans.py file. Example: ebt -j btrfs_root_full btrfs_root_diff
If you specified few jobs they will be executed in sequential order.
You can use for restoring diff backups of Libvirt VM's or for creating diff backups from external scripts.
Backup examples:
ddd --if /backup/full_disk_copy.raw --df /dev/lvm/changed_disk_snapshot --of /backup/diff_disk_copy.ddd
ddd --if /backup/full_disk_copy.raw --df /dev/lvm/changed_disk_snapshot > /backup/diff_disk_copy.ddd
ddd --if /backup/full_disk_copy.raw --df /dev/lvm/changed_disk_snapshot | gzip > /backup/diff_disk_copy.ddd.gz
ddd --if <(zcat /backup/full_disk_copy.raw.gz) --df /dev/lvm/changed_disk_snapshot --of /backup/diff_disk_copy.ddd
zcat /backup/full_disk_copy.raw.gz | ddd --df /dev/lvm/changed_disk_snapshot --of /backup/diff_disk_copy.ddd
cat /dev/lvm/changed_disk_snapshot | ddd --if <(zcat /backup/full_disk_copy.raw.gz) --of /backup/diff_disk_copy.ddd
ddd --if <(ssh remotehost cat /backup/full_disk_copy.raw) --df <(ssh remote2host cat /dev/lvm/changed_disk_snapshot) | ssh remote3host dd of=/backup/diff_disk_copy.ddd
Restore examples:
ddd --mode restore --if /backup/full_disk_copy.raw --df /backup/diff_disk_copy.ddd --of /dev/lvm/disk
ddd --mode restore --if <(zcat /backup/full_disk_copy.raw.gz) --df <(zcat /backup/diff_disk_copy.ddd.gz) > /dev/lvm/disk
zcat /backup/full_disk_copy.raw.gz | ddd --mode restore --df <(ssh remotehost cat /backup/diff_disk_copy.ddd.gz) | ssh remote2host dd of=/dev/lvm/disk
Backup:
import ebt_files
import gzip
iffd = gzip.open('/backup/full_disk_copy.raw.gz', 'rb')
dffd = open('/dev/lvm/changed_disk_snapshot', 'rb')
offd = gzip.open('/backup/diff_disk_copy.ddd', 'wb')
differ = ebt_files.ddd.CreateDiff(iffd=iffd, dffd=dffd, offd=offd, block_size=16384)
differ.start()
Restore:
import ebt_files
iffd = gzip.open('/backup/full_disk_copy.raw.gz', 'rb')
dffd = gzip.open('/backup/diff_disk_copy.ddd', 'rb')
offd = open('/dev/lvm/disk', 'wb')
differ_restore = ebt_files.ddd.RestoreDiff(iffd=iffd, dffd=dffd, offd=offd, block_size=16384)
differ_restore.start()
GPLv3