fuxino/simple_backup
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

Compare commits

base: fuxino:master
Branches Tags
fuxino:master
fuxino:development
fuxino:4.1.6
fuxino:4.1.5
fuxino:4.1.4
fuxino:4.1.3
fuxino:4.1.2
fuxino:4.1.1
fuxino:4.1.0
fuxino:4.0.2
fuxino:4.0.1
fuxino:4.0.0
fuxino:3.5.0
fuxino:3.4.1
fuxino:3.4.0
fuxino:3.3.0
fuxino:3.2.8
fuxino:3.2.7
fuxino:3.2.6
fuxino:3.2.5
fuxino:3.2.4
fuxino:3.2.3
fuxino:3.2.2
fuxino:3.2.1
fuxino:3.2.0
fuxino:3.1.2
fuxino:3.1.1
fuxino:3.1.0
fuxino:3.0.2
fuxino:3.0.1
fuxino:3.0.0
fuxino:2.1.1
fuxino:2.1.0
fuxino:2.0.1
fuxino:2.0.0
fuxino:1.4.2
fuxino:1.4.1
fuxino:1.4
fuxino:1.3.4
fuxino:1.3.3
fuxino:1.3.2
fuxino:1.3.1
fuxino:1.3
fuxino:1.2.3b
fuxino:1.2.3a
fuxino:1.2.3
fuxino:1.2.2
fuxino:1.2.1
fuxino:1.2
fuxino:1.1
..
compare: fuxino:3.4.1
Branches Tags
fuxino:master
fuxino:development
fuxino:4.1.6
fuxino:4.1.5
fuxino:4.1.4
fuxino:4.1.3
fuxino:4.1.2
fuxino:4.1.1
fuxino:4.1.0
fuxino:4.0.2
fuxino:4.0.1
fuxino:4.0.0
fuxino:3.5.0
fuxino:3.4.1
fuxino:3.4.0
fuxino:3.3.0
fuxino:3.2.8
fuxino:3.2.7
fuxino:3.2.6
fuxino:3.2.5
fuxino:3.2.4
fuxino:3.2.3
fuxino:3.2.2
fuxino:3.2.1
fuxino:3.2.0
fuxino:3.1.2
fuxino:3.1.1
fuxino:3.1.0
fuxino:3.0.2
fuxino:3.0.1
fuxino:3.0.0
fuxino:2.1.1
fuxino:2.1.0
fuxino:2.0.1
fuxino:2.0.0
fuxino:1.4.2
fuxino:1.4.1
fuxino:1.4
fuxino:1.3.4
fuxino:1.3.3
fuxino:1.3.2
fuxino:1.3.1
fuxino:1.3
fuxino:1.2.3b
fuxino:1.2.3a
fuxino:1.2.3
fuxino:1.2.2
fuxino:1.2.1
fuxino:1.2
fuxino:1.1

No commits in common. "master" and "3.4.1" have entirely different histories.
master ... 3.4.1

7 changed files with 177 additions and 883 deletions
Show Stats Expand all files Collapse all files

40
PKGBUILD
View File

@ -1,40 +0,0 @@
# PKGBUILD
# Maintainer: Daniele Fucini <dfucini [at] gmail [dot] com>
pkgname=simple_backup
pkgdesc='Simple backup script that uses rsync to copy files'
pkgver=4.1.6
pkgrel=1
url="https://git.shouldnt.work/fuxino/${pkgname}"
arch=('any')
license=('GPL-3.0-or-later')
makedepends=('git'
'python-setuptools'
'python-build'
'python-installer'
'python-wheel')
depends=('python>=3.10'
'rsync'
'python-dotenv')
optdepends=('python-systemd: use systemd log'
'python-dbus: for desktop notifications'
'python-paramiko: for remote backup through ssh')
conflicts=('simple_backup-git')
source=(git+${url}?signed#tag=${pkgver})
validpgpkeys=('7E12BC1FF3B6EDB2CD8053EB981A2B2A3BBF5514')
sha256sums=('b3b29d9e2e1b7b949e95674d9a401e8eeb0d5f898e8450473dce94f799ee9df3')
build()
{
cd ${srcdir}/${pkgname}
python -m build --wheel --no-isolation
}
package()
{
cd ${srcdir}/${pkgname}
python -m installer --destdir=${pkgdir} dist/*.whl
install -Dm644 ${pkgname}/${pkgname}.conf ${pkgdir}/usr/share/doc/${pkgname}/${pkgname}.conf
install -Dm644 man/${pkgname}.1 ${pkgdir}/usr/share/man/man1/${pkgname}.1
}

20
README.md
View File

@ -49,23 +49,3 @@ For Arch Linux and Arch-based distros, two packages are available in the AUR (au
- **simple_backup** for the release version
- **simple_backup-git** for the git version
## Remote backup
> **Warning**
> This feature is experimental
It's possible to use a remote server as destination for the backup. Just use the --username (or -u) and --host arguments (or set them in the configuration file).
For this to work, rsync must be installed on the server too.
### Server authentication
The best way to handle the authentication is to have an ssh agent running on your system, otherwise if a passphrase is necessary to unlock the ssh key, it will be necessary to enter it more than once.
If needed, it's possible to specify the ssh key location with the --keyfile argument or in the configuration file.
To be able to connect to the user's ssh agent when running simple_backup with sudo, make sure to preserve the SSH_AUTH_SOCK environment variable. For example:
```bash
sudo --preserve-env=SSH_AUTH_SOCK -s simple_backup [options]
```
or by editing the sudoers file.
If SSH key authentication is not available, password authentication will be used instead.
Check the man page for more details.

162
man/simple_backup.1
View File

@ -1,13 +1,13 @@
.TH SIMPLE_BACKUP 1 2025-03-30 SIMPLE_BACKUP 4.1.6
.TH SIMPLE_BACKUP 1 2023-06-01 SIMPLE_BACKUP 3.2.6
.SH NAME
simple_backup \- Backup files and folders using rsync
.SH SYNOPSIS
.B simple_backup
.BR simple_backup
\-h, \-\-help
.PD 0
.P
.PD
.B simple_backup
.BR simple_backup
[\-c, \-\-config FILE]
[\-i, \-\-input INPUT [INPUT...]]
[\-o, \-\-output DIR]
@ -16,177 +16,71 @@ simple_backup \- Backup files and folders using rsync
.PD
.RS 14 [\-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]]
[\-k, \-\-keep N]
[\-\-ssh\-host HOSTNAME]
[\-\-ssh\-user USERNAME]
[\-\-keyfile FILE]
.PD 0
.P
.PD
[\-s, \-\-checksum]
[\-z, \-\-compress]
[\-\-remove\-before\-backup]
.RE
.SH DESCRIPTION
.B simple_backup
is a python script for performing backup of files and folders.
.P
It uses rsync to copy the files to the specified location. Parameters for the backup such as
input files/directories, output location and files or folders to exclude can be specified
in a configuration file (default location $HOME/.config/simple_backup/simple_backup.conf)
or directly on the command line.
.P
.BR simple_backup
is a python script for performing backup of files and folders. It uses rsync to copy the files to the specified location.
Parameters for the backup such as input files/directories, output location and files or folders to exclude can be specified
in a configuration file (default location $HOME/.config/simple_backup/simple_backup.conf) or directly on the command line.
Parameters specified on the command line will override those in the configuration file.
.SH OPTIONS
.TP
.B \-h, \-\-help
Print a short help message and exit.
Print a short help message and exit
.TP
.B \-c, \-\-config FILE
Specify the configuration file, useful to specify a different one from the default.
.TP
.B \-i, \-\-input INPUT [INPUT...]
Specify the files and directories to backup. Multiple inputs can be specified, just separate
them with a space. If filenames or paths contain spaces, don't forget to escape them,
or to use single or double quotes around them.
Specify the files and directories to backup. Multiple inputs can be specified, just separate them with a space.
If filenames or paths contain spaces, don't forget to escape them, or to use single or double quotes around them.
.TP
.B \-o, \-\-output DIR
Specify the directory where the files will be copied. The program will automatically
create a subdirectory called \(aqsimple_backup\(aq (if it does not already exist) and
inside this directory the actual backup directory (using the current date and time).
Specify the directory where the files will be copied. The program will automatically create a subdirectory called
\(aqsimple_backup\(aq (if it does not already exist) and inside this directory the actual backup directory (using
the current date and time)
.TP
.B \-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]]
Specify files, directories or patterns to exclude from the backup. Matching files and directories
will not be copied. Multiple elements can be specified, in the same way as for the \-\-input option.
Specify files, directories or patterns to exclude from the backup. Matching files and directories will not be copied.
Multiple elements can be specified, in the same way as for the \-\-input option
.TP
.B \-k, \-\-keep N
Specify how many old backups (so excluding the current one) will be kept. The default behavior
is to keep them all (same as N=\-1).
.TP
.B \-u, \-\-user USERNAME
Explicitly specify the user running the backup (in case it is needed for home directory expansion).
.TP
.B \-\-ssh\-host HOSTNAME
Hostname of the server where to copy the files in case of remote backup through SSH.
.TP
.B \-\-ssh\-user USERNAME
Username for connecting to the server in case of remote backup.
.TP
.B \-\-keyfile FILE
Location of the SSH key for server authentication.
Specify how many old backups (so excluding the current one) will be kept. The default behavior is to keep them all
(same as N=\-1)
.TP
.B \-s, \-\-checksums
Same as rsync option \(aq\-\-checksum\(aq, use checksums instead of mod\-time and size
to skip files.
.TP
.B \-z, \-\-compress
Compress data during transfer (rsync \(aq\-\-compress\(aq option). Useful for remote backup
if saving bandwith is needed.
Same as rsync option \(aq\-\-checksum\(aq, use checksums instead of mod\-time and size to skip files.
.TP
.B \-\-remove\-before\-backup
Remove old backups (if necessary) before creating the new backup. Useful to free some space
before performing the backup.
Remove old backups (if necessary) before creating the new backup. Useful to free some space before performing the backup.
Default behavior is to remove old backups after successfully completing the backup.
.TP
.B \-\-no\-syslog
Don't use systemd journal for logging.
.TP
.B \-\-rsync\-options OPTIONS [OPTION...]
By default, the following rsync options are used:
.RS
.P
\-a \-r \-v \-h \-s \-H \-X
.P
Using \-\-rsync\-options it is possible to manually select which options to use. Supported values are the following:
.P
\-a, \-l, \-p, \-t, \-g, \-o, \-c, \-h, \-D, \-H, \-X, \-s
.P
Options \-r and \-v are used in any case. Not that options must be specified without dash (\-), for example:
.P
.EX
simple_backup \-\-rsync\-options a l p
.EE
.P
Check
.BR rsync (1)
for details about the options.
.RE
.TP
.B \-\-remote\-sudo
Run rsync on the remote server with sudo. This is needed if you want to preserve the owner of the files/folders to be copied (rsync \-\-owner option). For this to work the user used to login to the server obviously need to be allowed to use sudo. In addition, the user need to be able to run rsync with sudo without a password. To do this, /etc/sudoers on the server need to be edited adding a line like this one:
.RS
.P
<username> ALL=NOPASSWD:<path/to/rsync>
.P
To be able to remove old backups generated with \-\-remote\-sudo (see \-\-keep option), also
.BR rm (1)
needs to be allowed to run without password in the same way.
.RE
.TP
.B \-\-numeric\-ids
Use rsync \-\-numeric\-ids option. This causes rsync to use numeric uid/gid instead of trying to map uid/gid names from the local machine to the server.
Don't use systemd journal for logging
.SH CONFIGURATION
An example configuration file is provided at \(aq/usr/share/doc/simple_backup/simple_backup.conf\(aq.
Copy it to the default location ($HOME/.config/simple_backup) and edit it as needed.
.SH REMOTE BACKUP
It is possible to choose a directory on a remote server as destination for the backup. The files
are copied by rsync through SSH. Server hostname and username must be specified, either in the
configuration file, or on the command line (\(aq\-\-ssh\-host\(aq and \(aq\-\-ssh\-user\(aq options).
.SS AUTHENTICATION
For authentication, it is possible to use SSH key or password.
.P
When using SSH key, the best way to connect to the server is to have an SSH agent running.
Otherwise, if the SSH key is encrypted, it will be necessary to enter the passphrase more
than once. It is possible to specify the SSH key to use with the option \(aq\-\-keyfile\(aq,
if necessary.
.P
When running
.B simple_backup
with
.B sudo,
in order to connect to the user\(aqs SSH agent it is necessary to preserve the \(aqSSH_AUTH_SOCK\(aq environment variable, for example:
.P
.EX
sudo --preserve-env=SSH_AUTH_SOCK -s simple_backup [options]
.EE
.P
It is also possible to make this permanent by editing the
.B sudoers
file (see
.BR sudoers (5))
.P
If SSH key authentication is not available, password authentication will be used instead.
Note that in this case
.B sshpass
(if available) will be used to send the password to rsync, to avoid prompting the user for
the password multiple
times. This can pose some security risks, see
.BR sshpass (1)
for details. For this reason, use SSH key authentication if possible.
An example configuration file is provided at \(aq/etc/simple_backup/simple_backup.conf\(aq. Copy it to the default location
($HOME/.config/simple_backup) and edit it as needed.
.SH EXIT STATUS
.TP
.B 0
The backup was completed without errors.
The backup was completed without errors
.TP
.B 1
No valid inputs selected for backup.
No valid inputs selected for backup
.TP
.B 2
Backup failed because output directory for storing the backup does not exist.
Backup failed because output directory for storing the backup does not exist
.TP
.B 3
Permission denied to access the output directory.
Permission denied to access the output directory
.TP
.B 4
rsync error (rsync returned a non-zero value).
.TP
.B 5
SSH connection failed.
.TP
.B 6
Bad configuration file.
rsync error (rsync returned a non-zero value)
.SH SEE ALSO
.BR rsync (1)
.SH AUTHORS
.MT https://git.shouldnt.work/fuxino
.MT https://github.com/Fuxino
Daniele Fucini
.ME

13
setup.cfg
View File

@ -6,22 +6,23 @@ long_description = file: README.md
author = Daniele Fucini
author_email = dfucini@gmail.com
license = GPL3
url = https://git.shouldnt.work/fuxino
url = https://github.com/Fuxino/simple_backup
classifiers =
Development Status :: 4 - Beta
Development Status :: 5 - Production/Stable
Environment :: Console
License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Natural Language :: English
Operating System :: POSIX :: Linux
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: 3.13
Topic :: System :: Archiving :: Backup
[options]
packages = simple_backup
python_requires = >=3.10
python_requires = >=3.7
install_requires =
python-dotenv
@ -30,8 +31,6 @@ JOURNAL =
systemd-python
NOTIFICATIONS =
dbus-python
REMOTE =
paramiko
[options.entry_points]
console_scripts =

2
simple_backup/__init__.py
View File

@ -1,3 +1,3 @@
"""Init."""
__version__ = '4.1.6'
__version__ = '3.4.1'

12
simple_backup/simple_backup.conf
View File

@ -1,8 +1,8 @@
# Example config file for simple_backup
[backup]
[default]
# Files and directories to backup. Multiple items can be separated using a comma (','). It is possible to use wildcards (i.e. '*' to match multiple characters and '~' for the user's home directory).
inputs=/home/user
inputs=/home/my_home,/etc
# Output directory.
backup_dir=/media/Backup
@ -12,11 +12,3 @@ exclude=*.bak
# Number of old backups (i.e. excluding the one that's being created) to keep (use -1 to keep all)
keep=-1
# Uncomment the following section to enable backup to remote server through ssh
# [server]
# ssh_host=
# ssh_user=
# ssh_keyfile=
# remote_sudo=
# numeric_ids=

735
simple_backup/simple_backup.py
View File

@ -1,24 +1,10 @@
#!/usr/bin/env python3
"""
A simple python script that calls rsync to perform a backup
Parameters can be specified on the command line or using a configuration file
Backup to a remote server is also supported (experimental)
Classes:
MyFormatter
Backup
"""
#!/usr/bin/python3
# Import libraries
import sys
import os
from typing import Callable, List, Optional, ParamSpec, TypeVar, Union
import warnings
from functools import wraps
from shutil import rmtree, which
import shlex
from shutil import rmtree
import argparse
import configparser
import logging
@ -27,19 +13,10 @@ from timeit import default_timer
from subprocess import Popen, PIPE, STDOUT
from datetime import datetime
from tempfile import mkstemp
from getpass import GetPassWarning, getpass
from glob import glob
from dotenv import load_dotenv
warnings.filterwarnings('error')
try:
import paramiko
from paramiko import RSAKey, Ed25519Key, ECDSAKey, DSSKey
except ImportError:
pass
try:
from systemd import journal
except ImportError:
@ -52,6 +29,15 @@ except ImportError:
load_dotenv()
euid = os.geteuid()
if euid == 0:
user = os.getenv('SUDO_USER')
homedir = os.path.expanduser(f'~{user}')
else:
user = os.getenv('USER')
homedir = os.getenv('HOME')
logging.getLogger().setLevel(logging.DEBUG)
logger = logging.getLogger(os.path.basename(__file__))
c_handler = StreamHandler()
@ -68,131 +54,56 @@ if journal:
j_handler.setFormatter(j_format)
logger.addHandler(j_handler)
P = ParamSpec('P')
R = TypeVar('R')
def timing(func: Callable[P, R]) -> Callable[P, R]:
"""Decorator to measure execution time of a function
Parameters:
func: Function to decorate
"""
def timing(_logger):
def decorator_timing(func):
@wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
def wrapper_timing(*args, **kwargs):
start = default_timer()
value = func(*args, **kwargs)
end = default_timer()
logger.info('Elapsed time: %.3f seconds', end - start)
_logger.info(f'Elapsed time: {end - start:.3f} seconds')
return value
return wrapper
return wrapper_timing
return decorator_timing
class MyFormatter(argparse.RawTextHelpFormatter, argparse.ArgumentDefaultsHelpFormatter):
"""Custom format for argparse help text"""
pass
class Backup:
"""Main class defining parameters and functions for performing backup
Attributes:
inputs: list
Files and folders that will be backup up
output: str
Path where the backup will be saved
exclude: list
List of files/folders/patterns to exclude from backup
options: str
String representing main backup options for rsync
keep: int
Number of old backup to preserve
ssh_host: str
Hostname of server (for remote backup)
ssh_user: str
Username for server login (for remote backup)
ssh_keyfile: str
Location of ssh key
remote_sudo: bool
Run remote rsync with sudo
remove_before: bool
Indicate if removing old backups will be performed before copying files
Methods:
check_params():
Check if parameters for the backup are valid
define_backup_dir():
Define the actual backup dir
remove_old_backups():
Remove old backups if there are more than indicated by 'keep'
find_last_backup():
Get path of last backup (from last_backup symlink) for rsync --link-dest
run():
Perform the backup
"""
def __init__(self, inputs: List[str], output: str, exclude: List[str], keep: int, options: str,
ssh_host: Optional[str] = None, ssh_user: Optional[str] = None, ssh_keyfile: Optional[str] = None,
remote_sudo: bool = False, remove_before: bool = False, verbose: bool = False) -> None:
def __init__(self, inputs, output, exclude, keep, options, remove_before=False):
self.inputs = inputs
self.output = output
self.exclude = exclude
self.options = options
self.keep = keep
self.ssh_host = ssh_host
self.ssh_user = ssh_user
self.ssh_keyfile = ssh_keyfile
self.remote_sudo = remote_sudo
self._remove_before = remove_before
self._verbose = verbose
self.remove_before = remove_before
self._last_backup = ''
self._server = ''
self._output_dir = ''
self._inputs_path = ''
self._exclude_path = ''
self._remote = False
self._ssh = None
self._password_auth = False
self._password = None
def check_params(self, homedir: str = '') -> int:
"""Check if parameters for the backup are valid"""
self._err_flag = False
def check_params(self):
if self.inputs is None or len(self.inputs) == 0:
logger.info(
'No existing files or directories specified for backup. Nothing to do')
logger.info('No existing files or directories specified for backup. Nothing to do')
return 1
if self.output is None:
logger.critical(
'No output path specified. Use -o argument or specify output path in configuration file')
logger.critical('No output path specified. Use -o argument or specify output path in configuration file')
return 2
if self.ssh_host is not None and self.ssh_user is not None:
self._remote = True
if self._remote:
self._ssh = self._ssh_connect(homedir)
if self._ssh is None:
return 5
_, stdout, _ = self._ssh.exec_command(
f'if [ -d "{self.output}" ]; then echo "ok"; fi')
output = stdout.read().decode('utf-8').strip()
if output != 'ok':
logger.critical('Output path for backup does not exist')
return 2
else:
if not os.path.isdir(self.output):
logger.critical('Output path for backup does not exist')
@ -206,63 +117,19 @@ class Backup:
return 0
# Function to create the actual backup directory
def define_backup_dir(self) -> None:
"""Define the actual backup dir"""
def create_backup_dir(self):
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
self._output_dir = f'{self.output}/simple_backup/{now}'
if self._remote:
self._server = f'{self.ssh_user}@{self.ssh_host}:'
os.makedirs(self._output_dir, exist_ok=True)
def remove_old_backups(self) -> None:
"""Remove old backups if there are more than indicated by 'keep'"""
if self._remote:
assert self._ssh is not None
_, stdout, _ = self._ssh.exec_command(
f'ls {self.output}/simple_backup')
dirs = stdout.read().decode('utf-8').strip().split('\n')
n_backup = len(dirs)
if not self._remove_before:
n_backup -= 1
count = 0
if n_backup > self.keep:
logger.info('Removing old backups...')
dirs.sort()
for i in range(n_backup - self.keep):
if self.remote_sudo:
_, _, stderr = self._ssh.exec_command(
f'sudo rm -r "{self.output}/simple_backup/{dirs[i]}"')
else:
_, _, stderr = self._ssh.exec_command(
f'rm -r "{self.output}/simple_backup/{dirs[i]}"')
err = stderr.read().decode('utf-8').strip().split('\n')[0]
if err != '':
logger.error(
'Error while removing backup %s.', {dirs[i]})
logger.error(err)
else:
count += 1
else:
def remove_old_backups(self):
try:
dirs = os.listdir(f'{self.output}/simple_backup')
except FileNotFoundError:
return
n_backup = len(dirs)
if not self._remove_before:
n_backup -= 1
n_backup = len(dirs) - 1
count = 0
if n_backup > self.keep:
@ -274,47 +141,27 @@ class Backup:
rmtree(f'{self.output}/simple_backup/{dirs[i]}')
count += 1
except FileNotFoundError:
logger.error(
'Error while removing backup %s. Directory not found', dirs[i])
logger.error(f'Error while removing backup {dirs[i]}. Directory not found')
except PermissionError:
logger.error(
'Error while removing backup %s. Permission denied', dirs[i])
logger.error(f'Error while removing backup {dirs[i]}. Permission denied')
if count == 1:
logger.info('Removed %d backup', count)
elif count > 1:
logger.info('Removed %d backups', count)
def find_last_backup(self) -> None:
"""Get path of last backup (from last_backup symlink) for rsync --link-dest"""
if self._remote:
if self._ssh is None:
logger.critical('SSH connection to server failed')
sys.exit(5)
_, stdout, _ = self._ssh.exec_command(
f'find {self.output}/simple_backup/ -mindepth 1 -maxdepth 1 -type d | sort')
output = stdout.read().decode('utf-8').strip().split('\n')
if output[-1] != '':
self._last_backup = output[-1]
else:
logger.info('No previous backups available')
logger.info(f'Removed {count} backup')
else:
logger.info(f'Removed {count} backups')
def find_last_backup(self):
try:
dirs = sorted([f.path for f in os.scandir(
f'{self.output}/simple_backup') if f.is_dir(follow_symlinks=False)])
dirs = sorted([f.path for f in os.scandir(f'{self.output}/simple_backup') if f.is_dir(follow_symlinks=False)])
except FileNotFoundError:
logger.info('No previous backups available')
return
except PermissionError:
logger.critical(
'Cannot access the backup directory. Permission denied')
logger.critical('Cannot access the backup directory. Permission denied')
try:
_notify('Backup failed (check log for details)')
notify('Backup failed (check log for details)')
except NameError:
pass
@ -325,197 +172,36 @@ class Backup:
except IndexError:
logger.info('No previous backups available')
def _ssh_connect(self, homedir: str = '') -> paramiko.client.SSHClient:
try:
ssh = paramiko.SSHClient()
except NameError:
logger.error('Install paramiko for ssh support')
return None
try:
ssh.load_host_keys(filename=f'{homedir}/.ssh/known_hosts')
except FileNotFoundError:
logger.warning('Cannot find file %s/.ssh/known_hosts', homedir)
ssh.set_missing_host_key_policy(paramiko.WarningPolicy())
try:
ssh.connect(self.ssh_host, username=self.ssh_user)
return ssh
except UserWarning:
k = input(
f'Unknown key for host {self.ssh_host}. Continue anyway? (Y/N) ')
if k[0].upper() == 'Y':
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
else:
return None
except paramiko.BadHostKeyException as e:
logger.critical('Can\'t connect to the server.')
logger.critical(e)
return None
except paramiko.SSHException:
pass
try:
ssh.connect(self.ssh_host, username=self.ssh_user)
return ssh
except paramiko.SSHException:
pass
if self.ssh_keyfile is None:
try:
password = getpass(
f'{self.ssh_user}@{self.ssh_host}\'s password: ')
ssh.connect(self.ssh_host, username=self.ssh_user,
password=password)
self._password_auth = True
os.environ['SSHPASS'] = password
return ssh
except GetPassWarning as e:
logger.critical('Unable to get password')
logger.critical(e)
return None
except paramiko.SSHException as e:
logger.critical('Can\'t connect to the server.')
logger.critical(e)
return None
pkey = None
try:
pkey = RSAKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
password = getpass(
f'Enter passwphrase for key \'{self.ssh_keyfile}\': ')
try:
pkey = RSAKey.from_private_key_file(self.ssh_keyfile, password)
except paramiko.SSHException:
pass
if pkey is None:
try:
pkey = Ed25519Key.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
try:
pkey = Ed25519Key.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException:
pass
if pkey is None:
try:
pkey = ECDSAKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
try:
pkey = ECDSAKey.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException:
pass
if pkey is None:
try:
pkey = DSSKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
try:
pkey = DSSKey.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException:
pass
try:
ssh.connect(self.ssh_host, username=self.ssh_user, pkey=pkey)
except paramiko.SSHException:
logger.critical('SSH connection to server failed')
return None
return ssh
def _returncode_log(self, returncode: int) -> None:
match returncode:
case 2:
logger.error(
'Rsync error (return code 2) - Protocol incompatibility')
case 3:
logger.error(
'Rsync error (return code 3) - Errors selecting input/output files, dirs')
case 4:
logger.error(
'Rsync error (return code 4) - Requested action not supported')
case 5:
logger.error(
'Rsync error (return code 5) - Error starting client-server protocol')
case 10:
logger.error(
'Rsync error (return code 10) - Error in socket I/O')
case 11:
logger.error(
'Rsync error (return code 11) - Error in file I/O')
case 12:
logger.error(
'Rsync error (return code 12) - Error in rsync protocol data stream')
case 22:
logger.error(
'Rsync error (return code 22) - Error allocating core memory buffers')
case 23:
logger.warning(
'Rsync error (return code 23) - Partial transfer due to error')
case 24:
logger.warning(
'Rsync error (return code 24) - Partial transfer due to vanished source files')
case 30:
logger.error(
'Rsync error (return code 30) - Timeout in data send/receive')
case 35:
logger.error(
'Rsync error (return code 35) - Timeout waiting for daemon connection')
case _:
logger.error(
'Rsync error (return code %d) - Check rsync(1) for details', returncode)
# Function to read configuration file
@timing
def run(self) -> int:
"""Perform the backup"""
@timing(logger)
def run(self):
logger.info('Starting backup...')
try:
_notify('Starting backup...')
notify('Starting backup...')
except NameError:
pass
self.define_backup_dir()
self.find_last_backup()
self.create_backup_dir()
_, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True)
count = 0
with open(self._inputs_path, 'w', encoding='utf-8') as fp:
with open(self._inputs_path, 'w') as fp:
for i in self.inputs:
if not os.path.exists(i):
logger.warning('Input %s not found. Skipping', i)
logger.warning(f'Input {i} not found. Skipping')
else:
fp.write(i)
fp.write('\n')
count += 1
if count == 0:
logger.info(
'No existing files or directories specified for backup. Nothing to do')
logger.info('No existing files or directories specified for backup. Nothing to do')
try:
_notify('Backup finished. No files copied')
notify('Backup finished. No files copied')
except NameError:
pass
@ -523,391 +209,174 @@ class Backup:
_, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True)
with open(self._exclude_path, 'w', encoding='utf-8') as fp:
with open(self._exclude_path, 'w') as fp:
if self.exclude is not None:
for e in self.exclude:
fp.write(e)
fp.write('\n')
if self.keep != -1 and self._remove_before:
if self.keep != -1 and self.remove_before:
self.remove_old_backups()
logger.info('Copying files. This may take a long time...')
if self._last_backup == '':
rsync = f'/usr/bin/rsync {self.options} --exclude-from={self._exclude_path} ' +\
f'--files-from={self._inputs_path} / "{self._server}{self._output_dir}"'
rsync = f'rsync {self.options} --exclude-from={self._exclude_path} ' +\
f'--files-from={self._inputs_path} / "{self._output_dir}" ' +\
'--ignore-missing-args'
else:
rsync = f'/usr/bin/rsync {self.options} --link-dest="{self._last_backup}" --exclude-from=' +\
f'{self._exclude_path} --files-from={self._inputs_path} / "{self._server}{self._output_dir}"'
rsync = f'rsync {self.options} --link-dest="{self._last_backup}" --exclude-from=' +\
f'{self._exclude_path} --files-from={self._inputs_path} / "{self._output_dir}" ' +\
'--ignore-missing-args'
euid = os.geteuid()
if euid == 0 and self.ssh_keyfile is not None:
rsync = f'{rsync} -e \'ssh -i {self.ssh_keyfile} -o StrictHostKeyChecking=no\''
elif self._password_auth and which('sshpass'):
rsync = f'{rsync} -e \'sshpass -e ssh -l {self.ssh_user} -o StrictHostKeyChecking=no\''
else:
rsync = f'{rsync} -e \'ssh -o StrictHostKeyChecking=no\''
if self._remote and self.remote_sudo:
rsync = f'{rsync} --rsync-path="sudo rsync"'
args = shlex.split(rsync)
with Popen(args, stdin=PIPE, stdout=PIPE, stderr=STDOUT, shell=False) as p:
output: Union[bytes, List[str]]
p = Popen(rsync, stdout=PIPE, stderr=STDOUT, shell=True)
output, _ = p.communicate()
try:
del os.environ['SSHPASS']
except KeyError:
pass
returncode = p.returncode
if p.returncode != 0:
self._err_flag = True
output = output.decode("utf-8").split('\n')
if returncode == 0:
if self._verbose:
logger.info('rsync: %s', output)
else:
logger.info('rsync: %s', output[-3])
logger.info('rsync: %s', output[-2])
else:
self._returncode_log(returncode)
logger.info(f'rsync: {output[-3]}')
logger.info(f'rsync: {output[-2]}')
if self._verbose:
if returncode in [23, 24]:
logger.warning(output)
else:
logger.error(output)
if self.keep != -1 and not self._remove_before:
if self.keep != -1 and not self.remove_before:
self.remove_old_backups()
os.remove(self._inputs_path)
os.remove(self._exclude_path)
if self._remote:
assert self._ssh is not None
_, stdout, _ = self._ssh.exec_command(
f'if [ -d "{self._output_dir}" ]; then echo "ok"; fi')
output = stdout.read().decode('utf-8').strip()
if output == 'ok':
logger.info('Backup completed')
try:
_notify('Backup completed')
except NameError:
pass
else:
logger.error('Backup failed')
if self._err_flag:
logger.warning('Some errors occurred (check log for details)')
try:
_notify('Backup failed (check log for details)')
except NameError:
pass
if self._ssh:
self._ssh.close()
else:
if returncode != 0:
logger.error(
'Some errors occurred while performing the backup')
try:
_notify(
'Some errors occurred while performing the backup. Check log for details')
notify('Backup finished with errors (check log for details)')
except NameError:
pass
return 4
logger.info('Backup completed')
else:
try:
_notify('Backup completed')
notify('Backup finished')
except NameError:
pass
return 0
def _parse_arguments() -> argparse.Namespace:
euid = os.geteuid()
if euid == 0:
user = os.getenv('SUDO_USER')
else:
user = os.getenv('USER')
homedir = os.path.expanduser(f'~{user}')
def _parse_arguments():
parser = argparse.ArgumentParser(prog='simple_backup',
description='Simple backup script written in Python that uses rsync to copy files',
epilog='See simple_backup(1) manpage for full documentation',
epilog='Report bugs to dfucini<at>gmail<dot>com',
formatter_class=MyFormatter)
parser.add_argument('-v', '--verbose', action='store_true',
help='More verbose output')
parser.add_argument('-c', '--config', default=f'{homedir}/.config/simple_backup/simple_backup.conf',
help='Specify location of configuration file')
parser.add_argument('-i', '--inputs', nargs='+',
help='Paths/files to backup')
parser.add_argument(
'-o', '--output', help='Output directory for the backup')
parser.add_argument('-e', '--exclude', nargs='+',
help='Files/directories/patterns to exclude from the backup')
parser.add_argument('-k', '--keep', type=int,
help='Number of old backups to keep')
parser.add_argument(
'-u', '--user', help='Explicitly specify the user running the backup')
parser.add_argument('-i', '--input', nargs='+', help='Paths/files to backup')
parser.add_argument('-o', '--output', help='Output directory for the backup')
parser.add_argument('-e', '--exclude', nargs='+', help='Files/directories/patterns to exclude from the backup')
parser.add_argument('-k', '--keep', type=int, help='Number of old backups to keep')
parser.add_argument('-s', '--checksum', action='store_true',
help='Use checksum rsync option to compare files')
parser.add_argument(
'--ssh-host', help='Server hostname (for remote backup)')
parser.add_argument(
'--ssh-user', help='Username to connect to server (for remote backup)')
parser.add_argument('--keyfile', help='SSH key location')
parser.add_argument('-z', '--compress', action='store_true',
help='Compress data during the transfer')
help='Use checksum rsync option to compare files (MUCH SLOWER)')
parser.add_argument('--remove-before-backup', action='store_true',
help='Remove old backups before executing the backup, instead of after')
parser.add_argument('--no-syslog', action='store_true',
help='Disable systemd journal logging')
parser.add_argument('--rsync-options', nargs='+',
choices=['a', 'l', 'p', 't', 'g', 'o',
'c', 'h', 's', 'D', 'H', 'X'],
help='Specify options for rsync')
parser.add_argument('--remote-sudo', action='store_true',
help='Run rsync on remote server with sudo if allowed')
parser.add_argument('--numeric-ids', action='store_true',
help='Use rsync \'--numeric-ids\' option (don\'t map uid/gid values by name)')
parser.add_argument('--no-syslog', action='store_true', help='Disable systemd journal logging')
args = parser.parse_args()
return args
def _expand_inputs(inputs, user: Optional[str] = None):
def _expand_inputs(inputs):
expanded_inputs = []
for i in inputs:
if i == '':
continue
if user is not None:
i_ex = glob(os.path.expanduser(i.replace('~', f'~{user}')))
else:
i_ex = glob(i)
if '~' in i:
logger.warning('Cannot expand \'~\'. No user specified')
if len(i_ex) == 0:
logger.warning(
'No file or directory matching input %s. Skipping...', i)
logger.warning(f'No file or directory matching input {i}. Skipping...')
else:
expanded_inputs.extend(i_ex)
return expanded_inputs
def _read_config(config_file, user: Optional[str] = None):
config_args = {'inputs': None,
'output': None,
'exclude': None,
'keep': -1,
'ssh_host': None,
'ssh_user': None,
'ssh_keyfile': None,
'remote_sudo': False,
'numeric_ids': False}
def _read_config(config_file):
if not os.path.isfile(config_file):
if user is not None:
logger.warning('Config file %s does not exist', config_file)
else:
logger.warning(
'User not specified. Can\'t read configuration file')
logger.warning(f'Config file {config_file} does not exist')
return config_args
return None, None, None, None
config = configparser.ConfigParser()
config.read(config_file)
section = 'backup'
# Allow compatibility with previous version of config file
try:
inputs = config.get(section, 'inputs')
except configparser.NoSectionError:
section = 'default'
inputs = config.get(section, 'inputs')
inputs = config.get('default', 'inputs')
inputs = inputs.split(',')
inputs = _expand_inputs(inputs, user)
inputs = _expand_inputs(inputs)
inputs = list(set(inputs))
config_args['inputs'] = inputs
output = config.get(section, 'backup_dir')
if user is not None:
output = config.get('default', 'backup_dir')
output = os.path.expanduser(output.replace('~', f'~{user}'))
elif user is None and '~' in output:
logger.warning('Cannot expand \'~\', no user specified')
config_args['output'] = output
try:
exclude = config.get(section, 'exclude')
exclude = config.get('default', 'exclude')
exclude = exclude.split(',')
except configparser.NoOptionError:
exclude = []
keep = config.getint('default', 'keep')
config_args['exclude'] = exclude
try:
keep = config.getint(section, 'keep')
except configparser.NoOptionError:
keep = -1
config_args['keep'] = keep
try:
ssh_host = config.get('server', 'ssh_host')
ssh_user = config.get('server', 'ssh_user')
except (configparser.NoSectionError, configparser.NoOptionError):
ssh_host = None
ssh_user = None
config_args['ssh_host'] = ssh_host
config_args['ssh_user'] = ssh_user
try:
ssh_keyfile = config.get('server', 'ssh_keyfile')
except (configparser.NoSectionError, configparser.NoOptionError):
ssh_keyfile = None
config_args['ssh_keyfile'] = ssh_keyfile
try:
remote_sudo = config.getboolean('server', 'remote_sudo')
except (configparser.NoSectionError, configparser.NoOptionError):
remote_sudo = False
config_args['remote_sudo'] = remote_sudo
try:
numeric_ids = config.getboolean('server', 'numeric_ids')
except (configparser.NoSectionError, configparser.NoOptionError):
numeric_ids = False
config_args['numeric_ids'] = numeric_ids
return config_args
return inputs, output, exclude, keep
def _notify(text: str) -> None:
def notify(text):
euid = os.geteuid()
if euid == 0:
uid = os.getenv('SUDO_UID')
else:
uid = euid
if uid is None:
return
uid = os.geteuid()
os.seteuid(int(uid))
os.environ['DBUS_SESSION_BUS_ADDRESS'] = f'unix:path=/run/user/{uid}/bus'
obj = dbus.SessionBus().get_object('org.freedesktop.Notifications',
'/org/freedesktop/Notifications')
obj = dbus.SessionBus().get_object('org.freedesktop.Notifications', '/org/freedesktop/Notifications')
obj = dbus.Interface(obj, 'org.freedesktop.Notifications')
obj.Notify('', 0, '', 'simple_backup', text, [], {'urgency': 1}, 10000)
os.seteuid(int(euid))
def simple_backup() -> int:
"""Main"""
def simple_backup():
args = _parse_arguments()
if args.user:
user = args.user
homedir = os.path.expanduser(f'~{user}')
else:
euid = os.geteuid()
if euid == 0:
user = os.getenv('SUDO_USER')
if user is not None:
homedir = os.path.expanduser(f'~{user}')
else:
logger.warning(
'Failed to detect user. You can use -u/--user parameter to manually specify it')
homedir = None
else:
user = os.getenv('USER')
homedir = os.getenv('HOME')
if homedir is None:
homedir = ''
if args.no_syslog:
try:
logger.removeHandler(j_handler)
except NameError:
pass
try:
config_args = _read_config(args.config, user)
except (configparser.NoSectionError, configparser.NoOptionError):
logger.critical('Bad configuration file')
inputs, output, exclude, keep = _read_config(args.config)
return 6
if args.input is not None:
inputs = args.input
inputs = args.inputs if args.inputs is not None else config_args['inputs']
output = args.output if args.output is not None else config_args['output']
exclude = args.exclude if args.exclude is not None else config_args['exclude']
keep = args.keep if args.keep is not None else config_args['keep']
ssh_host = args.ssh_host if args.ssh_host is not None else config_args['ssh_host']
ssh_user = args.ssh_user if args.ssh_user is not None else config_args['ssh_user']
ssh_keyfile = args.keyfile if args.keyfile is not None else config_args['ssh_keyfile']
remote_sudo = args.remote_sudo or config_args['remote_sudo']
if args.output is not None:
output = args.output
if args.rsync_options is None:
rsync_options = ['-a', '-r', '-v', '-h', '-H',
'-X', '-s', '--ignore-missing-args', '--mkpath']
else:
rsync_options = ['-r', '-v']
if args.exclude is not None:
exclude = args.exclude
for ro in args.rsync_options:
rsync_options.append(f'-{ro}')
if args.keep is not None:
keep = args.keep
if args.checksum:
rsync_options.append('-c')
backup_options = '-arcvh -H -X'
else:
backup_options = '-arvh -H -X'
if args.compress:
rsync_options.append('-z')
backup = Backup(inputs, output, exclude, keep, backup_options, remove_before=args.remove_before_backup)
if args.numeric_ids or config_args['numeric_ids']:
rsync_options.append('--numeric-ids')
rsync_options = ' '.join(rsync_options)
backup = Backup(inputs, output, exclude, keep, rsync_options, ssh_host, ssh_user, ssh_keyfile,
remote_sudo, remove_before=args.remove_before_backup, verbose=args.verbose)
return_code = backup.check_params(homedir)
return_code = backup.check_params()
if return_code == 0:
return backup.run()