Compare commits

...
43 Commits
Author SHA1 Message Date
Fuxino 3cbca8e4d8 Bump version to 4.2.0 2026-10-08 18:23:03 +02:00
Fuxino 54880651be Fix outdated and wrong information in README
Use git.shouldnt.work as the clone URL, don't install wheel for
building, as setuptools no longer needs it, list paramiko among the
optional dependencies, and use the correct names of the remote
backup options (--ssh-user, --ssh-host).
2026-10-08 18:19:35 +02:00
Fuxino 793a30fad8 Remove unused python-dotenv dependency
load_dotenv() was called, but no setting was ever read from a .env
file. Drop it from the code, pyproject.toml, PKGBUILD and README.
2026-10-08 18:15:07 +02:00
Fuxino 871dd0f49a Warn if a backup fails after old backups were removed
With --remove-before-backup, a failed backup leaves only the kept
backups, or none with keep=0. Log an error and send a notification
in that case, and document it in the man page.
2026-10-08 18:13:13 +02:00
Fuxino 30e4416f8f Look for the last backup after removing old ones
With --remove-before-backup, the last backup was selected for
--link-dest before old backups were removed, so with keep=0 rsync
was given a directory that no longer existed.
2026-10-08 18:11:25 +02:00
Fuxino d0f888cbb4 Always use --mkpath and --ignore-missing-args with rsync
With --rsync-options, only -r and -v were kept, so the first backup
failed because the simple_backup directory did not exist yet, and a
missing input made rsync fail. Use these options in any case.
2026-10-08 18:08:28 +02:00
Fuxino 7ef29cf551 Report invalid config files instead of crashing
Invalid values (e.g. an empty or non-numeric 'keep') and syntax
errors in the config file raised uncaught exceptions. Catch them
and exit with code 6, including the reason in the error message.
2026-10-08 18:06:33 +02:00
Fuxino 4f5499eba9 Allow '%' in config file values
ConfigParser interpolation treats '%' as special, so a path or
pattern containing it crashed the program. Disable interpolation,
which is not used by the config file.
2026-10-08 18:05:11 +02:00
Fuxino fc165fccbf Only expand '~' at the start of config paths
Every '~' in inputs and backup_dir was replaced with '~user', so
paths like 'notes~' or '~otheruser' did not match. Expand only a
leading '~' or '~/'.
2026-10-08 18:04:25 +02:00
Fuxino d79ea56c0f Ignore whitespace around comma-separated config values
'inputs=/a, /b' was read as '/a' and ' /b', so /b was skipped, and
excluded patterns with spaces around them never matched. Strip
whitespace from each value, and ignore empty exclude entries.
2026-10-08 18:03:18 +02:00
Fuxino 5189910131 Don't use ':' in backup directory names
':' is not allowed on FAT/exFAT filesystems, so backups to such
drives failed. Use '%Y-%m-%d_%H-%M-%S' instead; existing backups
with the old format still sort correctly. Document the limitations
of FAT/exFAT in the man page.
2026-10-08 18:01:07 +02:00
Fuxino 7d4c43f8ff Always remove temp files and close their descriptors
The inputs temp file was left behind when no inputs existed, and
both temp files if rsync failed to start. The descriptors returned
by mkstemp were also never closed. Remove the temp files when run()
exits in any way, and write them through the mkstemp descriptors.
2026-10-08 17:58:32 +02:00
Fuxino ac7e6ec21b Don't print tracebacks when desktop notifications fail
Without dbus-python, or without a session bus (e.g. when running
from a timer), each notification printed a traceback. Skip
notifications if dbus is not installed, and log a single warning
if sending one fails. Remove the now unneeded NameError handling
around _notify calls.
2026-10-08 17:55:57 +02:00
Fuxino 578d992a19 Only turn warnings into errors where needed
warnings.filterwarnings('error') applied to the whole program, so
any warning raised by a library (e.g. a DeprecationWarning) would
crash it. Limit it to the ssh connection (unknown host key) and
the password prompt (no terminal available).
2026-10-08 17:50:47 +02:00
Fuxino e2d8897a45 Fix crash on Python < 3.14 when paramiko is not installed
The return annotation of _ssh_connect referenced paramiko, which
Python < 3.14 evaluates at class definition, raising NameError when
paramiko (an optional dependency) is missing. Postpone evaluation
of annotations, and mark the return type as Optional.
2026-10-08 17:47:02 +02:00
Fuxino dadc4b0265 Resolve default config path after the user is known
The default config path was computed before -u was parsed, so
running as root without sudo/doas used '~None/...' even when the
user was given with -u. Expand '~' only after the user is detected.
2026-10-08 17:45:02 +02:00
Fuxino 7e7d11c520 Detect the invoking user when running with doas
User detection only checked SUDO_USER, so with doas the user's
config file was not found unless -u was given. Fall back to
DOAS_USER, as already done for notifications.
2026-10-08 17:43:28 +02:00
Fuxino 22e0556b50 Document that old backups are kept if rsync fails 2026-10-08 17:41:17 +02:00
Fuxino 2b91178e94 Don't remove old backups if rsync fails
If rsync returns an error, the new backup may be incomplete. Keep
the old backups in that case, except for return code 24 (vanished
source files), which is expected when backing up a live system.
2026-10-08 17:38:34 +02:00
Fuxino 2c2649113d Fix relative input paths being silently skipped
rsync reads --files-from entries relative to the source directory
('/'), so relative inputs pointed to the wrong path and were ignored
due to --ignore-missing-args. Convert inputs to absolute paths.
2026-10-08 17:35:35 +02:00
Fuxino 0f4a100ac7 Move package metadata from setup.cfg to pyproject.toml
Drop python-wheel from PKGBUILD makedepends, as setuptools
builds wheels on its own.
2026-10-08 17:29:40 +02:00
Fuxino 60d83151dc Update PKGBUILD 2026-10-03 21:07:16 +02:00
Fuxino 47aec456be Fix notifications when using doas 2026-10-03 21:04:17 +02:00
Fuxino c715188973 Fix code with autopep8 2025-03-30 15:02:50 +02:00
Fuxino 57613649c2 Add missing type hints 2025-03-30 15:02:04 +02:00
Fuxino 67dbb49a67 Update PKGBUILD 2025-03-30 14:53:53 +02:00
Fuxino e62a668f3e Update metadata 2025-03-30 14:39:59 +02:00
Fuxino bd8934d57f Update version tag 2025-03-30 14:38:56 +02:00
Fuxino 96fe3c7813 Update man page 2025-03-30 14:38:29 +02:00
Fuxino 9fdb959540 Improve handling of user detection failure 2025-03-30 14:37:30 +02:00
Fuxino eb889beee7 Add Python 3.13 to setup.cfg 2025-03-30 12:16:04 +02:00
Fuxino 2a37eb4172 Add PKGBUILD 2024-11-13 19:38:12 +01:00
Fuxino 45a07205a1 Add type hints plus minor code fixes 2024-09-28 09:47:33 +02:00
Fuxino 9390cd2de8 Allow removing remote old backups with sudo if possible
If allowed by the remote server, try using sudo to remove
old backups (rm needs to be allowed in sudoers to run
without password)
2023-10-15 15:39:06 +02:00
Fuxino 77661c0964 Handle getpass exception 2023-07-16 08:22:51 +02:00
Fuxino e3a970217f Improve logging 2023-06-25 11:49:02 +02:00
Fuxino 8968fcd1a8 Fix parsing option bug 2023-06-25 10:12:07 +02:00
Fuxino 35b87c859e Add --user arg 2023-06-20 19:22:22 +02:00
Fuxino f3d5ebd276 Merge branch 'development' 2023-06-20 17:36:40 +02:00
Fuxino fb726a80ab Fix bug
Remove forgotten test code -.-
2023-06-20 17:36:12 +02:00
Fuxino 4701ee0b05 Fix typo 2023-06-19 16:00:21 +02:00
Fuxino dd779d242b Fix crash when config file missing 2023-06-18 22:58:53 +02:00
Fuxino 22a3e8d60f Make paramiko optional 2023-06-18 22:53:29 +02:00
8 changed files with 576 additions and 272 deletions
+38
View File
@@ -0,0 +1,38 @@
# PKGBUILD
# Maintainer: Daniele Fucini <dfucini [at] gmail [dot] com>
pkgname=simple_backup
pkgdesc='Simple backup script that uses rsync to copy files'
pkgver=4.1.7
pkgrel=1
url="https://git.shouldnt.work/fuxino/${pkgname}"
arch=('any')
license=('GPL-3.0-or-later')
makedepends=('git'
'python-setuptools'
'python-build'
'python-installer')
depends=('python>=3.10'
'rsync')
optdepends=('python-systemd: use systemd log'
'python-dbus: for desktop notifications'
'python-paramiko: for remote backup through ssh')
conflicts=('simple_backup-git')
source=(git+${url}?signed#tag=${pkgver})
validpgpkeys=('7E12BC1FF3B6EDB2CD8053EB981A2B2A3BBF5514')
sha256sums=('011cd5546745e413ac7603b64531def2688c2bf6e4ea19876a67547a1db208ce')
build()
{
cd ${srcdir}/${pkgname}
python -m build --wheel --no-isolation
}
package()
{
cd ${srcdir}/${pkgname}
python -m installer --destdir=${pkgdir} dist/*.whl
install -Dm644 ${pkgname}/${pkgname}.conf ${pkgdir}/usr/share/doc/${pkgname}/${pkgname}.conf
install -Dm644 man/${pkgname}.1 ${pkgdir}/usr/share/man/man1/${pkgname}.1
}
+10 -7
View File
@@ -20,21 +20,19 @@ The script uses rsync to actually run the backup, so you will have to install it
sudo pacman -Syu rsync sudo pacman -Syu rsync
``` ```
It's also required to have python-dotenv Optional dependencies are systemd-python to enable using systemd journal for logging, dbus-python for desktop notifications, and paramiko for remote backup.
Optional dependencies are systemd-python to enable using systemd journal for logging, and dbus-python for desktop notifications.
## Install ## Install
To install the program, first clone the repository: To install the program, first clone the repository:
```bash ```bash
git clone https://github.com/Fuxino/simple_backup.git git clone https://git.shouldnt.work/fuxino/simple_backup.git
``` ```
Then install the tools required to build the package: Then install the tools required to build the package:
```bash ```bash
pip install --upgrade build wheel pip install --upgrade build
``` ```
Finally, run: Finally, run:
@@ -49,11 +47,16 @@ For Arch Linux and Arch-based distros, two packages are available in the AUR (au
- **simple_backup** for the release version - **simple_backup** for the release version
- **simple_backup-git** for the git version - **simple_backup-git** for the git version
## Old backups
The number of old backups to keep can be set with the --keep (or -k) argument (or in the configuration file).
Old backups are only removed if rsync completes successfully, so that they are not deleted when the new backup may be incomplete.
The only rsync error that is ignored is return code 24 (source files vanished during the transfer), which is common when backing up a live system.
## Remote backup ## Remote backup
> **Warning** > **Warning**
> This feature is experimental > This feature is experimental
It's possible to use a remote server as destination for the backup. Just use the --username (or -u) and --host arguments (or set them in the configuration file). It's possible to use a remote server as destination for the backup. Just use the --ssh-user and --ssh-host arguments (or set them in the configuration file).
For this to work, rsync must be installed on the server too. For this to work, rsync must be installed on the server too.
### Server authentication ### Server authentication
@@ -68,4 +71,4 @@ sudo --preserve-env=SSH_AUTH_SOCK -s simple_backup [options]
or by editing the sudoers file. or by editing the sudoers file.
If SSH key authentication is not available, password authentication will be used instead. If SSH key authentication is not available, password authentication will be used instead.
Check the man page for more details.
+62 -35
View File
@@ -1,13 +1,13 @@
.TH SIMPLE_BACKUP 1 2023-06-15 SIMPLE_BACKUP 3.2.6 .TH SIMPLE_BACKUP 1 2026-10-08 SIMPLE_BACKUP 4.2.0
.SH NAME .SH NAME
simple_backup \- Backup files and folders using rsync simple_backup \- Backup files and folders using rsync
.SH SYNOPSIS .SH SYNOPSIS
.BR simple_backup .B simple_backup
\-h, \-\-help \-h, \-\-help
.PD 0 .PD 0
.P .P
.PD .PD
.BR simple_backup .B simple_backup
[\-c, \-\-config FILE] [\-c, \-\-config FILE]
[\-i, \-\-input INPUT [INPUT...]] [\-i, \-\-input INPUT [INPUT...]]
[\-o, \-\-output DIR] [\-o, \-\-output DIR]
@@ -16,8 +16,8 @@ simple_backup \- Backup files and folders using rsync
.PD .PD
.RS 14 [\-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]] .RS 14 [\-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]]
[\-k, \-\-keep N] [\-k, \-\-keep N]
[\-\-host HOSTNAME] [\-\-ssh\-host HOSTNAME]
[\-u, \-\-username USERNAME] [\-\-ssh\-user USERNAME]
[\-\-keyfile FILE] [\-\-keyfile FILE]
.PD 0 .PD 0
.P .P
@@ -27,7 +27,7 @@ simple_backup \- Backup files and folders using rsync
[\-\-remove\-before\-backup] [\-\-remove\-before\-backup]
.RE .RE
.SH DESCRIPTION .SH DESCRIPTION
.BR simple_backup .B simple_backup
is a python script for performing backup of files and folders. is a python script for performing backup of files and folders.
.P .P
It uses rsync to copy the files to the specified location. Parameters for the backup such as It uses rsync to copy the files to the specified location. Parameters for the backup such as
@@ -39,7 +39,7 @@ Parameters specified on the command line will override those in the configuratio
.SH OPTIONS .SH OPTIONS
.TP .TP
.B \-h, \-\-help .B \-h, \-\-help
Print a short help message and exit Print a short help message and exit.
.TP .TP
.B \-c, \-\-config FILE .B \-c, \-\-config FILE
Specify the configuration file, useful to specify a different one from the default. Specify the configuration file, useful to specify a different one from the default.
@@ -52,21 +52,36 @@ or to use single or double quotes around them.
.B \-o, \-\-output DIR .B \-o, \-\-output DIR
Specify the directory where the files will be copied. The program will automatically Specify the directory where the files will be copied. The program will automatically
create a subdirectory called \(aqsimple_backup\(aq (if it does not already exist) and create a subdirectory called \(aqsimple_backup\(aq (if it does not already exist) and
inside this directory the actual backup directory (using the current date and time) inside this directory the actual backup directory (using the current date and time, for example
\(aq2026\-10\-08_17\-34\-47\(aq).
.RS
.P
Backups are best stored on a Linux filesystem (e.g. ext4 or btrfs). FAT and exFAT filesystems
are supported, but they don\(aqt support hard links, so each backup will be a full copy instead
of only storing files that changed since the previous one. File ownership and permissions are
not preserved either.
.RE
.TP .TP
.B \-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]] .B \-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]]
Specify files, directories or patterns to exclude from the backup. Matching files and directories Specify files, directories or patterns to exclude from the backup. Matching files and directories
will not be copied. Multiple elements can be specified, in the same way as for the \-\-input option will not be copied. Multiple elements can be specified, in the same way as for the \-\-input option.
.TP .TP
.B \-k, \-\-keep N .B \-k, \-\-keep N
Specify how many old backups (so excluding the current one) will be kept. The default behavior Specify how many old backups (so excluding the current one) will be kept. The default behavior
is to keep them all (same as N=\-1) is to keep them all (same as N=\-1).
Old backups are only removed if rsync completes successfully, to avoid deleting them when the
new backup may be incomplete. rsync return code 24 (source files vanished during the transfer)
is treated as success, while any other error (including return code 23, partial transfer due to error)
will prevent old backups from being removed. This does not apply when using \-\-remove\-before\-backup.
.TP .TP
.B \-\-host HOSTNAME .B \-u, \-\-user USERNAME
Hostname of the server where to copy the files in case of remote backup through SSH Explicitly specify the user running the backup (in case it is needed for home directory expansion).
.TP .TP
.B \-u, \-\-username USERNAME .B \-\-ssh\-host HOSTNAME
Username for connecting to the server in case of remote backup Hostname of the server where to copy the files in case of remote backup through SSH.
.TP
.B \-\-ssh\-user USERNAME
Username for connecting to the server in case of remote backup.
.TP .TP
.B \-\-keyfile FILE .B \-\-keyfile FILE
Location of the SSH key for server authentication. Location of the SSH key for server authentication.
@@ -83,47 +98,54 @@ if saving bandwith is needed.
Remove old backups (if necessary) before creating the new backup. Useful to free some space Remove old backups (if necessary) before creating the new backup. Useful to free some space
before performing the backup. before performing the backup.
Default behavior is to remove old backups after successfully completing the backup. Default behavior is to remove old backups after successfully completing the backup.
Note that if the backup fails after old backups have been removed, only the backups kept
according to \-\-keep will be left, and the new backup may be incomplete. In particular, with
\-\-keep 0 there will be no complete backup available until the next successful one.
.TP .TP
.B \-\-no\-syslog .B \-\-no\-syslog
Don't use systemd journal for logging Don't use systemd journal for logging.
.TP .TP
.B \-\-rsync\-options OPTIONS [OPTION...] .B \-\-rsync\-options OPTIONS [OPTION...]
By default, the following rsync options are used: By default, the following rsync options are used:
.RS .RS
.PP .P
\-a \-r \-v \-h \-s \-H \-X \-a \-r \-v \-h \-s \-H \-X
.PP .P
Using \-\-rsync\-options it is possible to manually select which options to use. Supported values are the following: Using \-\-rsync\-options it is possible to manually select which options to use. Supported values are the following:
.PP .P
\-a, \-l, \-p, \-t, \-g, \-o, \-c, \-h, \-D, \-H, \-X, \-s \-a, \-l, \-p, \-t, \-g, \-o, \-c, \-h, \-D, \-H, \-X, \-s
.PP .P
Options \-r and \-v are used in any case. Not that options must be specified without dash (\-), for example: Options \-r, \-v, \-\-ignore\-missing\-args and \-\-mkpath are used in any case. Note that options must be specified without dash (\-), for example:
.PP .P
.EX .EX
simple_backup \-\-rsync\-options a l p simple_backup \-\-rsync\-options a l p
.EE .EE
.TP .P
Check Check
.B rsync (1) .BR rsync (1)
for details about the options. for details about the options.
.RE .RE
.TP .TP
.B \-\-remote\-sudo .B \-\-remote\-sudo
Run rsync on the remote server with sudo. This is needed if you want to preserve the owner of the files/folders to be copied (rsync \-\-owner option). For this to work the user used to login to the server obviously need to be allowed to use sudo. In addition, the user need to be able to run rsync with sudo without a password. To do this, /etc/sudoers on the server need to be edited adding a line like this one: Run rsync on the remote server with sudo. This is needed if you want to preserve the owner of the files/folders to be copied (rsync \-\-owner option). For this to work the user used to login to the server obviously need to be allowed to use sudo. In addition, the user need to be able to run rsync with sudo without a password. To do this, /etc/sudoers on the server need to be edited adding a line like this one:
.RS .RS
.PP .P
<username> ALL=NOPASSWD:<path/to/rsync> <username> ALL=NOPASSWD:<path/to/rsync>
.P
To be able to remove old backups generated with \-\-remote\-sudo (see \-\-keep option), also
.BR rm (1)
needs to be allowed to run without password in the same way.
.RE .RE
.TP .TP
.B \-\-numeric\-ids .B \-\-numeric\-ids
Use rsync \-\-numeric\-ids option. This causes rsync to use numeric uid/gid instead of trying to map uid/gid names from the local machine to the server Use rsync \-\-numeric\-ids option. This causes rsync to use numeric uid/gid instead of trying to map uid/gid names from the local machine to the server.
.SH CONFIGURATION .SH CONFIGURATION
An example configuration file is provided at \(aq/usr/share/doc/simple_backup/simple_backup.conf\(aq. An example configuration file is provided at \(aq/usr/share/doc/simple_backup/simple_backup.conf\(aq.
Copy it to the default location ($HOME/.config/simple_backup) and edit it as needed. Copy it to the default location ($HOME/.config/simple_backup) and edit it as needed.
.SH REMOTE BACKUP .SH REMOTE BACKUP
It is possible to choose a directory on a remote server as destination for the backup. The files It is possible to choose a directory on a remote server as destination for the backup. The files
are copied by rsync through SSH. Server hostname and username must be specified, either in the are copied by rsync through SSH. Server hostname and username must be specified, either in the
configuration file, or on the command line (\(aq\-\-host\(aq and \(aq\-\-username\(aq options). configuration file, or on the command line (\(aq\-\-ssh\-host\(aq and \(aq\-\-ssh\-user\(aq options).
.SS AUTHENTICATION .SS AUTHENTICATION
For authentication, it is possible to use SSH key or password. For authentication, it is possible to use SSH key or password.
.P .P
@@ -145,8 +167,7 @@ in order to connect to the user\(aq s SSH agent it is necessary to preserve the
It is also possible to make this permanent by editing the It is also possible to make this permanent by editing the
.B sudoers .B sudoers
file (see file (see
.B sudoers (5) .BR sudoers (5))
)
.P .P
If SSH key authentication is not available, password authentication will be used instead. If SSH key authentication is not available, password authentication will be used instead.
Note that in this case Note that in this case
@@ -154,27 +175,33 @@ Note that in this case
(if available) will be used to send the password to rsync, to avoid prompting the user for (if available) will be used to send the password to rsync, to avoid prompting the user for
the password multiple the password multiple
times. This can pose some security risks, see times. This can pose some security risks, see
.B sshpass (1) .BR sshpass (1)
for details. For this reason, use SSH key authentication if possible. for details. For this reason, use SSH key authentication if possible.
.SH EXIT STATUS .SH EXIT STATUS
.TP .TP
.B 0 .B 0
The backup was completed without errors The backup was completed without errors.
.TP .TP
.B 1 .B 1
No valid inputs selected for backup No valid inputs selected for backup.
.TP .TP
.B 2 .B 2
Backup failed because output directory for storing the backup does not exist Backup failed because output directory for storing the backup does not exist.
.TP .TP
.B 3 .B 3
Permission denied to access the output directory Permission denied to access the output directory.
.TP .TP
.B 4 .B 4
rsync error (rsync returned a non-zero value) rsync error (rsync returned a non-zero value).
.TP
.B 5
SSH connection failed.
.TP
.B 6
Bad configuration file.
.SH SEE ALSO .SH SEE ALSO
.BR rsync (1) .BR rsync (1)
.SH AUTHORS .SH AUTHORS
.MT https://github.com/Fuxino .MT https://git.shouldnt.work/fuxino
Daniele Fucini Daniele Fucini
.ME .ME
+46 -1
View File
@@ -1,3 +1,48 @@
[build-system] [build-system]
requires = ['setuptools'] requires = ['setuptools>=77']
build-backend = 'setuptools.build_meta' build-backend = 'setuptools.build_meta'
[project]
name = "simple_backup"
dynamic = ["version"]
description = "Simple backup script that uses rsync to copy files"
readme = "README.md"
requires-python = ">=3.10"
license = "GPL-3.0-or-later"
license-files = ["LICENSE"]
authors = [
{ name = "Daniele Fucini", email = "dfucini@gmail.com" }
]
maintainers = [
{ name = "Daniele Fucini", email = "dfucini@gmail.com" }
]
classifiers = [
"Development Status :: 4 - Beta",
"Environment :: Console",
"Natural Language :: English",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: System :: Archiving :: Backup",
]
[project.optional-dependencies]
JOURNAL = ["systemd-python"]
NOTIFICATIONS = ["dbus-python"]
REMOTE = ["paramiko"]
[project.urls]
Homepage = "https://git.shouldnt.work/fuxino/simple_backup"
[project.scripts]
simple_backup = "simple_backup.simple_backup:simple_backup"
[tool.setuptools]
packages = ["simple_backup"]
[tool.setuptools.dynamic]
version = { attr = "simple_backup.__version__" }
-40
View File
@@ -1,40 +0,0 @@
[metadata]
name = simple_backup
version = attr: simple_backup.__version__
description = Simple backup script using rsync
long_description = file: README.md
author = Daniele Fucini
author_email = dfucini@gmail.com
license = GPL3
url = https://github.com/Fuxino/simple_backup
classifiers =
Development Status :: 4 - Beta
Environment :: Console
License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Natural Language :: English
Operating System :: POSIX :: Linux
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Topic :: System :: Archiving :: Backup
[options]
packages = simple_backup
python_requires = >=3.7
install_requires =
python-dotenv
[options.extras_require]
JOURNAL =
systemd-python
NOTIFICATIONS =
dbus-python
REMOTE =
paramiko
[options.entry_points]
console_scripts =
simple_backup = simple_backup.simple_backup:simple_backup
+1 -1
View File
@@ -1,3 +1,3 @@
"""Init.""" """Init."""
__version__ = '4.0.0' __version__ = '4.2.0'
+3 -3
View File
@@ -2,7 +2,7 @@
[backup] [backup]
# Files and directories to backup. Multiple items can be separated using a comma (','). It is possible to use wildcards (i.e. '*' to match multiple characters and '~' for the user's home directory). # Files and directories to backup. Multiple items can be separated using a comma (','). It is possible to use wildcards (i.e. '*' to match multiple characters and '~' for the user's home directory).
inputs=/home/my_home,/etc inputs=/home/user
# Output directory. # Output directory.
backup_dir=/media/Backup backup_dir=/media/Backup
@@ -15,8 +15,8 @@ keep=-1
# Uncomment the following section to enable backup to remote server through ssh # Uncomment the following section to enable backup to remote server through ssh
# [server] # [server]
# host= # ssh_host=
# username= # ssh_user=
# ssh_keyfile= # ssh_keyfile=
# remote_sudo= # remote_sudo=
# numeric_ids= # numeric_ids=
+397 -166
View File
@@ -10,9 +10,15 @@ Classes:
MyFormatter MyFormatter
Backup Backup
""" """
# Avoid evaluating annotations at definition time, as they reference optional modules (paramiko)
from __future__ import annotations
# Import libraries # Import libraries
import sys import sys
import os import os
import pwd
from typing import Callable, List, Optional, ParamSpec, TypeVar, Union
import warnings import warnings
from functools import wraps from functools import wraps
from shutil import rmtree, which from shutil import rmtree, which
@@ -25,14 +31,14 @@ from timeit import default_timer
from subprocess import Popen, PIPE, STDOUT from subprocess import Popen, PIPE, STDOUT
from datetime import datetime from datetime import datetime
from tempfile import mkstemp from tempfile import mkstemp
from getpass import getpass from getpass import GetPassWarning, getpass
from glob import glob from glob import glob
from dotenv import load_dotenv try:
import paramiko import paramiko
from paramiko import RSAKey, Ed25519Key, ECDSAKey, DSSKey from paramiko import RSAKey, Ed25519Key, ECDSAKey, DSSKey
except ImportError:
warnings.filterwarnings('error') pass
try: try:
from systemd import journal from systemd import journal
@@ -42,19 +48,9 @@ except ImportError:
try: try:
import dbus import dbus
except ImportError: except ImportError:
pass dbus = None
load_dotenv()
euid = os.geteuid()
if euid == 0:
user = os.getenv('SUDO_USER')
homedir = os.path.expanduser(f'~{user}')
else:
user = os.getenv('USER')
homedir = os.getenv('HOME')
logging.getLogger().setLevel(logging.DEBUG) logging.getLogger().setLevel(logging.DEBUG)
logger = logging.getLogger(os.path.basename(__file__)) logger = logging.getLogger(os.path.basename(__file__))
c_handler = StreamHandler() c_handler = StreamHandler()
@@ -71,29 +67,29 @@ if journal:
j_handler.setFormatter(j_format) j_handler.setFormatter(j_format)
logger.addHandler(j_handler) logger.addHandler(j_handler)
P = ParamSpec('P')
R = TypeVar('R')
def timing(_logger):
def timing(func: Callable[P, R]) -> Callable[P, R]:
"""Decorator to measure execution time of a function """Decorator to measure execution time of a function
Parameters: Parameters:
_logger: Logger object func: Function to decorate
""" """
def decorator_timing(func):
@wraps(func) @wraps(func)
def wrapper_timing(*args, **kwargs): def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
start = default_timer() start = default_timer()
value = func(*args, **kwargs) value = func(*args, **kwargs)
end = default_timer() end = default_timer()
_logger.info(f'Elapsed time: {end - start:.3f} seconds') logger.info('Elapsed time: %.3f seconds', end - start)
return value return value
return wrapper_timing return wrapper
return decorator_timing
class MyFormatter(argparse.RawTextHelpFormatter, argparse.ArgumentDefaultsHelpFormatter): class MyFormatter(argparse.RawTextHelpFormatter, argparse.ArgumentDefaultsHelpFormatter):
@@ -114,9 +110,9 @@ class Backup:
String representing main backup options for rsync String representing main backup options for rsync
keep: int keep: int
Number of old backup to preserve Number of old backup to preserve
host: str ssh_host: str
Hostname of server (for remote backup) Hostname of server (for remote backup)
username: str ssh_user: str
Username for server login (for remote backup) Username for server login (for remote backup)
ssh_keyfile: str ssh_keyfile: str
Location of ssh key Location of ssh key
@@ -138,52 +134,57 @@ class Backup:
Perform the backup Perform the backup
""" """
def __init__(self, inputs, output, exclude, keep, options, host=None, username=None, def __init__(self, inputs: List[str], output: str, exclude: List[str], keep: int, options: str,
ssh_keyfile=None, remote_sudo=False, remove_before=False): ssh_host: Optional[str] = None, ssh_user: Optional[str] = None, ssh_keyfile: Optional[str] = None,
remote_sudo: bool = False, remove_before: bool = False, verbose: bool = False) -> None:
self.inputs = inputs self.inputs = inputs
self.output = output self.output = output
self.exclude = exclude self.exclude = exclude
self.options = options self.options = options
self.keep = keep self.keep = keep
self.host = host self.ssh_host = ssh_host
self.username = username self.ssh_user = ssh_user
self.ssh_keyfile = ssh_keyfile self.ssh_keyfile = ssh_keyfile
self.remote_sudo = remote_sudo self.remote_sudo = remote_sudo
self._remove_before = remove_before self._remove_before = remove_before
self._verbose = verbose
self._last_backup = '' self._last_backup = ''
self._server = '' self._server = ''
self._output_dir = '' self._output_dir = ''
self._inputs_path = '' self._inputs_path = ''
self._exclude_path = '' self._exclude_path = ''
self._remote = None self._remote = False
self._err_flag = False
self._ssh = None self._ssh = None
self._password_auth = False self._password_auth = False
self._password = None self._password = None
self._removed_count = 0
def check_params(self): def check_params(self, homedir: str = '') -> int:
"""Check if parameters for the backup are valid""" """Check if parameters for the backup are valid"""
if self.inputs is None or len(self.inputs) == 0: if self.inputs is None or len(self.inputs) == 0:
logger.info('No existing files or directories specified for backup. Nothing to do') logger.info(
'No existing files or directories specified for backup. Nothing to do')
return 1 return 1
if self.output is None: if self.output is None:
logger.critical('No output path specified. Use -o argument or specify output path in configuration file') logger.critical(
'No output path specified. Use -o argument or specify output path in configuration file')
return 2 return 2
if self.host is not None and self.username is not None: if self.ssh_host is not None and self.ssh_user is not None:
self._remote = True self._remote = True
if self._remote: if self._remote:
self._ssh = self._ssh_connect() self._ssh = self._ssh_connect(homedir)
if self._ssh is None: if self._ssh is None:
sys.exit(1) return 5
_, stdout, _ = self._ssh.exec_command(f'if [ -d "{self.output}" ]; then echo "ok"; fi') _, stdout, _ = self._ssh.exec_command(
f'if [ -d "{self.output}" ]; then echo "ok"; fi')
output = stdout.read().decode('utf-8').strip() output = stdout.read().decode('utf-8').strip()
@@ -205,19 +206,24 @@ class Backup:
return 0 return 0
# Function to create the actual backup directory # Function to create the actual backup directory
def define_backup_dir(self): def define_backup_dir(self) -> None:
"""Define the actual backup dir""" """Define the actual backup dir"""
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S') # Avoid ':', which is not allowed on FAT/exFAT filesystems.
# Backups created with the old format ('%Y-%m-%d %H:%M:%S') still sort correctly
now = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
self._output_dir = f'{self.output}/simple_backup/{now}' self._output_dir = f'{self.output}/simple_backup/{now}'
if self._remote: if self._remote:
self._server = f'{self.username}@{self.host}:' self._server = f'{self.ssh_user}@{self.ssh_host}:'
def remove_old_backups(self): def remove_old_backups(self) -> None:
"""Remove old backups if there are more than indicated by 'keep'""" """Remove old backups if there are more than indicated by 'keep'"""
if self._remote: if self._remote:
_, stdout, _ = self._ssh.exec_command(f'ls {self.output}/simple_backup') assert self._ssh is not None
_, stdout, _ = self._ssh.exec_command(
f'ls {self.output}/simple_backup')
dirs = stdout.read().decode('utf-8').strip().split('\n') dirs = stdout.read().decode('utf-8').strip().split('\n')
@@ -233,12 +239,18 @@ class Backup:
dirs.sort() dirs.sort()
for i in range(n_backup - self.keep): for i in range(n_backup - self.keep):
_, _, stderr = self._ssh.exec_command(f'rm -r "{self.output}/simple_backup/{dirs[i]}"') if self.remote_sudo:
_, _, stderr = self._ssh.exec_command(
f'sudo rm -r "{self.output}/simple_backup/{dirs[i]}"')
else:
_, _, stderr = self._ssh.exec_command(
f'rm -r "{self.output}/simple_backup/{dirs[i]}"')
err = stderr.read().decode('utf-8').strip().split('\n')[0] err = stderr.read().decode('utf-8').strip().split('\n')[0]
if err != '': if err != '':
logger.error('Error while removing backup %s.', {dirs[i]}) logger.error(
'Error while removing backup %s.', {dirs[i]})
logger.error(err) logger.error(err)
else: else:
count += 1 count += 1
@@ -264,24 +276,29 @@ class Backup:
rmtree(f'{self.output}/simple_backup/{dirs[i]}') rmtree(f'{self.output}/simple_backup/{dirs[i]}')
count += 1 count += 1
except FileNotFoundError: except FileNotFoundError:
logger.error('Error while removing backup %s. Directory not found', dirs[i]) logger.error(
'Error while removing backup %s. Directory not found', dirs[i])
except PermissionError: except PermissionError:
logger.error('Error while removing backup %s. Permission denied', dirs[i]) logger.error(
'Error while removing backup %s. Permission denied', dirs[i])
self._removed_count = count
if count == 1: if count == 1:
logger.info('Removed %d backup', count) logger.info('Removed %d backup', count)
elif count > 1: elif count > 1:
logger.info('Removed %d backups', count) logger.info('Removed %d backups', count)
def find_last_backup(self): def find_last_backup(self) -> None:
"""Get path of last backup (from last_backup symlink) for rsync --link-dest""" """Get path of last backup (from last_backup symlink) for rsync --link-dest"""
if self._remote: if self._remote:
if self._ssh is None: if self._ssh is None:
logger.critical('SSH connection to server failed') logger.critical('SSH connection to server failed')
sys.exit(1) sys.exit(5)
_, stdout, _ = self._ssh.exec_command(f'find {self.output}/simple_backup/ -mindepth 1 -maxdepth 1 -type d | sort') _, stdout, _ = self._ssh.exec_command(
f'find {self.output}/simple_backup/ -mindepth 1 -maxdepth 1 -type d | sort')
output = stdout.read().decode('utf-8').strip().split('\n') output = stdout.read().decode('utf-8').strip().split('\n')
if output[-1] != '': if output[-1] != '':
@@ -290,18 +307,17 @@ class Backup:
logger.info('No previous backups available') logger.info('No previous backups available')
else: else:
try: try:
dirs = sorted([f.path for f in os.scandir(f'{self.output}/simple_backup') if f.is_dir(follow_symlinks=False)]) dirs = sorted([f.path for f in os.scandir(
f'{self.output}/simple_backup') if f.is_dir(follow_symlinks=False)])
except FileNotFoundError: except FileNotFoundError:
logger.info('No previous backups available') logger.info('No previous backups available')
return return
except PermissionError: except PermissionError:
logger.critical('Cannot access the backup directory. Permission denied') logger.critical(
'Cannot access the backup directory. Permission denied')
try: _notify('Backup failed (check log for details)')
notify('Backup failed (check log for details)')
except NameError:
pass
sys.exit(3) sys.exit(3)
@@ -310,22 +326,31 @@ class Backup:
except IndexError: except IndexError:
logger.info('No previous backups available') logger.info('No previous backups available')
def _ssh_connect(self): def _ssh_connect(self, homedir: str = '') -> Optional[paramiko.client.SSHClient]:
try:
ssh = paramiko.SSHClient() ssh = paramiko.SSHClient()
except NameError:
logger.error('Install paramiko for ssh support')
return None
try: try:
ssh.load_host_keys(filename=f'{homedir}/.ssh/known_hosts') ssh.load_host_keys(filename=f'{homedir}/.ssh/known_hosts')
except FileNotFoundError: except FileNotFoundError:
logger.warning(f'Cannot find file {homedir}/.ssh/known_hosts') logger.warning('Cannot find file %s/.ssh/known_hosts', homedir)
ssh.set_missing_host_key_policy(paramiko.WarningPolicy()) ssh.set_missing_host_key_policy(paramiko.WarningPolicy())
try: try:
ssh.connect(self.host, username=self.username) # WarningPolicy emits a UserWarning for unknown host keys
with warnings.catch_warnings():
warnings.simplefilter('error', UserWarning)
ssh.connect(self.ssh_host, username=self.ssh_user)
return ssh return ssh
except UserWarning: except UserWarning:
k = input(f'Unknown key for host {self.host}. Continue anyway? (Y/N) ') k = input(
f'Unknown key for host {self.ssh_host}. Continue anyway? (Y/N) ')
if k[0].upper() == 'Y': if k[0].upper() == 'Y':
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
@@ -340,7 +365,7 @@ class Backup:
pass pass
try: try:
ssh.connect(self.host, username=self.username) ssh.connect(self.ssh_host, username=self.ssh_user)
return ssh return ssh
except paramiko.SSHException: except paramiko.SSHException:
@@ -348,13 +373,24 @@ class Backup:
if self.ssh_keyfile is None: if self.ssh_keyfile is None:
try: try:
password = getpass(f'{self.username}@{self.host}\'s password: ') # Fail instead of reading the password with echo if no terminal is available
ssh.connect(self.host, username=self.username, password=password) with warnings.catch_warnings():
warnings.simplefilter('error', GetPassWarning)
password = getpass(
f'{self.ssh_user}@{self.ssh_host}\'s password: ')
ssh.connect(self.ssh_host, username=self.ssh_user,
password=password)
self._password_auth = True self._password_auth = True
os.environ['SSHPASS'] = password os.environ['SSHPASS'] = password
return ssh return ssh
except GetPassWarning as e:
logger.critical('Unable to get password')
logger.critical(e)
return None
except paramiko.SSHException as e: except paramiko.SSHException as e:
logger.critical('Can\'t connect to the server.') logger.critical('Can\'t connect to the server.')
logger.critical(e) logger.critical(e)
@@ -366,7 +402,8 @@ class Backup:
try: try:
pkey = RSAKey.from_private_key_file(self.ssh_keyfile) pkey = RSAKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException: except paramiko.PasswordRequiredException:
password = getpass(f'Enter passwphrase for key \'{self.ssh_keyfile}\': ') password = getpass(
f'Enter passwphrase for key \'{self.ssh_keyfile}\': ')
try: try:
pkey = RSAKey.from_private_key_file(self.ssh_keyfile, password) pkey = RSAKey.from_private_key_file(self.ssh_keyfile, password)
@@ -378,7 +415,8 @@ class Backup:
pkey = Ed25519Key.from_private_key_file(self.ssh_keyfile) pkey = Ed25519Key.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException: except paramiko.PasswordRequiredException:
try: try:
pkey = Ed25519Key.from_private_key_file(self.ssh_keyfile, password) pkey = Ed25519Key.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException: except paramiko.SSHException:
pass pass
@@ -387,7 +425,8 @@ class Backup:
pkey = ECDSAKey.from_private_key_file(self.ssh_keyfile) pkey = ECDSAKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException: except paramiko.PasswordRequiredException:
try: try:
pkey = ECDSAKey.from_private_key_file(self.ssh_keyfile, password) pkey = ECDSAKey.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException: except paramiko.SSHException:
pass pass
@@ -396,12 +435,13 @@ class Backup:
pkey = DSSKey.from_private_key_file(self.ssh_keyfile) pkey = DSSKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException: except paramiko.PasswordRequiredException:
try: try:
pkey = DSSKey.from_private_key_file(self.ssh_keyfile, password) pkey = DSSKey.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException: except paramiko.SSHException:
pass pass
try: try:
ssh.connect(self.host, username=self.username, pkey=pkey) ssh.connect(self.ssh_host, username=self.ssh_user, pkey=pkey)
except paramiko.SSHException: except paramiko.SSHException:
logger.critical('SSH connection to server failed') logger.critical('SSH connection to server failed')
@@ -409,46 +449,97 @@ class Backup:
return ssh return ssh
def _returncode_log(self, returncode: int) -> None:
match returncode:
case 2:
logger.error(
'Rsync error (return code 2) - Protocol incompatibility')
case 3:
logger.error(
'Rsync error (return code 3) - Errors selecting input/output files, dirs')
case 4:
logger.error(
'Rsync error (return code 4) - Requested action not supported')
case 5:
logger.error(
'Rsync error (return code 5) - Error starting client-server protocol')
case 10:
logger.error(
'Rsync error (return code 10) - Error in socket I/O')
case 11:
logger.error(
'Rsync error (return code 11) - Error in file I/O')
case 12:
logger.error(
'Rsync error (return code 12) - Error in rsync protocol data stream')
case 22:
logger.error(
'Rsync error (return code 22) - Error allocating core memory buffers')
case 23:
logger.warning(
'Rsync error (return code 23) - Partial transfer due to error')
case 24:
logger.warning(
'Rsync error (return code 24) - Partial transfer due to vanished source files')
case 30:
logger.error(
'Rsync error (return code 30) - Timeout in data send/receive')
case 35:
logger.error(
'Rsync error (return code 35) - Timeout waiting for daemon connection')
case _:
logger.error(
'Rsync error (return code %d) - Check rsync(1) for details', returncode)
# Function to read configuration file # Function to read configuration file
@timing(logger) @timing
def run(self): def run(self) -> int:
"""Perform the backup""" """Perform the backup"""
logger.info('Starting backup...')
try: try:
_notify('Starting backup...') return self._run()
except NameError: finally:
self._remove_temp_files()
def _remove_temp_files(self) -> None:
for path in [self._inputs_path, self._exclude_path]:
if path != '':
try:
os.remove(path)
except FileNotFoundError:
pass pass
self.define_backup_dir() def _run(self) -> int:
self.find_last_backup() logger.info('Starting backup...')
_, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True) _notify('Starting backup...')
self.define_backup_dir()
fd, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True)
count = 0 count = 0
with open(self._inputs_path, 'w', encoding='utf-8') as fp: with os.fdopen(fd, 'w', encoding='utf-8') as fp:
for i in self.inputs: for i in self.inputs:
if not os.path.exists(i): if not os.path.exists(i):
logger.warning('Input %s not found. Skipping', i) logger.warning('Input %s not found. Skipping', i)
else: else:
fp.write(i) # rsync reads --files-from paths relative to the source ('/')
fp.write(os.path.abspath(i))
fp.write('\n') fp.write('\n')
count += 1 count += 1
if count == 0: if count == 0:
logger.info('No existing files or directories specified for backup. Nothing to do') logger.info(
'No existing files or directories specified for backup. Nothing to do')
try: _notify('Backup finished. No files copied')
notify('Backup finished. No files copied')
except NameError:
pass
return 1 return 1
_, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True) fd, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True)
with open(self._exclude_path, 'w', encoding='utf-8') as fp: with os.fdopen(fd, 'w', encoding='utf-8') as fp:
if self.exclude is not None: if self.exclude is not None:
for e in self.exclude: for e in self.exclude:
fp.write(e) fp.write(e)
@@ -457,6 +548,9 @@ class Backup:
if self.keep != -1 and self._remove_before: if self.keep != -1 and self._remove_before:
self.remove_old_backups() self.remove_old_backups()
# Only look for the last backup now, as it may have been removed above
self.find_last_backup()
logger.info('Copying files. This may take a long time...') logger.info('Copying files. This may take a long time...')
if self._last_backup == '': if self._last_backup == '':
@@ -466,10 +560,12 @@ class Backup:
rsync = f'/usr/bin/rsync {self.options} --link-dest="{self._last_backup}" --exclude-from=' +\ rsync = f'/usr/bin/rsync {self.options} --link-dest="{self._last_backup}" --exclude-from=' +\
f'{self._exclude_path} --files-from={self._inputs_path} / "{self._server}{self._output_dir}"' f'{self._exclude_path} --files-from={self._inputs_path} / "{self._server}{self._output_dir}"'
euid = os.geteuid()
if euid == 0 and self.ssh_keyfile is not None: if euid == 0 and self.ssh_keyfile is not None:
rsync = f'{rsync} -e \'ssh -i {self.ssh_keyfile} -o StrictHostKeyChecking=no\'' rsync = f'{rsync} -e \'ssh -i {self.ssh_keyfile} -o StrictHostKeyChecking=no\''
elif self._password_auth and which('sshpass'): elif self._password_auth and which('sshpass'):
rsync = f'{rsync} -e \'sshpass -e ssh -l {self.username} -o StrictHostKeyChecking=no\'' rsync = f'{rsync} -e \'sshpass -e ssh -l {self.ssh_user} -o StrictHostKeyChecking=no\''
else: else:
rsync = f'{rsync} -e \'ssh -o StrictHostKeyChecking=no\'' rsync = f'{rsync} -e \'ssh -o StrictHostKeyChecking=no\''
@@ -479,6 +575,7 @@ class Backup:
args = shlex.split(rsync) args = shlex.split(rsync)
with Popen(args, stdin=PIPE, stdout=PIPE, stderr=STDOUT, shell=False) as p: with Popen(args, stdin=PIPE, stdout=PIPE, stderr=STDOUT, shell=False) as p:
output: Union[bytes, List[str]]
output, _ = p.communicate() output, _ = p.communicate()
try: try:
@@ -486,125 +583,188 @@ class Backup:
except KeyError: except KeyError:
pass pass
if p.returncode != 0: returncode = p.returncode
self._err_flag = True
output = output.decode("utf-8").split('\n') output = output.decode("utf-8").split('\n')
if self._err_flag: if returncode == 0:
logger.error('rsync: %s', output) if self._verbose:
logger.info('rsync: %s', output)
else: else:
logger.info('rsync: %s', output[-3]) logger.info('rsync: %s', output[-3])
logger.info('rsync: %s', output[-2]) logger.info('rsync: %s', output[-2])
else:
self._returncode_log(returncode)
if self._verbose:
if returncode in [23, 24]:
logger.warning(output)
else:
logger.error(output)
if self._removed_count > 0 and returncode not in [0, 24]:
# With --remove-before-backup, old backups are already gone if the new one failed
if self.keep == 0:
message = 'All old backups were removed before the backup, and the new backup may be ' +\
'incomplete. No complete backup is available'
else:
message = 'Old backups were removed before the backup, and the new backup may be incomplete'
logger.error(message)
_notify(message)
if self.keep != -1 and not self._remove_before: if self.keep != -1 and not self._remove_before:
# Don't delete old backups if the new one may be incomplete
if returncode in [0, 24]:
self.remove_old_backups() self.remove_old_backups()
else:
os.remove(self._inputs_path) logger.warning('Backup not completed successfully. Old backups will not be removed')
os.remove(self._exclude_path)
if self._remote: if self._remote:
_, stdout, _ = self._ssh.exec_command(f'if [ -d "{self._output_dir}" ]; then echo "ok"; fi') assert self._ssh is not None
_, stdout, _ = self._ssh.exec_command(
f'if [ -d "{self._output_dir}" ]; then echo "ok"; fi')
output = stdout.read().decode('utf-8').strip() output = stdout.read().decode('utf-8').strip()
if output == 'ok': if output == 'ok':
logger.info('Backup completed') logger.info('Backup completed')
try:
_notify('Backup completed') _notify('Backup completed')
except NameError:
pass
else: else:
logger.error('Backup failed') logger.error('Backup failed')
try:
_notify('Backup failed (check log for details)') _notify('Backup failed (check log for details)')
except NameError:
pass
if self._ssh: if self._ssh:
self._ssh.close() self._ssh.close()
else: else:
if self._err_flag: if returncode != 0:
logger.error('Some errors occurred while performing the backup') logger.error(
'Some errors occurred while performing the backup')
try: _notify(
_notify('Some errors occurred while performing the backup. Check log for details') 'Some errors occurred while performing the backup. Check log for details')
except NameError:
pass
return 4 return 4
else:
logger.info('Backup completed') logger.info('Backup completed')
try:
_notify('Backup completed') _notify('Backup completed')
except NameError:
pass
return 0 return 0
def _parse_arguments(): def _detect_user() -> Optional[str]:
if os.geteuid() == 0:
return os.getenv('SUDO_USER') or os.getenv('DOAS_USER')
return os.getenv('USER')
def _parse_arguments() -> argparse.Namespace:
parser = argparse.ArgumentParser(prog='simple_backup', parser = argparse.ArgumentParser(prog='simple_backup',
description='Simple backup script written in Python that uses rsync to copy files', description='Simple backup script written in Python that uses rsync to copy files',
epilog='See simple_backup(1) manpage for full documentation', epilog='See simple_backup(1) manpage for full documentation',
formatter_class=MyFormatter) formatter_class=MyFormatter)
parser.add_argument('-c', '--config', default=f'{homedir}/.config/simple_backup/simple_backup.conf', parser.add_argument('-v', '--verbose', action='store_true',
help='More verbose output')
parser.add_argument('-c', '--config', default='~/.config/simple_backup/simple_backup.conf',
help='Specify location of configuration file') help='Specify location of configuration file')
parser.add_argument('-i', '--inputs', nargs='+', help='Paths/files to backup') parser.add_argument('-i', '--inputs', nargs='+',
parser.add_argument('-o', '--output', help='Output directory for the backup') help='Paths/files to backup')
parser.add_argument('-e', '--exclude', nargs='+', help='Files/directories/patterns to exclude from the backup') parser.add_argument(
parser.add_argument('-k', '--keep', type=int, help='Number of old backups to keep') '-o', '--output', help='Output directory for the backup')
parser.add_argument('--host', help='Server hostname (for remote backup)') parser.add_argument('-e', '--exclude', nargs='+',
parser.add_argument('-u', '--username', help='Username to connect to server (for remote backup)') help='Files/directories/patterns to exclude from the backup')
parser.add_argument('--keyfile', help='SSH key location') parser.add_argument('-k', '--keep', type=int,
help='Number of old backups to keep')
parser.add_argument(
'-u', '--user', help='Explicitly specify the user running the backup')
parser.add_argument('-s', '--checksum', action='store_true', parser.add_argument('-s', '--checksum', action='store_true',
help='Use checksum rsync option to compare files') help='Use checksum rsync option to compare files')
parser.add_argument('-z', '--compress', action='store_true', help='Compress data during the transfer') parser.add_argument(
'--ssh-host', help='Server hostname (for remote backup)')
parser.add_argument(
'--ssh-user', help='Username to connect to server (for remote backup)')
parser.add_argument('--keyfile', help='SSH key location')
parser.add_argument('-z', '--compress', action='store_true',
help='Compress data during the transfer')
parser.add_argument('--remove-before-backup', action='store_true', parser.add_argument('--remove-before-backup', action='store_true',
help='Remove old backups before executing the backup, instead of after') help='Remove old backups before executing the backup, instead of after')
parser.add_argument('--no-syslog', action='store_true', help='Disable systemd journal logging') parser.add_argument('--no-syslog', action='store_true',
help='Disable systemd journal logging')
parser.add_argument('--rsync-options', nargs='+', parser.add_argument('--rsync-options', nargs='+',
choices=['a', 'l', 'p', 't', 'g', 'o', 'c', 'h', 's', 'D', 'H', 'X'], choices=['a', 'l', 'p', 't', 'g', 'o',
'c', 'h', 's', 'D', 'H', 'X'],
help='Specify options for rsync') help='Specify options for rsync')
parser.add_argument('--remote-sudo', action='store_true', help='Run rsync on remote server with sudo if allowed') parser.add_argument('--remote-sudo', action='store_true',
help='Run rsync on remote server with sudo if allowed')
parser.add_argument('--numeric-ids', action='store_true', parser.add_argument('--numeric-ids', action='store_true',
help='Use rsync \'--numeric-ids\' option (don\'t map uid/gid values by name') help='Use rsync \'--numeric-ids\' option (don\'t map uid/gid values by name)')
args = parser.parse_args() args = parser.parse_args()
return args return args
def _expand_inputs(inputs): def _expand_home(path: str, user: str) -> str:
"""Expand a leading '~' to the home directory of user (other '~' are left as they are)"""
if path == '~' or path.startswith('~/'):
path = f'~{user}{path[1:]}'
return os.path.expanduser(path)
def _expand_inputs(inputs, user: Optional[str] = None):
expanded_inputs = [] expanded_inputs = []
for i in inputs: for i in inputs:
if i == '': if i == '':
continue continue
i_ex = glob(os.path.expanduser(i.replace('~', f'~{user}'))) if user is not None:
i_ex = glob(_expand_home(i, user))
else:
i_ex = glob(i)
if i.startswith('~'):
logger.warning('Cannot expand \'~\'. No user specified')
if len(i_ex) == 0: if len(i_ex) == 0:
logger.warning('No file or directory matching input %s. Skipping...', i) logger.warning(
'No file or directory matching input %s. Skipping...', i)
else: else:
expanded_inputs.extend(i_ex) expanded_inputs.extend(i_ex)
return expanded_inputs return expanded_inputs
def _read_config(config_file): def _read_config(config_file, user: Optional[str] = None):
config_args = {} config_args = {'inputs': None,
'output': None,
'exclude': None,
'keep': -1,
'ssh_host': None,
'ssh_user': None,
'ssh_keyfile': None,
'remote_sudo': False,
'numeric_ids': False}
if not os.path.isfile(config_file): if not os.path.isfile(config_file):
if user is not None:
logger.warning('Config file %s does not exist', config_file) logger.warning('Config file %s does not exist', config_file)
else:
logger.warning(
'User not specified. Can\'t read configuration file')
return config_args return config_args
config = configparser.ConfigParser() # Disable interpolation, so that '%' can be used in paths and patterns
config = configparser.ConfigParser(interpolation=None)
config.read(config_file) config.read(config_file)
section = 'backup' section = 'backup'
@@ -616,20 +776,24 @@ def _read_config(config_file):
section = 'default' section = 'default'
inputs = config.get(section, 'inputs') inputs = config.get(section, 'inputs')
inputs = inputs.split(',') inputs = [i.strip() for i in inputs.split(',')]
inputs = _expand_inputs(inputs) inputs = _expand_inputs(inputs, user)
inputs = list(set(inputs)) inputs = list(set(inputs))
config_args['inputs'] = inputs config_args['inputs'] = inputs
output = config.get(section, 'backup_dir') output = config.get(section, 'backup_dir')
output = os.path.expanduser(output.replace('~', f'~{user}'))
if user is not None:
output = _expand_home(output, user)
elif user is None and output.startswith('~'):
logger.warning('Cannot expand \'~\', no user specified')
config_args['output'] = output config_args['output'] = output
try: try:
exclude = config.get(section, 'exclude') exclude = config.get(section, 'exclude')
exclude = exclude.split(',') exclude = [e.strip() for e in exclude.split(',') if e.strip() != '']
except configparser.NoOptionError: except configparser.NoOptionError:
exclude = [] exclude = []
@@ -643,14 +807,14 @@ def _read_config(config_file):
config_args['keep'] = keep config_args['keep'] = keep
try: try:
host = config.get('server', 'host') ssh_host = config.get('server', 'ssh_host')
username = config.get('server', 'username') ssh_user = config.get('server', 'ssh_user')
except (configparser.NoSectionError, configparser.NoOptionError): except (configparser.NoSectionError, configparser.NoOptionError):
host = None ssh_host = None
username = None ssh_user = None
config_args['host'] = host config_args['ssh_host'] = ssh_host
config_args['username'] = username config_args['ssh_user'] = ssh_user
try: try:
ssh_keyfile = config.get('server', 'ssh_keyfile') ssh_keyfile = config.get('server', 'ssh_keyfile')
@@ -676,55 +840,122 @@ def _read_config(config_file):
return config_args return config_args
def _notify(text): def _target_uid() -> int | None:
_euid = os.geteuid() euid = os.geteuid()
if _euid == 0: if euid != 0:
uid = os.getenv('SUDO_UID') return euid
else:
uid = os.geteuid()
os.seteuid(int(uid)) if (val := os.getenv('SUDO_UID')) is not None:
return int(val)
if (name := os.getenv('DOAS_USER')) is not None:
try:
return pwd.getpwnam(name).pw_uid
except KeyError:
return None
return None
_notify_failed = False
def _notify(text: str) -> None:
global _notify_failed
if dbus is None or _notify_failed:
return
euid = os.geteuid()
uid = _target_uid()
if uid is None:
logger.debug('Cannot send desktop notification: user not found')
return
os.seteuid(uid)
try:
os.environ['DBUS_SESSION_BUS_ADDRESS'] = f'unix:path=/run/user/{uid}/bus' os.environ['DBUS_SESSION_BUS_ADDRESS'] = f'unix:path=/run/user/{uid}/bus'
obj = dbus.SessionBus().get_object('org.freedesktop.Notifications', '/org/freedesktop/Notifications') obj = dbus.SessionBus().get_object('org.freedesktop.Notifications',
'/org/freedesktop/Notifications')
obj = dbus.Interface(obj, 'org.freedesktop.Notifications') obj = dbus.Interface(obj, 'org.freedesktop.Notifications')
obj.Notify('', 0, '', 'simple_backup', text, [], {'urgency': 1}, 10000) obj.Notify('simple_backup', 0, '', 'simple_backup', text, [], {'urgency': dbus.Byte(1)}, 10000)
except Exception as e:
os.seteuid(int(_euid)) # Report the failure only once (e.g. no session bus available), then stop trying
_notify_failed = True
logger.warning('Cannot send desktop notification: %s', e)
finally:
os.seteuid(euid)
def simple_backup(): def simple_backup() -> int:
"""Main""" """Main"""
args = _parse_arguments() args = _parse_arguments()
if args.user:
user = args.user
homedir = os.path.expanduser(f'~{user}')
else:
euid = os.geteuid()
if euid == 0:
user = _detect_user()
if user is not None:
homedir = os.path.expanduser(f'~{user}')
else:
logger.warning(
'Failed to detect user. You can use -u/--user parameter to manually specify it')
homedir = None
else:
user = os.getenv('USER')
homedir = os.getenv('HOME')
if homedir is None:
homedir = ''
if args.no_syslog: if args.no_syslog:
try: try:
logger.removeHandler(j_handler) logger.removeHandler(j_handler)
except NameError: except NameError:
pass pass
config_file = args.config
# Expand '~' to the home of the user running the backup, not root's
if config_file.startswith('~/') and user is not None:
config_file = f'~{user}{config_file[1:]}'
config_file = os.path.expanduser(config_file)
try: try:
config_args = _read_config(args.config) config_args = _read_config(config_file, user)
except (configparser.NoSectionError, configparser.NoOptionError): except (configparser.Error, ValueError) as e:
logger.critical('Bad configuration file') # configparser.Error: missing sections/options or bad syntax, ValueError: invalid values (e.g. keep=abc)
sys.exit(1) logger.critical('Bad configuration file: %s', e)
return 6
inputs = args.inputs if args.inputs is not None else config_args['inputs'] inputs = args.inputs if args.inputs is not None else config_args['inputs']
output = args.output if args.output is not None else config_args['output'] output = args.output if args.output is not None else config_args['output']
exclude = args.exclude if args.exclude is not None else config_args['exclude'] exclude = args.exclude if args.exclude is not None else config_args['exclude']
keep = args.keep if args.keep is not None else config_args['keep'] keep = args.keep if args.keep is not None else config_args['keep']
host = args.host if args.host is not None else config_args['host'] ssh_host = args.ssh_host if args.ssh_host is not None else config_args['ssh_host']
username = args.username if args.username is not None else config_args['username'] ssh_user = args.ssh_user if args.ssh_user is not None else config_args['ssh_user']
ssh_keyfile = args.keyfile if args.keyfile is not None else config_args['ssh_keyfile'] ssh_keyfile = args.keyfile if args.keyfile is not None else config_args['ssh_keyfile']
remote_sudo = args.remote_sudo if args.remote_sudo is not None else config_args['remote_sudo'] remote_sudo = args.remote_sudo or config_args['remote_sudo']
# Options needed in any case (--mkpath creates the backup directory on the first run)
rsync_options = ['-r', '-v', '--ignore-missing-args', '--mkpath']
if args.rsync_options is None: if args.rsync_options is None:
rsync_options = ['-a', '-r', '-v', '-h', '-H', '-X', '-s', '--ignore-missing-args', '--mkpath'] rsync_options.extend(['-a', '-h', '-H', '-X', '-s'])
else: else:
rsync_options = ['-r', '-v']
for ro in args.rsync_options: for ro in args.rsync_options:
rsync_options.append(f'-{ro}') rsync_options.append(f'-{ro}')
@@ -739,10 +970,10 @@ def simple_backup():
rsync_options = ' '.join(rsync_options) rsync_options = ' '.join(rsync_options)
backup = Backup(inputs, output, exclude, keep, rsync_options, host, username, ssh_keyfile, backup = Backup(inputs, output, exclude, keep, rsync_options, ssh_host, ssh_user, ssh_keyfile,
remote_sudo, remove_before=args.remove_before_backup) remote_sudo, remove_before=args.remove_before_backup, verbose=args.verbose)
return_code = backup.check_params() return_code = backup.check_params(homedir)
if return_code == 0: if return_code == 0:
return backup.run() return backup.run()