Compare commits

...
50 Commits
Author SHA1 Message Date
Fuxino 0829c33053 Fix mypy errors in simple_backup() and Backup
Accept None for the missing inputs, output, exclude and keep values and replace them with their defaults in __init__, and annotate the SSH client, which mypy typed as None and so skipped checking the remote code.
2026-10-14 08:37:06 +02:00
Fuxino cfe3b28436 Fix mypy errors in the tests
The config values are now a TypedDict, so check inputs for None before sorting, use literal keys and use ConfigArgs as the fake _read_config() return type.
2026-10-14 08:37:06 +02:00
Fuxino 1fb0513467 Add type hints to the config reading functions
Return a TypedDict from _read_config() and stop reusing variables for different types, so that mypy can check the config values. No change in behaviour.
2026-10-14 08:33:45 +02:00
Fuxino d803790c1a Fix mypy error for the SSH user in the sshpass command
Check that ssh_user is set before quoting it. It always is in remote mode, but mypy can't know that.
2026-10-14 08:32:00 +02:00
Fuxino 8d08032b5d Add tests for remote backups
Use a fake SSH client that runs commands locally to test remote path resolution, removing old backups, finding the last backup and the rsync arguments, plus the login paths with a fake paramiko client and real keys made by ssh-keygen.
2026-10-14 08:23:25 +02:00
Fuxino 28ecff2306 Add tests for local backups
Run real rsync into a temp dir to test hard links, keep, remove-before-backup, stray files in the backup dir and the old folder name format, and use a fake Popen to test how rsync return codes are handled.
2026-10-14 08:15:07 +02:00
Fuxino dbb0aadc84 Add tests for command line handling and user detection
Test how command line and config values are combined, the exit codes for an incomplete SSH setup and a bad config, and the user detection under sudo/doas (simulated by patching geteuid and the environment).
2026-10-14 08:10:02 +02:00
Fuxino 41a45dad41 Add pytest setup and configuration tests
Configure pytest in pyproject.toml to import the repo copy, add a conftest that keeps tests out of the journal and desktop notifications, and test reading the config file and expanding paths.
2026-10-14 08:05:30 +02:00
Fuxino 4b28cb7edf Update README for 4.3.0
Document the remote output path and that both SSH host and user are required, the paramiko version,
running as root with sudo/doas, and installing from source with pipx instead of pip.
2026-10-13 20:50:03 +02:00
Fuxino 63774e92d7 Update PKGBUILD 2026-10-08 20:48:14 +02:00
Fuxino 166c9dbc47 Bump version to 4.3.0 2026-10-08 20:47:04 +02:00
Fuxino 28d1b0d0d0 Add a timeout to the SSH connection
paramiko's connect had no timeout, so a server that doesn't answer
made the program hang until the OS gave up (about 2 minutes). Give up
after 15 seconds and exit with code 5.
2026-10-08 20:45:07 +02:00
Fuxino cd80c941ad Treat empty SSH settings in the config as not set
The example config has empty remote_sudo and numeric_ids values, which
made getboolean fail with "Bad configuration file" if the [server]
section was uncommented, and an empty ssh_keyfile was used as the key
path. Treat empty values as false or not set, and show false in the
example config.
2026-10-08 20:42:05 +02:00
Fuxino 29958896a2 Always close the SSH connection
The connection was only closed at the end of a backup that ran to
completion, not on early returns (e.g. missing output folder or no
inputs) or exceptions, and a client that failed to log in was left
open. Close it in all cases, and remove the unused _password field.
2026-10-08 20:40:41 +02:00
Fuxino d815b0832a Only count directories when removing old local backups
Old backups were listed with os.listdir, so a stray file or symlink
in the backup folder counted as a backup: depending on its name,
rmtree crashed on it or one backup too many was removed. An
unreadable backup folder also crashed with PermissionError. Count
only directories, and log an error if the folder can't be read.
2026-10-08 20:38:20 +02:00
Fuxino e6ac9e6cab Fix remote removal of old backups deleting the wrong folders
Old backups on the server were listed with ls, which also lists files
and prints nothing on errors. A stray file made one backup too many
get removed, and an empty or unreadable backup folder was counted as
one backup with an empty name, which made rm -r delete the whole
simple_backup folder. List only directories with find, and don't
remove anything if listing fails.
2026-10-08 20:36:55 +02:00
Fuxino 9298f4232c Fix crash on empty answer to the unknown host key prompt
An empty answer raised IndexError, and running without a terminal
(e.g. from cron) raised EOFError. Treat both as "no", and log why the
connection was aborted.
2026-10-08 20:34:02 +02:00
Fuxino 1dca989310 Fail if only one of SSH host and SSH user is set
Remote backup needs both, but if only one was set the program silently
ran a local backup instead, and a missing ssh_user in the config also
dropped ssh_host. Exit with an error (code 6) instead.
2026-10-08 20:31:39 +02:00
Fuxino 9a68a8660d Resolve the remote output path on the server
The output path was made absolute with os.path.abspath and '~' was
expanded to the local home, so relative paths and '~' pointed to the
wrong folder on the server. Resolve the path on the server instead,
where '~' and relative paths refer to the remote user's home, and
document it.
2026-10-08 20:29:21 +02:00
Fuxino 73de4f8d25 Build the rsync command as an argument list
The rsync command was built as a string and split with shlex.split,
so paths containing quotes broke it, and the key file path in the -e
command was split on spaces. Pass each path as its own argument, and
quote the key file and user name inside the -e command, which rsync
splits itself.
2026-10-08 20:24:53 +02:00
Fuxino baf5b07d23 Quote paths in commands run on the SSH server
Remote commands were built with f-strings, with paths either unquoted
or in double quotes. Paths with spaces broke ls and find, and $ or
backticks in a path were expanded by the server's shell. Quote every
path with shlex.quote.
2026-10-08 20:22:35 +02:00
Fuxino 8b30fefed3 Don't disable host key checking for rsync's ssh
rsync's ssh command used StrictHostKeyChecking=no, so it connected
even if the server's host key had changed. Use accept-new instead:
unknown hosts are still added automatically, but a changed key for a
known host makes the connection fail.
2026-10-08 20:19:46 +02:00
Fuxino 2a5562d1a9 Pass the SSH key file to rsync also when not running as root
The key file given with --keyfile was only passed to rsync's ssh
command when running as root, so as a normal user rsync ignored it
and fell back to the default keys or the agent.
2026-10-08 20:17:29 +02:00
Fuxino 7091118663 Handle SSH connection errors instead of crashing
An unknown hostname, a refused connection or an unreachable server
raise OSError (socket.gaierror, NoValidConnectionsError), which
paramiko.SSHException doesn't cover, so the program crashed with a
traceback. Log the error and exit with code 5 instead.
2026-10-08 20:14:39 +02:00
Fuxino 3113e182c7 Treat rsync return code 24 as success
Code 24 (source files vanished during the transfer) already counted as
success when deciding whether to remove old backups, but the program
still exited with 4 and reported errors. Exit with 0 and report the
backup as completed, logging only a warning, so both checks agree.
2026-10-08 20:12:29 +02:00
Fuxino 8fc3c42a5e Fix exit code and success check for remote backups
The remote branch of run() returned None, so the program always exited
with 0, and it decided whether the backup worked by checking that the
backup folder existed on the server, ignoring rsync's return code.
Use the same return code check for local and remote backups.
2026-10-08 20:10:52 +02:00
Fuxino 0671f86579 Fix SSH key loading for ed25519 and ECDSA keys
Unencrypted ed25519 and ECDSA keys crashed with an uncaught
SSHException, and the DSA fallback raised NameError since paramiko 4
removed DSSKey. Load the key with paramiko.PKey.from_path instead,
ask for the passphrase only when the key is encrypted, and log an
error instead of crashing when the key can't be loaded.

Require paramiko >= 3.2 for PKey.from_path.
2026-10-08 20:06:57 +02:00
Fuxino 2f207efcf9 Update PKGBUILD 2026-10-08 18:25:08 +02:00
Fuxino 3cbca8e4d8 Bump version to 4.2.0 2026-10-08 18:23:03 +02:00
Fuxino 54880651be Fix outdated and wrong information in README
Use git.shouldnt.work as the clone URL, don't install wheel for
building, as setuptools no longer needs it, list paramiko among the
optional dependencies, and use the correct names of the remote
backup options (--ssh-user, --ssh-host).
2026-10-08 18:19:35 +02:00
Fuxino 793a30fad8 Remove unused python-dotenv dependency
load_dotenv() was called, but no setting was ever read from a .env
file. Drop it from the code, pyproject.toml, PKGBUILD and README.
2026-10-08 18:15:07 +02:00
Fuxino 871dd0f49a Warn if a backup fails after old backups were removed
With --remove-before-backup, a failed backup leaves only the kept
backups, or none with keep=0. Log an error and send a notification
in that case, and document it in the man page.
2026-10-08 18:13:13 +02:00
Fuxino 30e4416f8f Look for the last backup after removing old ones
With --remove-before-backup, the last backup was selected for
--link-dest before old backups were removed, so with keep=0 rsync
was given a directory that no longer existed.
2026-10-08 18:11:25 +02:00
Fuxino d0f888cbb4 Always use --mkpath and --ignore-missing-args with rsync
With --rsync-options, only -r and -v were kept, so the first backup
failed because the simple_backup directory did not exist yet, and a
missing input made rsync fail. Use these options in any case.
2026-10-08 18:08:28 +02:00
Fuxino 7ef29cf551 Report invalid config files instead of crashing
Invalid values (e.g. an empty or non-numeric 'keep') and syntax
errors in the config file raised uncaught exceptions. Catch them
and exit with code 6, including the reason in the error message.
2026-10-08 18:06:33 +02:00
Fuxino 4f5499eba9 Allow '%' in config file values
ConfigParser interpolation treats '%' as special, so a path or
pattern containing it crashed the program. Disable interpolation,
which is not used by the config file.
2026-10-08 18:05:11 +02:00
Fuxino fc165fccbf Only expand '~' at the start of config paths
Every '~' in inputs and backup_dir was replaced with '~user', so
paths like 'notes~' or '~otheruser' did not match. Expand only a
leading '~' or '~/'.
2026-10-08 18:04:25 +02:00
Fuxino d79ea56c0f Ignore whitespace around comma-separated config values
'inputs=/a, /b' was read as '/a' and ' /b', so /b was skipped, and
excluded patterns with spaces around them never matched. Strip
whitespace from each value, and ignore empty exclude entries.
2026-10-08 18:03:18 +02:00
Fuxino 5189910131 Don't use ':' in backup directory names
':' is not allowed on FAT/exFAT filesystems, so backups to such
drives failed. Use '%Y-%m-%d_%H-%M-%S' instead; existing backups
with the old format still sort correctly. Document the limitations
of FAT/exFAT in the man page.
2026-10-08 18:01:07 +02:00
Fuxino 7d4c43f8ff Always remove temp files and close their descriptors
The inputs temp file was left behind when no inputs existed, and
both temp files if rsync failed to start. The descriptors returned
by mkstemp were also never closed. Remove the temp files when run()
exits in any way, and write them through the mkstemp descriptors.
2026-10-08 17:58:32 +02:00
Fuxino ac7e6ec21b Don't print tracebacks when desktop notifications fail
Without dbus-python, or without a session bus (e.g. when running
from a timer), each notification printed a traceback. Skip
notifications if dbus is not installed, and log a single warning
if sending one fails. Remove the now unneeded NameError handling
around _notify calls.
2026-10-08 17:55:57 +02:00
Fuxino 578d992a19 Only turn warnings into errors where needed
warnings.filterwarnings('error') applied to the whole program, so
any warning raised by a library (e.g. a DeprecationWarning) would
crash it. Limit it to the ssh connection (unknown host key) and
the password prompt (no terminal available).
2026-10-08 17:50:47 +02:00
Fuxino e2d8897a45 Fix crash on Python < 3.14 when paramiko is not installed
The return annotation of _ssh_connect referenced paramiko, which
Python < 3.14 evaluates at class definition, raising NameError when
paramiko (an optional dependency) is missing. Postpone evaluation
of annotations, and mark the return type as Optional.
2026-10-08 17:47:02 +02:00
Fuxino dadc4b0265 Resolve default config path after the user is known
The default config path was computed before -u was parsed, so
running as root without sudo/doas used '~None/...' even when the
user was given with -u. Expand '~' only after the user is detected.
2026-10-08 17:45:02 +02:00
Fuxino 7e7d11c520 Detect the invoking user when running with doas
User detection only checked SUDO_USER, so with doas the user's
config file was not found unless -u was given. Fall back to
DOAS_USER, as already done for notifications.
2026-10-08 17:43:28 +02:00
Fuxino 22e0556b50 Document that old backups are kept if rsync fails 2026-10-08 17:41:17 +02:00
Fuxino 2b91178e94 Don't remove old backups if rsync fails
If rsync returns an error, the new backup may be incomplete. Keep
the old backups in that case, except for return code 24 (vanished
source files), which is expected when backing up a live system.
2026-10-08 17:38:34 +02:00
Fuxino 2c2649113d Fix relative input paths being silently skipped
rsync reads --files-from entries relative to the source directory
('/'), so relative inputs pointed to the wrong path and were ignored
due to --ignore-missing-args. Convert inputs to absolute paths.
2026-10-08 17:35:35 +02:00
Fuxino 0f4a100ac7 Move package metadata from setup.cfg to pyproject.toml
Drop python-wheel from PKGBUILD makedepends, as setuptools
builds wheels on its own.
2026-10-08 17:29:40 +02:00
Fuxino 60d83151dc Update PKGBUILD 2026-10-03 21:07:16 +02:00
13 changed files with 1895 additions and 343 deletions

No files matched your search

+4 -6
View File
@@ -4,7 +4,7 @@
pkgname=simple_backup
pkgdesc='Simple backup script that uses rsync to copy files'
pkgver=4.1.6
pkgver=4.3.0
pkgrel=1
url="https://git.shouldnt.work/fuxino/${pkgname}"
arch=('any')
@@ -12,18 +12,16 @@ license=('GPL-3.0-or-later')
makedepends=('git'
'python-setuptools'
'python-build'
'python-installer'
'python-wheel')
'python-installer')
depends=('python>=3.10'
'rsync'
'python-dotenv')
'rsync')
optdepends=('python-systemd: use systemd log'
'python-dbus: for desktop notifications'
'python-paramiko: for remote backup through ssh')
conflicts=('simple_backup-git')
source=(git+${url}?signed#tag=${pkgver})
validpgpkeys=('7E12BC1FF3B6EDB2CD8053EB981A2B2A3BBF5514')
sha256sums=('b3b29d9e2e1b7b949e95674d9a401e8eeb0d5f898e8450473dce94f799ee9df3')
sha256sums=('7d9ea50d5829ac3c681a979745f1ae7afd2453eb46ceeae4f441ef5ae4540bd2')
build()
{
+35 -17
View File
@@ -20,42 +20,60 @@ The script uses rsync to actually run the backup, so you will have to install it
sudo pacman -Syu rsync
```
It's also required to have python-dotenv
Optional dependencies are systemd-python to enable using systemd journal for logging, and dbus-python for desktop notifications.
Optional dependencies are systemd-python to enable using systemd journal for logging, dbus-python for desktop notifications, and paramiko (version 3.2 or later) for remote backup.
## Install
To install the program, first clone the repository:
For Arch Linux and Arch-based distros, two packages are available in the AUR (aur.archlinux.org):
- **simple_backup** for the release version
- **simple_backup-git** for the git version
To install it from source, first clone the repository:
```bash
git clone https://github.com/Fuxino/simple_backup.git
git clone https://git.shouldnt.work/fuxino/simple_backup.git
```
Then install the tools required to build the package:
```bash
pip install --upgrade build wheel
```
Finally, run:
Then build the package (this requires the Python `build` module, e.g. the python-build package on Arch Linux):
```bash
cd simple_backup
python -m build --wheel
python -m pip install dist/*.whl
```
For Arch Linux and Arch-based distros, two packages are available in the AUR (aur.archlinux.org):
- **simple_backup** for the release version
- **simple_backup-git** for the git version
Many distributions don't allow installing packages with pip into the system Python ("externally-managed-environment" error).
In that case, install the wheel in a virtual environment, for example with pipx:
```bash
pipx install dist/*.whl
```
Optional dependencies can be installed as extras, for example `pipx install 'dist/simple_backup-<version>-py3-none-any.whl[REMOTE]'` (available extras: JOURNAL, NOTIFICATIONS, REMOTE).
## Running as root
To back up files that your user can't read, run simple_backup with sudo or doas.
In that case the user who ran the command is detected automatically, so that '~' (in the inputs and in the default configuration file path) refers to that user's home directory, not root's.
When running as root in other ways (e.g. from a systemd service), specify the user with the --user (or -u) argument.
## Old backups
The number of old backups to keep can be set with the --keep (or -k) argument (or in the configuration file).
Old backups are only removed if rsync completes successfully, so that they are not deleted when the new backup may be incomplete.
The only rsync error that is ignored is return code 24 (source files vanished during the transfer), which is common when backing up a live system.
## Remote backup
> **Warning**
> This feature is experimental
It's possible to use a remote server as destination for the backup. Just use the --username (or -u) and --host arguments (or set them in the configuration file).
It's possible to use a remote server as destination for the backup. Just use the --ssh-user and --ssh-host arguments (or set them in the configuration file).
Both are required: if only one of them is set, simple_backup exits with an error.
For this to work, rsync must be installed on the server too.
The output directory is a path on the server. A leading '~' and relative paths refer to the user's home directory on the server.
On the command line, quote '~' so that your local shell doesn't expand it:
```bash
simple_backup --ssh-host server --ssh-user user -o '~/backups'
```
### Server authentication
The best way to handle the authentication is to have an ssh agent running on your system, otherwise if a passphrase is necessary to unlock the ssh key, it will be necessary to enter it more than once.
If needed, it's possible to specify the ssh key location with the --keyfile argument or in the configuration file.
+30 -6
View File
@@ -1,4 +1,4 @@
.TH SIMPLE_BACKUP 1 2025-03-30 SIMPLE_BACKUP 4.1.6
.TH SIMPLE_BACKUP 1 2026-10-08 SIMPLE_BACKUP 4.3.0
.SH NAME
simple_backup \- Backup files and folders using rsync
.SH SYNOPSIS
@@ -52,7 +52,15 @@ or to use single or double quotes around them.
.B \-o, \-\-output DIR
Specify the directory where the files will be copied. The program will automatically
create a subdirectory called \(aqsimple_backup\(aq (if it does not already exist) and
inside this directory the actual backup directory (using the current date and time).
inside this directory the actual backup directory (using the current date and time, for example
\(aq2026\-10\-08_17\-34\-47\(aq).
.RS
.P
Backups are best stored on a Linux filesystem (e.g. ext4 or btrfs). FAT and exFAT filesystems
are supported, but they don\(aqt support hard links, so each backup will be a full copy instead
of only storing files that changed since the previous one. File ownership and permissions are
not preserved either.
.RE
.TP
.B \-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]]
Specify files, directories or patterns to exclude from the backup. Matching files and directories
@@ -61,6 +69,10 @@ will not be copied. Multiple elements can be specified, in the same way as for t
.B \-k, \-\-keep N
Specify how many old backups (so excluding the current one) will be kept. The default behavior
is to keep them all (same as N=\-1).
Old backups are only removed if rsync completes successfully, to avoid deleting them when the
new backup may be incomplete. rsync return code 24 (source files vanished during the transfer)
is treated as success, while any other error (including return code 23, partial transfer due to error)
will prevent old backups from being removed. This does not apply when using \-\-remove\-before\-backup.
.TP
.B \-u, \-\-user USERNAME
Explicitly specify the user running the backup (in case it is needed for home directory expansion).
@@ -86,6 +98,9 @@ if saving bandwith is needed.
Remove old backups (if necessary) before creating the new backup. Useful to free some space
before performing the backup.
Default behavior is to remove old backups after successfully completing the backup.
Note that if the backup fails after old backups have been removed, only the backups kept
according to \-\-keep will be left, and the new backup may be incomplete. In particular, with
\-\-keep 0 there will be no complete backup available until the next successful one.
.TP
.B \-\-no\-syslog
Don't use systemd journal for logging.
@@ -100,7 +115,7 @@ Using \-\-rsync\-options it is possible to manually select which options to use.
.P
\-a, \-l, \-p, \-t, \-g, \-o, \-c, \-h, \-D, \-H, \-X, \-s
.P
Options \-r and \-v are used in any case. Not that options must be specified without dash (\-), for example:
Options \-r, \-v, \-\-ignore\-missing\-args and \-\-mkpath are used in any case. Note that options must be specified without dash (\-), for example:
.P
.EX
simple_backup \-\-rsync\-options a l p
@@ -131,6 +146,14 @@ Copy it to the default location ($HOME/.config/simple_backup) and edit it as nee
It is possible to choose a directory on a remote server as destination for the backup. The files
are copied by rsync through SSH. Server hostname and username must be specified, either in the
configuration file, or on the command line (\(aq\-\-ssh\-host\(aq and \(aq\-\-ssh\-user\(aq options).
.P
The output directory is a path on the server. A leading \(aq~\(aq and relative paths refer to the
home directory of the user on the server. On the command line, quote \(aq~\(aq so that the local
shell doesn\(aqt expand it, for example:
.P
.EX
simple_backup \-\-ssh\-host server \-\-ssh\-user user \-o \(aq~/backups\(aq
.EE
.SS AUTHENTICATION
For authentication, it is possible to use SSH key or password.
.P
@@ -165,7 +188,8 @@ for details. For this reason, use SSH key authentication if possible.
.SH EXIT STATUS
.TP
.B 0
The backup was completed without errors.
The backup was completed without errors. This includes rsync return code 24 (some source files
vanished during the transfer), which only logs a warning.
.TP
.B 1
No valid inputs selected for backup.
@@ -177,13 +201,13 @@ Backup failed because output directory for storing the backup does not exist.
Permission denied to access the output directory.
.TP
.B 4
rsync error (rsync returned a non-zero value).
rsync error (rsync returned a value other than 0 or 24).
.TP
.B 5
SSH connection failed.
.TP
.B 6
Bad configuration file.
Bad configuration file or options (for example, only one of SSH host and SSH user specified).
.SH SEE ALSO
.BR rsync (1)
.SH AUTHORS
+51 -1
View File
@@ -1,3 +1,53 @@
[build-system]
requires = ['setuptools']
requires = ['setuptools>=77']
build-backend = 'setuptools.build_meta'
[project]
name = "simple_backup"
dynamic = ["version"]
description = "Simple backup script that uses rsync to copy files"
readme = "README.md"
requires-python = ">=3.10"
license = "GPL-3.0-or-later"
license-files = ["LICENSE"]
authors = [
{ name = "Daniele Fucini", email = "dfucini@gmail.com" }
]
maintainers = [
{ name = "Daniele Fucini", email = "dfucini@gmail.com" }
]
classifiers = [
"Development Status :: 4 - Beta",
"Environment :: Console",
"Natural Language :: English",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: System :: Archiving :: Backup",
]
[project.optional-dependencies]
JOURNAL = ["systemd-python"]
NOTIFICATIONS = ["dbus-python"]
REMOTE = ["paramiko>=3.2"]
[project.urls]
Homepage = "https://git.shouldnt.work/fuxino/simple_backup"
[project.scripts]
simple_backup = "simple_backup.simple_backup:simple_backup"
[tool.setuptools]
packages = ["simple_backup"]
[tool.setuptools.dynamic]
version = { attr = "simple_backup.__version__" }
[tool.pytest.ini_options]
testpaths = ["tests"]
# Import the repo copy, not an installed one
pythonpath = ["."]
-39
View File
@@ -1,39 +0,0 @@
[metadata]
name = simple_backup
version = attr: simple_backup.__version__
description = Simple backup script using rsync
long_description = file: README.md
author = Daniele Fucini
author_email = dfucini@gmail.com
license = GPL3
url = https://git.shouldnt.work/fuxino
classifiers =
Development Status :: 4 - Beta
Environment :: Console
License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Natural Language :: English
Operating System :: POSIX :: Linux
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: 3.13
Topic :: System :: Archiving :: Backup
[options]
packages = simple_backup
python_requires = >=3.10
install_requires =
python-dotenv
[options.extras_require]
JOURNAL =
systemd-python
NOTIFICATIONS =
dbus-python
REMOTE =
paramiko
[options.entry_points]
console_scripts =
simple_backup = simple_backup.simple_backup:simple_backup
+1 -1
View File
@@ -1,3 +1,3 @@
"""Init."""
__version__ = '4.1.6'
__version__ = '4.3.0'
+3 -3
View File
@@ -4,7 +4,7 @@
# Files and directories to backup. Multiple items can be separated using a comma (','). It is possible to use wildcards (i.e. '*' to match multiple characters and '~' for the user's home directory).
inputs=/home/user
# Output directory.
# Output directory. For remote backups, this is a path on the server ('~' and relative paths refer to the home directory on the server).
backup_dir=/media/Backup
# Files, directories and patterns to exclude from the backup. Multiple items can be separated using a comma.
@@ -18,5 +18,5 @@ keep=-1
# ssh_host=
# ssh_user=
# ssh_keyfile=
# remote_sudo=
# numeric_ids=
# remote_sudo=false
# numeric_ids=false
+360 -270
View File
@@ -11,12 +11,14 @@ Classes:
Backup
"""
# Avoid evaluating annotations at definition time, as they reference optional modules (paramiko)
from __future__ import annotations
# Import libraries
import traceback
import sys
import os
import pwd
from typing import Callable, List, Optional, ParamSpec, TypeVar, Union
from typing import Callable, List, Optional, ParamSpec, TypedDict, TypeVar, Union
import warnings
from functools import wraps
from shutil import rmtree, which
@@ -32,13 +34,8 @@ from tempfile import mkstemp
from getpass import GetPassWarning, getpass
from glob import glob
from dotenv import load_dotenv
warnings.filterwarnings('error')
try:
import paramiko
from paramiko import RSAKey, Ed25519Key, ECDSAKey, DSSKey
except ImportError:
pass
@@ -50,10 +47,9 @@ except ImportError:
try:
import dbus
except ImportError:
pass
dbus = None
load_dotenv()
logging.getLogger().setLevel(logging.DEBUG)
logger = logging.getLogger(os.path.basename(__file__))
c_handler = StreamHandler()
@@ -73,6 +69,9 @@ if journal:
P = ParamSpec('P')
R = TypeVar('R')
# Seconds to wait for the TCP connection to the SSH server
SSH_TIMEOUT = 15
def timing(func: Callable[P, R]) -> Callable[P, R]:
"""Decorator to measure execution time of a function
@@ -137,14 +136,16 @@ class Backup:
Perform the backup
"""
def __init__(self, inputs: List[str], output: str, exclude: List[str], keep: int, options: str,
def __init__(self, inputs: Optional[List[str]], output: Optional[str], exclude: Optional[List[str]],
keep: Optional[int], options: str,
ssh_host: Optional[str] = None, ssh_user: Optional[str] = None, ssh_keyfile: Optional[str] = None,
remote_sudo: bool = False, remove_before: bool = False, verbose: bool = False) -> None:
self.inputs = inputs
self.output = output
self.exclude = exclude
# Missing values are reported by check_params()
self.inputs = inputs if inputs is not None else []
self.output = output if output is not None else ''
self.exclude = exclude if exclude is not None else []
self.options = options
self.keep = keep
self.keep = keep if keep is not None else -1
self.ssh_host = ssh_host
self.ssh_user = ssh_user
self.ssh_keyfile = ssh_keyfile
@@ -157,20 +158,20 @@ class Backup:
self._inputs_path = ''
self._exclude_path = ''
self._remote = False
self._ssh = None
self._ssh: Optional[paramiko.client.SSHClient] = None
self._password_auth = False
self._password = None
self._removed_count = 0
def check_params(self, homedir: str = '') -> int:
"""Check if parameters for the backup are valid"""
if self.inputs is None or len(self.inputs) == 0:
if len(self.inputs) == 0:
logger.info(
'No existing files or directories specified for backup. Nothing to do')
return 1
if self.output is None:
if self.output == '':
logger.critical(
'No output path specified. Use -o argument or specify output path in configuration file')
@@ -180,37 +181,63 @@ class Backup:
self._remote = True
if self._remote:
self._ssh = self._ssh_connect(homedir)
try:
self._ssh = self._ssh_connect(homedir)
except OSError as e:
# Unknown hostname, connection refused, unreachable host or timeout
logger.critical('Can\'t connect to the server %s', self.ssh_host)
logger.critical(e)
return 5
if self._ssh is None:
return 5
_, stdout, _ = self._ssh.exec_command(
f'if [ -d "{self.output}" ]; then echo "ok"; fi')
output = self._resolve_remote_path(self.output)
output = stdout.read().decode('utf-8').strip()
if output != 'ok':
if output is None:
logger.critical('Output path for backup does not exist')
return 2
self.output = output
else:
if not os.path.isdir(self.output):
logger.critical('Output path for backup does not exist')
return 2
self.output = os.path.abspath(self.output)
if self.keep is None:
self.keep = -1
self.output = os.path.abspath(self.output)
return 0
def _resolve_remote_path(self, path: str) -> Optional[str]:
"""Return the absolute path of a directory on the server, or None if it doesn't exist"""
assert self._ssh is not None
# Leave a leading '~' unquoted, so that the server's shell expands it to the remote home.
# Relative paths are resolved from the remote home, where the command starts
if path == '~' or path.startswith('~/'):
target = f'~/{shlex.quote(path[2:])}'
else:
target = shlex.quote(path)
_, stdout, _ = self._ssh.exec_command(f'cd -- {target} && pwd')
output = stdout.read().decode('utf-8').strip().split('\n')[-1]
if not output.startswith('/'):
return None
return output
# Function to create the actual backup directory
def define_backup_dir(self) -> None:
"""Define the actual backup dir"""
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
# Avoid ':', which is not allowed on FAT/exFAT filesystems.
# Backups created with the old format ('%Y-%m-%d %H:%M:%S') still sort correctly
now = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
self._output_dir = f'{self.output}/simple_backup/{now}'
if self._remote:
@@ -222,10 +249,23 @@ class Backup:
if self._remote:
assert self._ssh is not None
_, stdout, _ = self._ssh.exec_command(
f'ls {self.output}/simple_backup')
backup_dir = shlex.quote(self.output + '/simple_backup/')
dirs = stdout.read().decode('utf-8').strip().split('\n')
# List only directories, and nothing (without errors) if no backup was made yet
_, stdout, stderr = self._ssh.exec_command(
f'if [ -d {backup_dir} ]; then find {backup_dir} -mindepth 1 -maxdepth 1 -type d; fi')
output = stdout.read().decode('utf-8')
err = stderr.read().decode('utf-8').strip()
if err != '':
# Don't risk removing the wrong directories if the list may be incomplete
logger.error('Cannot list old backups on the server. Old backups will not be removed')
logger.error(err)
return
dirs = [os.path.basename(d) for d in output.split('\n') if d != '']
n_backup = len(dirs)
@@ -241,24 +281,30 @@ class Backup:
for i in range(n_backup - self.keep):
if self.remote_sudo:
_, _, stderr = self._ssh.exec_command(
f'sudo rm -r "{self.output}/simple_backup/{dirs[i]}"')
f'sudo rm -r {shlex.quote(f"{self.output}/simple_backup/{dirs[i]}")}')
else:
_, _, stderr = self._ssh.exec_command(
f'rm -r "{self.output}/simple_backup/{dirs[i]}"')
f'rm -r {shlex.quote(f"{self.output}/simple_backup/{dirs[i]}")}')
err = stderr.read().decode('utf-8').strip().split('\n')[0]
if err != '':
logger.error(
'Error while removing backup %s.', {dirs[i]})
'Error while removing backup %s.', dirs[i])
logger.error(err)
else:
count += 1
else:
try:
dirs = os.listdir(f'{self.output}/simple_backup')
# Only count directories, so that stray files don't count as backups
dirs = [f.name for f in os.scandir(f'{self.output}/simple_backup')
if f.is_dir(follow_symlinks=False)]
except FileNotFoundError:
return
except PermissionError:
logger.error('Cannot list old backups. Permission denied. Old backups will not be removed')
return
n_backup = len(dirs)
@@ -282,6 +328,8 @@ class Backup:
logger.error(
'Error while removing backup %s. Permission denied', dirs[i])
self._removed_count = count
if count == 1:
logger.info('Removed %d backup', count)
elif count > 1:
@@ -296,7 +344,7 @@ class Backup:
sys.exit(5)
_, stdout, _ = self._ssh.exec_command(
f'find {self.output}/simple_backup/ -mindepth 1 -maxdepth 1 -type d | sort')
f'find {shlex.quote(self.output + "/simple_backup/")} -mindepth 1 -maxdepth 1 -type d | sort')
output = stdout.read().decode('utf-8').strip().split('\n')
if output[-1] != '':
@@ -315,10 +363,7 @@ class Backup:
logger.critical(
'Cannot access the backup directory. Permission denied')
try:
_notify('Backup failed (check log for details)')
except NameError:
pass
_notify('Backup failed (check log for details)')
sys.exit(3)
@@ -327,7 +372,7 @@ class Backup:
except IndexError:
logger.info('No previous backups available')
def _ssh_connect(self, homedir: str = '') -> paramiko.client.SSHClient:
def _ssh_connect(self, homedir: str = '') -> Optional[paramiko.client.SSHClient]:
try:
ssh = paramiko.SSHClient()
except NameError:
@@ -335,6 +380,18 @@ class Backup:
return None
connected = None
try:
connected = self._ssh_login(ssh, homedir)
return connected
finally:
# Don't leave a half-open connection (e.g. after failed authentication)
if connected is None:
ssh.close()
def _ssh_login(self, ssh: paramiko.client.SSHClient, homedir: str) -> Optional[paramiko.client.SSHClient]:
try:
ssh.load_host_keys(filename=f'{homedir}/.ssh/known_hosts')
except FileNotFoundError:
@@ -343,16 +400,25 @@ class Backup:
ssh.set_missing_host_key_policy(paramiko.WarningPolicy())
try:
ssh.connect(self.ssh_host, username=self.ssh_user)
# WarningPolicy emits a UserWarning for unknown host keys
with warnings.catch_warnings():
warnings.simplefilter('error', UserWarning)
ssh.connect(self.ssh_host, username=self.ssh_user, timeout=SSH_TIMEOUT)
return ssh
except UserWarning:
k = input(
f'Unknown key for host {self.ssh_host}. Continue anyway? (Y/N) ')
try:
k = input(
f'Unknown key for host {self.ssh_host}. Continue anyway? (Y/N) ')
except EOFError:
# No terminal available (e.g. cron or systemd)
k = ''
if k[0].upper() == 'Y':
if k.strip().upper().startswith('Y'):
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
else:
logger.critical('Unknown host key for %s. Connection aborted', self.ssh_host)
return None
except paramiko.BadHostKeyException as e:
logger.critical('Can\'t connect to the server.')
@@ -363,7 +429,7 @@ class Backup:
pass
try:
ssh.connect(self.ssh_host, username=self.ssh_user)
ssh.connect(self.ssh_host, username=self.ssh_user, timeout=SSH_TIMEOUT)
return ssh
except paramiko.SSHException:
@@ -371,9 +437,13 @@ class Backup:
if self.ssh_keyfile is None:
try:
password = getpass(
f'{self.ssh_user}@{self.ssh_host}\'s password: ')
ssh.connect(self.ssh_host, username=self.ssh_user,
# Fail instead of reading the password with echo if no terminal is available
with warnings.catch_warnings():
warnings.simplefilter('error', GetPassWarning)
password = getpass(
f'{self.ssh_user}@{self.ssh_host}\'s password: ')
ssh.connect(self.ssh_host, username=self.ssh_user, timeout=SSH_TIMEOUT,
password=password)
self._password_auth = True
@@ -391,51 +461,13 @@ class Backup:
return None
pkey = None
pkey = self._load_ssh_key()
if pkey is None:
return None
try:
pkey = RSAKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
password = getpass(
f'Enter passwphrase for key \'{self.ssh_keyfile}\': ')
try:
pkey = RSAKey.from_private_key_file(self.ssh_keyfile, password)
except paramiko.SSHException:
pass
if pkey is None:
try:
pkey = Ed25519Key.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
try:
pkey = Ed25519Key.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException:
pass
if pkey is None:
try:
pkey = ECDSAKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
try:
pkey = ECDSAKey.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException:
pass
if pkey is None:
try:
pkey = DSSKey.from_private_key_file(self.ssh_keyfile)
except paramiko.PasswordRequiredException:
try:
pkey = DSSKey.from_private_key_file(
self.ssh_keyfile, password)
except paramiko.SSHException:
pass
try:
ssh.connect(self.ssh_host, username=self.ssh_user, pkey=pkey)
ssh.connect(self.ssh_host, username=self.ssh_user, timeout=SSH_TIMEOUT, pkey=pkey)
except paramiko.SSHException:
logger.critical('SSH connection to server failed')
@@ -443,6 +475,37 @@ class Backup:
return ssh
def _load_ssh_key(self) -> Optional[paramiko.PKey]:
try:
return paramiko.PKey.from_path(self.ssh_keyfile)
except TypeError:
# Raised when the key is encrypted and no passphrase was given
pass
except (OSError, ValueError, paramiko.UnknownKeyType) as e:
logger.critical('Unable to load SSH key %s', self.ssh_keyfile)
logger.critical(e)
return None
try:
# Fail instead of reading the passphrase with echo if no terminal is available
with warnings.catch_warnings():
warnings.simplefilter('error', GetPassWarning)
passphrase = getpass(
f'Enter passphrase for key \'{self.ssh_keyfile}\': ')
except GetPassWarning as e:
logger.critical('Unable to get passphrase')
logger.critical(e)
return None
try:
return paramiko.PKey.from_path(self.ssh_keyfile, passphrase.encode())
except (ValueError, paramiko.SSHException):
logger.critical('Unable to load SSH key %s, wrong passphrase?', self.ssh_keyfile)
return None
def _returncode_log(self, returncode: int) -> None:
match returncode:
case 2:
@@ -474,7 +537,7 @@ class Backup:
'Rsync error (return code 23) - Partial transfer due to error')
case 24:
logger.warning(
'Rsync error (return code 24) - Partial transfer due to vanished source files')
'Rsync warning (return code 24) - Some source files vanished during the transfer')
case 30:
logger.error(
'Rsync error (return code 30) - Timeout in data send/receive')
@@ -490,25 +553,43 @@ class Backup:
def run(self) -> int:
"""Perform the backup"""
try:
return self._run()
finally:
self._remove_temp_files()
def close(self) -> None:
"""Close the SSH connection, if open"""
if self._ssh is not None:
self._ssh.close()
self._ssh = None
def _remove_temp_files(self) -> None:
for path in [self._inputs_path, self._exclude_path]:
if path != '':
try:
os.remove(path)
except FileNotFoundError:
pass
def _run(self) -> int:
logger.info('Starting backup...')
try:
_notify('Starting backup...')
except NameError:
pass
_notify('Starting backup...')
self.define_backup_dir()
self.find_last_backup()
_, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True)
fd, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True)
count = 0
with open(self._inputs_path, 'w', encoding='utf-8') as fp:
with os.fdopen(fd, 'w', encoding='utf-8') as fp:
for i in self.inputs:
if not os.path.exists(i):
logger.warning('Input %s not found. Skipping', i)
else:
fp.write(i)
# rsync reads --files-from paths relative to the source ('/')
fp.write(os.path.abspath(i))
fp.write('\n')
count += 1
@@ -516,46 +597,46 @@ class Backup:
logger.info(
'No existing files or directories specified for backup. Nothing to do')
try:
_notify('Backup finished. No files copied')
except NameError:
pass
_notify('Backup finished. No files copied')
return 1
_, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True)
fd, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True)
with open(self._exclude_path, 'w', encoding='utf-8') as fp:
if self.exclude is not None:
for e in self.exclude:
fp.write(e)
fp.write('\n')
with os.fdopen(fd, 'w', encoding='utf-8') as fp:
for e in self.exclude:
fp.write(e)
fp.write('\n')
if self.keep != -1 and self._remove_before:
self.remove_old_backups()
# Only look for the last backup now, as it may have been removed above
self.find_last_backup()
logger.info('Copying files. This may take a long time...')
if self._last_backup == '':
rsync = f'/usr/bin/rsync {self.options} --exclude-from={self._exclude_path} ' +\
f'--files-from={self._inputs_path} / "{self._server}{self._output_dir}"'
else:
rsync = f'/usr/bin/rsync {self.options} --link-dest="{self._last_backup}" --exclude-from=' +\
f'{self._exclude_path} --files-from={self._inputs_path} / "{self._server}{self._output_dir}"'
# Build the argument list directly, so paths with spaces or quotes are passed unchanged
args = ['/usr/bin/rsync', *self.options.split()]
euid = os.geteuid()
if self._last_backup != '':
args.append(f'--link-dest={self._last_backup}')
if euid == 0 and self.ssh_keyfile is not None:
rsync = f'{rsync} -e \'ssh -i {self.ssh_keyfile} -o StrictHostKeyChecking=no\''
elif self._password_auth and which('sshpass'):
rsync = f'{rsync} -e \'sshpass -e ssh -l {self.ssh_user} -o StrictHostKeyChecking=no\''
args.extend([f'--exclude-from={self._exclude_path}', f'--files-from={self._inputs_path}',
'/', f'{self._server}{self._output_dir}'])
# rsync splits the -e command itself, so quote the parts that may contain spaces
if self.ssh_keyfile is not None:
ssh = f'ssh -i {shlex.quote(self.ssh_keyfile)} -o StrictHostKeyChecking=accept-new'
elif self._password_auth and self.ssh_user is not None and which('sshpass'):
ssh = f'sshpass -e ssh -l {shlex.quote(self.ssh_user)} -o StrictHostKeyChecking=accept-new'
else:
rsync = f'{rsync} -e \'ssh -o StrictHostKeyChecking=no\''
ssh = 'ssh -o StrictHostKeyChecking=accept-new'
args.extend(['-e', ssh])
if self._remote and self.remote_sudo:
rsync = f'{rsync} --rsync-path="sudo rsync"'
args = shlex.split(rsync)
args.append('--rsync-path=sudo rsync')
with Popen(args, stdin=PIPE, stdout=PIPE, stderr=STDOUT, shell=False) as p:
output: Union[bytes, List[str]]
@@ -585,70 +666,49 @@ class Backup:
else:
logger.error(output)
if self.keep != -1 and not self._remove_before:
self.remove_old_backups()
os.remove(self._inputs_path)
os.remove(self._exclude_path)
if self._remote:
assert self._ssh is not None
_, stdout, _ = self._ssh.exec_command(
f'if [ -d "{self._output_dir}" ]; then echo "ok"; fi')
output = stdout.read().decode('utf-8').strip()
if output == 'ok':
logger.info('Backup completed')
try:
_notify('Backup completed')
except NameError:
pass
if self._removed_count > 0 and returncode not in [0, 24]:
# With --remove-before-backup, old backups are already gone if the new one failed
if self.keep == 0:
message = 'All old backups were removed before the backup, and the new backup may be ' +\
'incomplete. No complete backup is available'
else:
logger.error('Backup failed')
message = 'Old backups were removed before the backup, and the new backup may be incomplete'
try:
_notify('Backup failed (check log for details)')
except NameError:
pass
logger.error(message)
_notify(message)
if self._ssh:
self._ssh.close()
else:
if returncode != 0:
logger.error(
'Some errors occurred while performing the backup')
if self.keep != -1 and not self._remove_before:
# Don't delete old backups if the new one may be incomplete
if returncode in [0, 24]:
self.remove_old_backups()
else:
logger.warning('Backup not completed successfully. Old backups will not be removed')
try:
_notify(
'Some errors occurred while performing the backup. Check log for details')
except NameError:
pass
# Files vanishing during the transfer is normal for a live system (return code 24)
if returncode not in [0, 24]:
logger.error(
'Some errors occurred while performing the backup')
return 4
_notify(
'Some errors occurred while performing the backup. Check log for details')
logger.info('Backup completed')
return 4
try:
_notify('Backup completed')
except NameError:
pass
logger.info('Backup completed')
return 0
_notify('Backup completed')
return 0
def _detect_user() -> Optional[str]:
if os.geteuid() == 0:
return os.getenv('SUDO_USER') or os.getenv('DOAS_USER')
return os.getenv('USER')
def _parse_arguments() -> argparse.Namespace:
euid = os.geteuid()
if euid == 0:
user = os.getenv('SUDO_USER')
else:
user = os.getenv('USER')
homedir = os.path.expanduser(f'~{user}')
parser = argparse.ArgumentParser(prog='simple_backup',
description='Simple backup script written in Python that uses rsync to copy files',
epilog='See simple_backup(1) manpage for full documentation',
@@ -656,7 +716,7 @@ def _parse_arguments() -> argparse.Namespace:
parser.add_argument('-v', '--verbose', action='store_true',
help='More verbose output')
parser.add_argument('-c', '--config', default=f'{homedir}/.config/simple_backup/simple_backup.conf',
parser.add_argument('-c', '--config', default='~/.config/simple_backup/simple_backup.conf',
help='Specify location of configuration file')
parser.add_argument('-i', '--inputs', nargs='+',
help='Paths/files to backup')
@@ -695,19 +755,28 @@ def _parse_arguments() -> argparse.Namespace:
return args
def _expand_inputs(inputs, user: Optional[str] = None):
expanded_inputs = []
def _expand_home(path: str, user: str) -> str:
"""Expand a leading '~' to the home directory of user (other '~' are left as they are)"""
if path == '~' or path.startswith('~/'):
path = f'~{user}{path[1:]}'
return os.path.expanduser(path)
def _expand_inputs(inputs: List[str], user: Optional[str] = None) -> List[str]:
expanded_inputs: List[str] = []
for i in inputs:
if i == '':
continue
if user is not None:
i_ex = glob(os.path.expanduser(i.replace('~', f'~{user}')))
i_ex = glob(_expand_home(i, user))
else:
i_ex = glob(i)
if '~' in i:
if i.startswith('~'):
logger.warning('Cannot expand \'~\'. No user specified')
if len(i_ex) == 0:
@@ -719,16 +788,38 @@ def _expand_inputs(inputs, user: Optional[str] = None):
return expanded_inputs
def _read_config(config_file, user: Optional[str] = None):
config_args = {'inputs': None,
'output': None,
'exclude': None,
'keep': -1,
'ssh_host': None,
'ssh_user': None,
'ssh_keyfile': None,
'remote_sudo': False,
'numeric_ids': False}
class ConfigArgs(TypedDict):
"""Values read from the configuration file"""
inputs: Optional[List[str]]
output: Optional[str]
exclude: Optional[List[str]]
keep: int
ssh_host: Optional[str]
ssh_user: Optional[str]
ssh_keyfile: Optional[str]
remote_sudo: bool
numeric_ids: bool
def _get_bool(config: configparser.ConfigParser, key: str) -> bool:
# Empty values (as in the example config) count as false
if config.get('server', key, fallback='').strip() == '':
return False
return config.getboolean('server', key)
def _read_config(config_file: str, user: Optional[str] = None) -> ConfigArgs:
config_args: ConfigArgs = {'inputs': None,
'output': None,
'exclude': None,
'keep': -1,
'ssh_host': None,
'ssh_user': None,
'ssh_keyfile': None,
'remote_sudo': False,
'numeric_ids': False}
if not os.path.isfile(config_file):
if user is not None:
@@ -739,7 +830,8 @@ def _read_config(config_file, user: Optional[str] = None):
return config_args
config = configparser.ConfigParser()
# Disable interpolation, so that '%' can be used in paths and patterns
config = configparser.ConfigParser(interpolation=None)
config.read(config_file)
section = 'backup'
@@ -751,66 +843,25 @@ def _read_config(config_file, user: Optional[str] = None):
section = 'default'
inputs = config.get(section, 'inputs')
inputs = inputs.split(',')
inputs = _expand_inputs(inputs, user)
inputs = list(set(inputs))
config_args['inputs'] = list(set(_expand_inputs([i.strip() for i in inputs.split(',')], user)))
config_args['inputs'] = inputs
# '~' is expanded later, since for remote backups it refers to the home on the server
config_args['output'] = config.get(section, 'backup_dir')
output = config.get(section, 'backup_dir')
exclude = config.get(section, 'exclude', fallback='')
config_args['exclude'] = [e.strip() for e in exclude.split(',') if e.strip() != '']
if user is not None:
output = os.path.expanduser(output.replace('~', f'~{user}'))
elif user is None and '~' in output:
logger.warning('Cannot expand \'~\', no user specified')
config_args['keep'] = config.getint(section, 'keep', fallback=-1)
config_args['output'] = output
# Read host and user separately, so that a missing one is reported instead of silently
# disabling remote backup. Empty values count as not set
config_args['ssh_host'] = config.get('server', 'ssh_host', fallback='').strip() or None
config_args['ssh_user'] = config.get('server', 'ssh_user', fallback='').strip() or None
try:
exclude = config.get(section, 'exclude')
exclude = exclude.split(',')
except configparser.NoOptionError:
exclude = []
config_args['ssh_keyfile'] = config.get('server', 'ssh_keyfile', fallback='').strip() or None
config_args['exclude'] = exclude
try:
keep = config.getint(section, 'keep')
except configparser.NoOptionError:
keep = -1
config_args['keep'] = keep
try:
ssh_host = config.get('server', 'ssh_host')
ssh_user = config.get('server', 'ssh_user')
except (configparser.NoSectionError, configparser.NoOptionError):
ssh_host = None
ssh_user = None
config_args['ssh_host'] = ssh_host
config_args['ssh_user'] = ssh_user
try:
ssh_keyfile = config.get('server', 'ssh_keyfile')
except (configparser.NoSectionError, configparser.NoOptionError):
ssh_keyfile = None
config_args['ssh_keyfile'] = ssh_keyfile
try:
remote_sudo = config.getboolean('server', 'remote_sudo')
except (configparser.NoSectionError, configparser.NoOptionError):
remote_sudo = False
config_args['remote_sudo'] = remote_sudo
try:
numeric_ids = config.getboolean('server', 'numeric_ids')
except (configparser.NoSectionError, configparser.NoOptionError):
numeric_ids = False
config_args['numeric_ids'] = numeric_ids
config_args['remote_sudo'] = _get_bool(config, 'remote_sudo')
config_args['numeric_ids'] = _get_bool(config, 'numeric_ids')
return config_args
@@ -833,12 +884,20 @@ def _target_uid() -> int | None:
return None
_notify_failed = False
def _notify(text: str) -> None:
global _notify_failed
if dbus is None or _notify_failed:
return
euid = os.geteuid()
uid = _target_uid()
if uid is None:
print('notify: no target uid (neither SUDO_UID nor DOAS_USER set)', file=sys.stderr)
logger.debug('Cannot send desktop notification: user not found')
return
@@ -851,8 +910,10 @@ def _notify(text: str) -> None:
'/org/freedesktop/Notifications')
obj = dbus.Interface(obj, 'org.freedesktop.Notifications')
obj.Notify('simple_backup', 0, '', 'simple_backup', text, [], {'urgency': dbus.Byte(1)}, 10000)
except Exception:
traceback.print_exc()
except Exception as e:
# Report the failure only once (e.g. no session bus available), then stop trying
_notify_failed = True
logger.warning('Cannot send desktop notification: %s', e)
finally:
os.seteuid(euid)
@@ -869,7 +930,7 @@ def simple_backup() -> int:
euid = os.geteuid()
if euid == 0:
user = os.getenv('SUDO_USER')
user = _detect_user()
if user is not None:
homedir = os.path.expanduser(f'~{user}')
@@ -890,28 +951,56 @@ def simple_backup() -> int:
except NameError:
pass
config_file = args.config
# Expand '~' to the home of the user running the backup, not root's
if config_file.startswith('~/') and user is not None:
config_file = f'~{user}{config_file[1:]}'
config_file = os.path.expanduser(config_file)
try:
config_args = _read_config(args.config, user)
except (configparser.NoSectionError, configparser.NoOptionError):
logger.critical('Bad configuration file')
config_args = _read_config(config_file, user)
except (configparser.Error, ValueError) as e:
# configparser.Error: missing sections/options or bad syntax, ValueError: invalid values (e.g. keep=abc)
logger.critical('Bad configuration file: %s', e)
return 6
inputs = args.inputs if args.inputs is not None else config_args['inputs']
output = args.output if args.output is not None else config_args['output']
exclude = args.exclude if args.exclude is not None else config_args['exclude']
keep = args.keep if args.keep is not None else config_args['keep']
ssh_host = args.ssh_host if args.ssh_host is not None else config_args['ssh_host']
ssh_user = args.ssh_user if args.ssh_user is not None else config_args['ssh_user']
if (ssh_host is None) != (ssh_user is None):
logger.critical('Both SSH host and SSH user are needed for remote backup')
return 6
if args.output is not None:
output = args.output
else:
output = config_args['output']
# For remote backups, '~' is expanded on the server
remote = ssh_host is not None and ssh_user is not None
if output is not None and not remote:
if user is not None:
output = _expand_home(output, user)
elif output.startswith('~'):
logger.warning('Cannot expand \'~\', no user specified')
ssh_keyfile = args.keyfile if args.keyfile is not None else config_args['ssh_keyfile']
remote_sudo = args.remote_sudo or config_args['remote_sudo']
if args.rsync_options is None:
rsync_options = ['-a', '-r', '-v', '-h', '-H',
'-X', '-s', '--ignore-missing-args', '--mkpath']
else:
rsync_options = ['-r', '-v']
# Options needed in any case (--mkpath creates the backup directory on the first run)
rsync_options = ['-r', '-v', '--ignore-missing-args', '--mkpath']
if args.rsync_options is None:
rsync_options.extend(['-a', '-h', '-H', '-X', '-s'])
else:
for ro in args.rsync_options:
rsync_options.append(f'-{ro}')
@@ -924,17 +1013,18 @@ def simple_backup() -> int:
if args.numeric_ids or config_args['numeric_ids']:
rsync_options.append('--numeric-ids')
rsync_options = ' '.join(rsync_options)
backup = Backup(inputs, output, exclude, keep, rsync_options, ssh_host, ssh_user, ssh_keyfile,
backup = Backup(inputs, output, exclude, keep, ' '.join(rsync_options), ssh_host, ssh_user, ssh_keyfile,
remote_sudo, remove_before=args.remove_before_backup, verbose=args.verbose)
return_code = backup.check_params(homedir)
try:
return_code = backup.check_params(homedir)
if return_code == 0:
return backup.run()
if return_code == 0:
return backup.run()
return return_code
return return_code
finally:
backup.close()
if __name__ == '__main__':
+30
View File
@@ -0,0 +1,30 @@
"""Shared fixtures for the simple_backup tests"""
import sys
from typing import Callable
import pytest
from simple_backup import simple_backup as sb
@pytest.fixture(autouse=True)
def no_side_effects(monkeypatch: pytest.MonkeyPatch) -> None:
"""Keep tests out of the systemd journal and the desktop notifications"""
if hasattr(sb, 'j_handler'):
sb.logger.removeHandler(sb.j_handler)
monkeypatch.setattr(sb, '_notify', lambda text: None)
@pytest.fixture
def run_cli(monkeypatch: pytest.MonkeyPatch) -> Callable[..., int]:
"""Run simple_backup() with the given command line arguments"""
def run(*argv: str) -> int:
monkeypatch.setattr(sys, 'argv', ['simple_backup', *argv])
return sb.simple_backup()
return run
+318
View File
@@ -0,0 +1,318 @@
"""Tests for combining command line and config values, and for detecting the user"""
import os
import pwd
from pathlib import Path
from typing import Any, Callable, Optional
import pytest
from simple_backup import simple_backup as sb
USER = pwd.getpwuid(os.getuid()).pw_name
HOME = pwd.getpwuid(os.getuid()).pw_dir
UID = os.getuid()
DEFAULT_OPTIONS = '-r -v --ignore-missing-args --mkpath -a -h -H -X -s'
RunCli = Callable[..., int]
class FakeBackup:
"""Records the values simple_backup() passes to Backup"""
instances: list['FakeBackup'] = []
check_code = 0
def __init__(self, inputs: Any, output: Any, exclude: Any, keep: Any, options: str,
ssh_host: Optional[str] = None, ssh_user: Optional[str] = None,
ssh_keyfile: Optional[str] = None, remote_sudo: bool = False,
remove_before: bool = False, verbose: bool = False) -> None:
self.inputs = inputs
self.output = output
self.exclude = exclude
self.keep = keep
self.options = options
self.ssh_host = ssh_host
self.ssh_user = ssh_user
self.ssh_keyfile = ssh_keyfile
self.remote_sudo = remote_sudo
self.remove_before = remove_before
self.verbose = verbose
self.homedir: Optional[str] = None
self.ran = False
self.closed = False
FakeBackup.instances.append(self)
def check_params(self, homedir: str = '') -> int:
self.homedir = homedir
return FakeBackup.check_code
def run(self) -> int:
self.ran = True
return 0
def close(self) -> None:
self.closed = True
@pytest.fixture(autouse=True)
def fake_backup(monkeypatch: pytest.MonkeyPatch) -> None:
FakeBackup.instances = []
FakeBackup.check_code = 0
monkeypatch.setattr(sb, 'Backup', FakeBackup)
@pytest.fixture
def as_user(monkeypatch: pytest.MonkeyPatch) -> None:
"""Run as the current (non-root) user"""
monkeypatch.setattr(os, 'geteuid', lambda: UID if UID != 0 else 1000)
monkeypatch.setenv('USER', USER)
monkeypatch.setenv('HOME', HOME)
@pytest.fixture
def as_root(monkeypatch: pytest.MonkeyPatch) -> None:
"""Run as root, with no sudo/doas variables set"""
monkeypatch.setattr(os, 'geteuid', lambda: 0)
for var in ['SUDO_USER', 'SUDO_UID', 'DOAS_USER']:
monkeypatch.delenv(var, raising=False)
@pytest.fixture
def config_calls(monkeypatch: pytest.MonkeyPatch) -> list[tuple[str, Optional[str]]]:
"""Record the config file path and user instead of reading the file"""
calls: list[tuple[str, Optional[str]]] = []
read_config = sb._read_config
def record(config_file: str, user: Optional[str] = None) -> sb.ConfigArgs:
calls.append((config_file, user))
return read_config('/nonexistent/simple_backup.conf', user)
monkeypatch.setattr(sb, '_read_config', record)
return calls
@pytest.fixture
def config(tmp_path: Path) -> Callable[[str], str]:
"""Write a config file with existing inputs, followed by the given text"""
(tmp_path / 'data').mkdir()
def write(text: str = '') -> str:
config_file = tmp_path / 'simple_backup.conf'
config_file.write_text(f'[backup]\ninputs={tmp_path}/data\n{text}\n')
return str(config_file)
return write
def last_backup() -> FakeBackup:
assert len(FakeBackup.instances) == 1
return FakeBackup.instances[0]
@pytest.mark.usefixtures('as_user')
class TestValues:
def test_config_values(self, run_cli: RunCli, config: Callable[[str], str], tmp_path: Path) -> None:
config_file = config("""backup_dir=/backups
exclude=*.bak
keep=3
[server]
ssh_host=server
ssh_user=user
ssh_keyfile=/keys/id
remote_sudo=true
numeric_ids=true
""")
assert run_cli('-c', config_file) == 0
b = last_backup()
assert b.inputs == [f'{tmp_path}/data']
assert b.output == '/backups'
assert b.exclude == ['*.bak']
assert b.keep == 3
assert (b.ssh_host, b.ssh_user, b.ssh_keyfile) == ('server', 'user', '/keys/id')
assert b.remote_sudo is True
assert b.options == f'{DEFAULT_OPTIONS} --numeric-ids'
assert b.ran and b.closed
def test_cli_overrides_config(self, run_cli: RunCli, config: Callable[[str], str]) -> None:
config_file = config("""backup_dir=/backups
exclude=*.bak
keep=3
[server]
ssh_host=server
ssh_user=user
ssh_keyfile=/keys/id
""")
assert run_cli('-c', config_file, '-i', '/cli/in', '-o', '/cli/out', '-e', '*.tmp', '-k', '0',
'--ssh-host', 'cli-host', '--ssh-user', 'cli-user', '--keyfile', '/cli/key') == 0
b = last_backup()
assert b.inputs == ['/cli/in']
assert b.output == '/cli/out'
assert b.exclude == ['*.tmp']
assert b.keep == 0
assert (b.ssh_host, b.ssh_user, b.ssh_keyfile) == ('cli-host', 'cli-user', '/cli/key')
def test_ssh_host_and_user_from_cli_and_config(self, run_cli: RunCli, config: Callable[[str], str]) -> None:
config_file = config('backup_dir=/backups\n[server]\nssh_user=user')
assert run_cli('-c', config_file, '--ssh-host', 'server') == 0
assert (last_backup().ssh_host, last_backup().ssh_user) == ('server', 'user')
@pytest.mark.parametrize('server, argv', [
('ssh_host=server', []),
('ssh_user=user', []),
('', ['--ssh-host', 'server']),
('', ['--ssh-user', 'user']),
])
def test_only_one_of_ssh_host_and_user(self, run_cli: RunCli, config: Callable[[str], str],
caplog: pytest.LogCaptureFixture, server: str,
argv: list[str]) -> None:
config_file = config(f'backup_dir=/backups\n[server]\n{server}')
assert run_cli('-c', config_file, *argv) == 6
assert FakeBackup.instances == []
assert 'Both SSH host and SSH user are needed' in caplog.text
def test_bad_config(self, run_cli: RunCli, config: Callable[[str], str],
caplog: pytest.LogCaptureFixture) -> None:
config_file = config('backup_dir=/backups\nkeep=abc')
assert run_cli('-c', config_file) == 6
assert FakeBackup.instances == []
assert 'Bad configuration file' in caplog.text
def test_local_output_expanded(self, run_cli: RunCli, config: Callable[[str], str]) -> None:
assert run_cli('-c', config('backup_dir=~/backups')) == 0
assert last_backup().output == f'{HOME}/backups'
def test_remote_output_not_expanded(self, run_cli: RunCli, config: Callable[[str], str]) -> None:
config_file = config('backup_dir=~/backups\n[server]\nssh_host=server\nssh_user=user')
assert run_cli('-c', config_file) == 0
# Expanded on the server
assert last_backup().output == '~/backups'
@pytest.mark.parametrize('argv, options', [
([], DEFAULT_OPTIONS),
(['--rsync-options', 'a', 'l'], '-r -v --ignore-missing-args --mkpath -a -l'),
(['-s', '-z', '--numeric-ids'], f'{DEFAULT_OPTIONS} -c -z --numeric-ids'),
])
def test_rsync_options(self, run_cli: RunCli, config: Callable[[str], str], argv: list[str],
options: str) -> None:
assert run_cli('-c', config('backup_dir=/backups'), *argv) == 0
assert last_backup().options == options
def test_flags(self, run_cli: RunCli, config: Callable[[str], str]) -> None:
assert run_cli('-c', config('backup_dir=/backups'), '--remote-sudo', '--remove-before-backup', '-v') == 0
b = last_backup()
assert b.remote_sudo and b.remove_before and b.verbose
def test_check_params_failure(self, run_cli: RunCli, config: Callable[[str], str]) -> None:
FakeBackup.check_code = 2
assert run_cli('-c', config('backup_dir=/backups')) == 2
b = last_backup()
assert not b.ran
assert b.closed
class TestUser:
@pytest.mark.usefixtures('as_user')
def test_current_user(self, run_cli: RunCli, config_calls: list[tuple[str, Optional[str]]]) -> None:
assert run_cli() == 0
assert config_calls == [(f'{HOME}/.config/simple_backup/simple_backup.conf', USER)]
assert last_backup().homedir == HOME
@pytest.mark.usefixtures('as_root')
def test_explicit_user(self, run_cli: RunCli, config_calls: list[tuple[str, Optional[str]]]) -> None:
assert run_cli('-u', USER, '-c', '~/other.conf') == 0
assert config_calls == [(f'{HOME}/other.conf', USER)]
assert last_backup().homedir == HOME
@pytest.mark.usefixtures('as_root')
@pytest.mark.parametrize('var', ['SUDO_USER', 'DOAS_USER'])
def test_sudo_or_doas_user(self, run_cli: RunCli, config_calls: list[tuple[str, Optional[str]]],
monkeypatch: pytest.MonkeyPatch, var: str) -> None:
monkeypatch.setenv(var, USER)
assert run_cli() == 0
assert config_calls == [(f'{HOME}/.config/simple_backup/simple_backup.conf', USER)]
assert last_backup().homedir == HOME
@pytest.mark.usefixtures('as_root')
def test_root_without_user(self, run_cli: RunCli, config_calls: list[tuple[str, Optional[str]]],
caplog: pytest.LogCaptureFixture) -> None:
assert run_cli() == 0
# '~' is left to expanduser, i.e. $HOME
assert config_calls[0][1] is None
assert last_backup().homedir == ''
assert 'Failed to detect user' in caplog.text
class TestDetectUser:
@pytest.mark.usefixtures('as_root')
def test_root_prefers_sudo_user(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv('SUDO_USER', 'alice')
monkeypatch.setenv('DOAS_USER', 'bob')
assert sb._detect_user() == 'alice'
@pytest.mark.usefixtures('as_root')
def test_root_doas_user(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv('DOAS_USER', 'bob')
assert sb._detect_user() == 'bob'
@pytest.mark.usefixtures('as_root')
def test_root_no_user(self) -> None:
assert sb._detect_user() is None
@pytest.mark.usefixtures('as_user')
def test_non_root(self) -> None:
assert sb._detect_user() == USER
class TestTargetUid:
@pytest.mark.usefixtures('as_user')
def test_non_root(self) -> None:
assert sb._target_uid() == os.geteuid()
@pytest.mark.usefixtures('as_root')
def test_sudo_uid(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv('SUDO_UID', '1234')
assert sb._target_uid() == 1234
@pytest.mark.usefixtures('as_root')
def test_doas_user(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv('DOAS_USER', USER)
assert sb._target_uid() == UID
@pytest.mark.usefixtures('as_root')
def test_unknown_doas_user(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv('DOAS_USER', 'no-such-user-simple-backup')
assert sb._target_uid() is None
@pytest.mark.usefixtures('as_root')
def test_no_user(self) -> None:
assert sb._target_uid() is None
+230
View File
@@ -0,0 +1,230 @@
"""Tests for reading the configuration file and expanding paths"""
import configparser
import os
import pwd
from pathlib import Path
import pytest
from simple_backup import simple_backup as sb
USER = pwd.getpwuid(os.getuid()).pw_name
HOME = pwd.getpwuid(os.getuid()).pw_dir
def write_config(path: Path, text: str) -> str:
config_file = path / 'simple_backup.conf'
config_file.write_text(text)
return str(config_file)
def test_repo_copy_imported() -> None:
repo = Path(__file__).resolve().parents[1]
assert Path(sb.__file__).resolve().is_relative_to(repo)
@pytest.mark.parametrize('path, expected', [
('~', HOME),
('~/docs', f'{HOME}/docs'),
('/abs/~/path', '/abs/~/path'),
('relative', 'relative'),
])
def test_expand_home(path: str, expected: str) -> None:
assert sb._expand_home(path, USER) == expected
def test_expand_inputs_glob(tmp_path: Path) -> None:
for name in ['a.txt', 'b.txt', 'c.log']:
(tmp_path / name).touch()
result = sb._expand_inputs([f'{tmp_path}/*.txt', ''], USER)
assert sorted(result) == [f'{tmp_path}/a.txt', f'{tmp_path}/b.txt']
def test_expand_inputs_no_match(tmp_path: Path, caplog: pytest.LogCaptureFixture) -> None:
assert sb._expand_inputs([f'{tmp_path}/missing'], USER) == []
assert 'No file or directory matching input' in caplog.text
def test_expand_inputs_tilde_without_user(caplog: pytest.LogCaptureFixture) -> None:
sb._expand_inputs(['~/nothing-here'], None)
assert 'Cannot expand' in caplog.text
def test_missing_config(tmp_path: Path, caplog: pytest.LogCaptureFixture) -> None:
config_args = sb._read_config(str(tmp_path / 'missing.conf'), USER)
assert config_args['inputs'] is None
assert config_args['output'] is None
assert config_args['keep'] == -1
assert 'does not exist' in caplog.text
def test_missing_config_without_user(tmp_path: Path, caplog: pytest.LogCaptureFixture) -> None:
sb._read_config(str(tmp_path / 'missing.conf'), None)
assert 'User not specified' in caplog.text
def test_read_config(tmp_path: Path) -> None:
(tmp_path / 'one').mkdir()
(tmp_path / 'two').mkdir()
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}/one, {tmp_path}/two, {tmp_path}/one
backup_dir=~/backups
exclude=*.bak, , *.tmp
keep=3
""")
config_args = sb._read_config(config_file, USER)
assert config_args['inputs'] is not None
assert sorted(config_args['inputs']) == [f'{tmp_path}/one', f'{tmp_path}/two']
# Expanded later, as it may be a path on the server
assert config_args['output'] == '~/backups'
assert config_args['exclude'] == ['*.bak', '*.tmp']
assert config_args['keep'] == 3
assert config_args['ssh_host'] is None
assert config_args['ssh_user'] is None
assert config_args['ssh_keyfile'] is None
assert config_args['remote_sudo'] is False
assert config_args['numeric_ids'] is False
def test_optional_keys_missing(tmp_path: Path) -> None:
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}
backup_dir=/backups
""")
config_args = sb._read_config(config_file, USER)
assert config_args['exclude'] == []
assert config_args['keep'] == -1
def test_old_default_section(tmp_path: Path) -> None:
config_file = write_config(tmp_path, f"""
[default]
inputs={tmp_path}
backup_dir=/backups
""")
config_args = sb._read_config(config_file, USER)
assert config_args['inputs'] == [str(tmp_path)]
assert config_args['output'] == '/backups'
def test_percent_in_values(tmp_path: Path) -> None:
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}
backup_dir=/backups/100%
exclude=%temp%
""")
config_args = sb._read_config(config_file, USER)
assert config_args['output'] == '/backups/100%'
assert config_args['exclude'] == ['%temp%']
def test_server_section(tmp_path: Path) -> None:
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}
backup_dir=backups
[server]
ssh_host=server
ssh_user=user
ssh_keyfile=/keys/id_ed25519
remote_sudo=yes
numeric_ids=true
""")
config_args = sb._read_config(config_file, USER)
assert config_args['ssh_host'] == 'server'
assert config_args['ssh_user'] == 'user'
assert config_args['ssh_keyfile'] == '/keys/id_ed25519'
assert config_args['remote_sudo'] is True
assert config_args['numeric_ids'] is True
def test_empty_server_values(tmp_path: Path) -> None:
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}
backup_dir=/backups
[server]
ssh_host=
ssh_user=
ssh_keyfile=
remote_sudo=
numeric_ids=
""")
config_args = sb._read_config(config_file, USER)
assert config_args['ssh_host'] is None
assert config_args['ssh_user'] is None
assert config_args['ssh_keyfile'] is None
assert config_args['remote_sudo'] is False
assert config_args['numeric_ids'] is False
def test_only_ssh_host(tmp_path: Path) -> None:
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}
backup_dir=/backups
[server]
ssh_host=server
""")
config_args = sb._read_config(config_file, USER)
# Kept, so that the missing user is reported instead of silently backing up locally
assert config_args['ssh_host'] == 'server'
assert config_args['ssh_user'] is None
@pytest.mark.parametrize('line', ['keep=abc', 'remote_sudo=maybe'])
def test_bad_value(tmp_path: Path, line: str) -> None:
section = '[server]' if line.startswith('remote_sudo') else ''
config_file = write_config(tmp_path, f"""
[backup]
inputs={tmp_path}
backup_dir=/backups
{section}
{line}
""")
with pytest.raises(ValueError):
sb._read_config(config_file, USER)
@pytest.mark.parametrize('text', [
'[backup]\nbackup_dir=/backups\n',
'[backup]\ninputs=/tmp\n',
'[other]\ninputs=/tmp\n',
'inputs=/tmp\n',
])
def test_bad_config(tmp_path: Path, text: str) -> None:
config_file = write_config(tmp_path, text)
with pytest.raises(configparser.Error):
sb._read_config(config_file, USER)
+267
View File
@@ -0,0 +1,267 @@
"""Tests for local backups, with real rsync runs and with a fake Popen"""
import os
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Callable, Iterator, Optional
import pytest
from simple_backup import simple_backup as sb
OPTIONS = '-r -v --ignore-missing-args --mkpath -a -h -H -X -s'
@pytest.fixture(autouse=True)
def fake_now(monkeypatch: pytest.MonkeyPatch) -> None:
"""Give each backup a new timestamp, as real runs within the same second would collide"""
def times() -> Iterator[datetime]:
t = datetime(2026, 10, 14, 10, 0, 0)
while True:
yield t
t += timedelta(hours=1)
it = times()
class FakeDatetime:
@staticmethod
def now() -> datetime:
return next(it)
monkeypatch.setattr(sb, 'datetime', FakeDatetime)
@pytest.fixture
def src(tmp_path: Path) -> Path:
src = tmp_path / 'src dir'
src.mkdir()
(src / 'same.txt').write_text('unchanged')
(src / 'changed.txt').write_text('first')
return src
@pytest.fixture
def out(tmp_path: Path) -> Path:
out = tmp_path / 'out dir'
out.mkdir()
return out
def make_backup(out: Path, inputs: list[str], keep: int = -1, exclude: Optional[list[str]] = None,
remove_before: bool = False) -> sb.Backup:
backup = sb.Backup(inputs, str(out), exclude or [], keep, OPTIONS, remove_before=remove_before)
assert backup.check_params() == 0
return backup
def run_backup(out: Path, inputs: list[str], **kwargs: Any) -> int:
return make_backup(out, inputs, **kwargs).run()
def backups(out: Path) -> list[str]:
return sorted(d.name for d in (out / 'simple_backup').iterdir() if d.is_dir() and not d.is_symlink())
def copied(out: Path, backup: str, path: Path) -> Path:
"""Path of a file inside a backup (rsync keeps the full source path)"""
return out / 'simple_backup' / backup / str(path).lstrip('/')
class TestCheckParams:
def test_no_inputs(self, out: Path) -> None:
assert sb.Backup([], str(out), [], -1, OPTIONS).check_params() == 1
def test_no_output(self) -> None:
assert sb.Backup(['/x'], None, [], -1, OPTIONS).check_params() == 2
def test_missing_output(self, tmp_path: Path) -> None:
assert sb.Backup(['/x'], str(tmp_path / 'missing'), [], -1, OPTIONS).check_params() == 2
def test_relative_output(self, out: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.chdir(out.parent)
backup = sb.Backup(['/x'], out.name, [], None, OPTIONS)
assert backup.check_params() == 0
assert backup.output == str(out)
assert backup.keep == -1
class TestRsync:
def test_first_backup(self, src: Path, out: Path, caplog: pytest.LogCaptureFixture) -> None:
assert run_backup(out, [str(src)]) == 0
assert backups(out) == ['2026-10-14_10-00-00']
assert copied(out, '2026-10-14_10-00-00', src / 'same.txt').read_text() == 'unchanged'
assert 'Backup completed' in caplog.text
assert 'rsync: sent' in caplog.text
def test_hard_links(self, src: Path, out: Path) -> None:
assert run_backup(out, [str(src)]) == 0
(src / 'changed.txt').write_text('second')
assert run_backup(out, [str(src)]) == 0
first, second = backups(out)
assert copied(out, first, src / 'same.txt').stat().st_ino == \
copied(out, second, src / 'same.txt').stat().st_ino
assert copied(out, first, src / 'changed.txt').stat().st_ino != \
copied(out, second, src / 'changed.txt').stat().st_ino
assert copied(out, second, src / 'changed.txt').read_text() == 'second'
def test_exclude_and_missing_input(self, src: Path, out: Path, caplog: pytest.LogCaptureFixture) -> None:
(src / 'skip.bak').touch()
assert run_backup(out, [str(src), str(src / 'missing')], exclude=['*.bak']) == 0
(backup,) = backups(out)
assert copied(out, backup, src / 'same.txt').exists()
assert not copied(out, backup, src / 'skip.bak').exists()
assert 'Input' in caplog.text and 'not found' in caplog.text
def test_all_inputs_missing(self, src: Path, out: Path) -> None:
assert run_backup(out, [str(src / 'missing')]) == 1
assert not (out / 'simple_backup').exists()
@pytest.mark.parametrize('keep, remaining', [
(-1, ['2026-10-14_10-00-00', '2026-10-14_11-00-00', '2026-10-14_12-00-00']),
(1, ['2026-10-14_11-00-00', '2026-10-14_12-00-00']),
(0, ['2026-10-14_12-00-00']),
])
def test_keep(self, src: Path, out: Path, keep: int, remaining: list[str]) -> None:
for _ in range(3):
assert run_backup(out, [str(src)], keep=keep) == 0
assert backups(out) == remaining
def test_remove_before(self, src: Path, out: Path) -> None:
assert run_backup(out, [str(src)]) == 0
assert run_backup(out, [str(src)], keep=0, remove_before=True) == 0
(backup,) = backups(out)
assert backup == '2026-10-14_11-00-00'
# Nothing to link against, as the old backup was removed first
assert copied(out, backup, src / 'same.txt').stat().st_nlink == 1
def test_stray_files_and_symlinks(self, src: Path, out: Path) -> None:
assert run_backup(out, [str(src)]) == 0
base = out / 'simple_backup'
(base / 'notes.txt').write_text('keep me')
# Left by old versions; must not count as a backup, nor be used for --link-dest
(base / 'last_backup').symlink_to(base / backups(out)[0])
assert run_backup(out, [str(src)], keep=0) == 0
assert backups(out) == ['2026-10-14_11-00-00']
assert (base / 'notes.txt').read_text() == 'keep me'
assert (base / 'last_backup').is_symlink()
def test_old_folder_name_format(self, src: Path, out: Path) -> None:
assert run_backup(out, [str(src)]) == 0
base = out / 'simple_backup'
(base / '2026-10-14_10-00-00').rename(base / '2026-10-03 20:15:25')
assert run_backup(out, [str(src)]) == 0
assert backups(out) == ['2026-10-03 20:15:25', '2026-10-14_11-00-00']
assert copied(out, '2026-10-03 20:15:25', src / 'same.txt').stat().st_ino == \
copied(out, '2026-10-14_11-00-00', src / 'same.txt').stat().st_ino
assert run_backup(out, [str(src)], keep=0) == 0
assert backups(out) == ['2026-10-14_12-00-00']
def test_cli(self, src: Path, out: Path, tmp_path: Path, run_cli: Callable[..., int]) -> None:
config_file = str(tmp_path / 'missing.conf')
for _ in range(2):
assert run_cli('-c', config_file, '-i', str(src), '-o', str(out), '-k', '0') == 0
assert backups(out) == ['2026-10-14_11-00-00']
class FakePopen:
"""Stands in for rsync: creates the destination and returns a chosen return code"""
returncode = 0
calls: list[list[str]] = []
def __init__(self, args: list[str], **kwargs: Any) -> None:
FakePopen.calls.append(args)
os.makedirs(args[args.index('/') + 1])
def __enter__(self) -> 'FakePopen':
return self
def __exit__(self, *args: Any) -> None:
pass
def communicate(self) -> tuple[bytes, None]:
return b'sending incremental file list\nsent 1 bytes received 2 bytes\ntotal size is 3\n', None
@pytest.fixture
def fake_popen(monkeypatch: pytest.MonkeyPatch) -> type[FakePopen]:
FakePopen.returncode = 0
FakePopen.calls = []
monkeypatch.setattr(sb, 'Popen', FakePopen)
return FakePopen
@pytest.fixture
def old_backups(out: Path) -> list[str]:
names = ['2026-10-01_10-00-00', '2026-10-02_10-00-00']
for name in names:
(out / 'simple_backup' / name).mkdir(parents=True)
return names
class TestReturnCodes:
def test_args(self, src: Path, out: Path, fake_popen: type[FakePopen], old_backups: list[str]) -> None:
assert run_backup(out, [str(src)]) == 0
(args,) = fake_popen.calls
assert args[0] == '/usr/bin/rsync'
assert f'--link-dest={out}/simple_backup/{old_backups[-1]}' in args
assert args[args.index('/') + 1] == f'{out}/simple_backup/2026-10-14_10-00-00'
@pytest.mark.parametrize('returncode, result, removed', [
(0, 0, True),
(24, 0, True),
(23, 4, False),
(2, 4, False),
])
def test_old_backups_removed_only_on_success(self, src: Path, out: Path, fake_popen: type[FakePopen],
old_backups: list[str], caplog: pytest.LogCaptureFixture,
returncode: int, result: int, removed: bool) -> None:
fake_popen.returncode = returncode
assert run_backup(out, [str(src)], keep=0) == result
if removed:
assert backups(out) == ['2026-10-14_10-00-00']
else:
assert backups(out) == [*old_backups, '2026-10-14_10-00-00']
assert 'Old backups will not be removed' in caplog.text
assert f'return code {returncode}' in caplog.text
@pytest.mark.parametrize('keep, message', [
(0, 'No complete backup is available'),
(1, 'Old backups were removed before the backup'),
])
def test_failure_after_remove_before(self, src: Path, out: Path, fake_popen: type[FakePopen],
old_backups: list[str], caplog: pytest.LogCaptureFixture,
keep: int, message: str) -> None:
fake_popen.returncode = 23
assert run_backup(out, [str(src)], keep=keep, remove_before=True) == 4
assert message in caplog.text
+566
View File
@@ -0,0 +1,566 @@
"""Tests for remote backups, with a fake SSH client that runs commands locally"""
import importlib.util
import io
import os
import shlex
import socket
import subprocess
import warnings
from getpass import GetPassWarning
from pathlib import Path
from shutil import which
from typing import Any, Optional
import pytest
from simple_backup import simple_backup as sb
OPTIONS = '-r -v --ignore-missing-args --mkpath -a -h -H -X -s'
needs_paramiko = pytest.mark.skipif(importlib.util.find_spec('paramiko') is None,
reason='paramiko not installed')
needs_ssh_keygen = pytest.mark.skipif(which('ssh-keygen') is None, reason='ssh-keygen not installed')
class FakeSSH:
"""SSHClient stand-in: runs commands with sh in a fake remote home"""
def __init__(self, home: Path, bin_dir: Path) -> None:
self.home = home
self.bin_dir = bin_dir
self.commands: list[str] = []
self.fail_on: Optional[str] = None
self.closed = False
def exec_command(self, command: str) -> tuple[None, io.BytesIO, io.BytesIO]:
self.commands.append(command)
if self.fail_on is not None and self.fail_on in command:
return None, io.BytesIO(b''), io.BytesIO(b'Permission denied\n')
env = {'HOME': str(self.home), 'PATH': f'{self.bin_dir}:{os.environ["PATH"]}'}
p = subprocess.run(['sh', '-c', command], cwd=self.home, env=env, capture_output=True, check=False)
return None, io.BytesIO(p.stdout), io.BytesIO(p.stderr)
def close(self) -> None:
self.closed = True
@pytest.fixture
def remote_home(tmp_path: Path) -> Path:
home = tmp_path / 'remote home'
(home / 'backups').mkdir(parents=True)
return home
@pytest.fixture
def ssh(remote_home: Path, tmp_path: Path) -> FakeSSH:
# A 'sudo' that just runs the command, for --remote-sudo
bin_dir = tmp_path / 'bin'
bin_dir.mkdir()
sudo = bin_dir / 'sudo'
sudo.write_text('#!/bin/sh\nexec "$@"\n')
sudo.chmod(0o755)
return FakeSSH(remote_home, bin_dir)
@pytest.fixture
def connect(monkeypatch: pytest.MonkeyPatch, ssh: FakeSSH) -> FakeSSH:
monkeypatch.setattr(sb.Backup, '_ssh_connect', lambda self, homedir='': ssh)
return ssh
def remote_backup(output: str = '~/backups', keep: int = -1, **kwargs: Any) -> sb.Backup:
return sb.Backup(['/x'], output, [], keep, OPTIONS, ssh_host='server', ssh_user='user', **kwargs)
def connected_backup(output: str = '~/backups', keep: int = -1, **kwargs: Any) -> sb.Backup:
backup = remote_backup(output, keep, **kwargs)
assert backup.check_params() == 0
return backup
def make_backups(remote_home: Path, *names: str) -> Path:
base = remote_home / 'backups' / 'simple_backup'
base.mkdir()
for name in names:
(base / name).mkdir()
return base
def remote_dirs(base: Path) -> list[str]:
return sorted(d.name for d in base.iterdir() if d.is_dir())
@pytest.mark.usefixtures('connect')
class TestRemotePath:
@pytest.mark.parametrize('output', ['~/backups', 'backups', '~/backups/', '{home}/backups'])
def test_resolved(self, remote_home: Path, output: str) -> None:
backup = connected_backup(output.format(home=remote_home))
assert backup.output == f'{remote_home}/backups'
def test_home(self, remote_home: Path) -> None:
assert connected_backup('~').output == str(remote_home)
def test_shell_characters_not_expanded(self, remote_home: Path) -> None:
name = 'a $HOME `touch pwned` b'
(remote_home / name).mkdir()
assert connected_backup(f'~/{name}').output == f'{remote_home}/{name}'
assert not (remote_home / 'pwned').exists()
def test_missing(self, caplog: pytest.LogCaptureFixture) -> None:
assert remote_backup('~/missing').check_params() == 2
assert 'Output path for backup does not exist' in caplog.text
def test_needs_host_and_user(self, tmp_path: Path) -> None:
# Only one of them: a local backup
backup = sb.Backup(['/x'], str(tmp_path), [], -1, OPTIONS, ssh_host='server')
assert backup.check_params() == 0
assert not backup._remote
class TestConnectionErrors:
def test_connection_failed(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(sb.Backup, '_ssh_connect', lambda self, homedir='': None)
assert remote_backup().check_params() == 5
@pytest.mark.parametrize('error', [socket.gaierror(-2, 'Name or service not known'),
ConnectionRefusedError(111, 'Connection refused'),
TimeoutError('timed out')])
def test_os_error(self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture,
error: OSError) -> None:
def fail(self: sb.Backup, homedir: str = '') -> None:
raise error
monkeypatch.setattr(sb.Backup, '_ssh_connect', fail)
assert remote_backup().check_params() == 5
assert 'Can\'t connect to the server server' in caplog.text
def test_paramiko_missing(self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture) -> None:
monkeypatch.delattr(sb, 'paramiko', raising=False)
assert remote_backup().check_params() == 5
assert 'Install paramiko' in caplog.text
@pytest.mark.usefixtures('connect')
class TestRemoveOldBackups:
def test_keep(self, remote_home: Path) -> None:
base = make_backups(remote_home, '2026-10-01 10:00:00', '2026-10-02_10-00-00', '2026-10-03_10-00-00')
(base / 'notes.txt').touch()
# The newest one stands for the backup just made
connected_backup(keep=1).remove_old_backups()
assert remote_dirs(base) == ['2026-10-02_10-00-00', '2026-10-03_10-00-00']
assert (base / 'notes.txt').exists()
def test_remove_before(self, remote_home: Path) -> None:
base = make_backups(remote_home, '2026-10-01_10-00-00', '2026-10-02_10-00-00')
connected_backup(keep=0, remove_before=True).remove_old_backups()
assert remote_dirs(base) == []
assert base.is_dir()
def test_remote_sudo(self, remote_home: Path, ssh: FakeSSH) -> None:
base = make_backups(remote_home, '2026-10-01_10-00-00', '2026-10-02_10-00-00')
connected_backup(keep=0, remote_sudo=True).remove_old_backups()
assert remote_dirs(base) == ['2026-10-02_10-00-00']
assert [c for c in ssh.commands if 'rm -r' in c][0].startswith('sudo rm -r ')
@pytest.mark.parametrize('names', [[], None])
def test_nothing_to_remove(self, remote_home: Path, names: Optional[list[str]]) -> None:
if names is not None:
make_backups(remote_home, *names)
connected_backup(keep=0, remove_before=True).remove_old_backups()
# The backup folder itself must never be removed
assert (remote_home / 'backups').is_dir()
assert (remote_home / 'backups' / 'simple_backup').is_dir() == (names is not None)
def test_list_error(self, remote_home: Path, ssh: FakeSSH, caplog: pytest.LogCaptureFixture) -> None:
base = make_backups(remote_home, '2026-10-01_10-00-00', '2026-10-02_10-00-00')
backup = connected_backup(keep=0)
ssh.fail_on = 'find'
backup.remove_old_backups()
assert remote_dirs(base) == ['2026-10-01_10-00-00', '2026-10-02_10-00-00']
assert 'Cannot list old backups on the server' in caplog.text
def test_remove_error(self, remote_home: Path, ssh: FakeSSH, caplog: pytest.LogCaptureFixture) -> None:
make_backups(remote_home, '2026-10-01_10-00-00', '2026-10-02_10-00-00')
backup = connected_backup(keep=0)
ssh.fail_on = 'rm -r'
backup.remove_old_backups()
assert 'Error while removing backup 2026-10-01_10-00-00' in caplog.text
assert backup._removed_count == 0
@pytest.mark.usefixtures('connect')
class TestFindLastBackup:
def test_newest(self, remote_home: Path) -> None:
base = make_backups(remote_home, '2026-10-01 10:00:00', '2026-10-02_10-00-00')
(base / 'zzz.txt').touch()
backup = connected_backup()
backup.find_last_backup()
assert backup._last_backup == f'{base}/2026-10-02_10-00-00'
@pytest.mark.parametrize('create', [True, False])
def test_none(self, remote_home: Path, caplog: pytest.LogCaptureFixture, create: bool) -> None:
if create:
make_backups(remote_home)
backup = connected_backup()
backup.find_last_backup()
assert backup._last_backup == ''
assert 'No previous backups available' in caplog.text
def test_no_connection(self) -> None:
backup = connected_backup()
backup._ssh = None
with pytest.raises(SystemExit) as e:
backup.find_last_backup()
assert e.value.code == 5
class RecordPopen:
"""Records the rsync arguments instead of running rsync"""
returncode = 0
calls: list[list[str]] = []
def __init__(self, args: list[str], **kwargs: Any) -> None:
RecordPopen.calls.append(args)
def __enter__(self) -> 'RecordPopen':
return self
def __exit__(self, *args: Any) -> None:
pass
def communicate(self) -> tuple[bytes, None]:
return b'sent 1 bytes received 2 bytes\ntotal size is 3\n', None
@pytest.mark.usefixtures('connect')
class TestRsyncArgs:
@pytest.fixture(autouse=True)
def record_popen(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
RecordPopen.returncode = 0
RecordPopen.calls = []
monkeypatch.setattr(sb, 'Popen', RecordPopen)
# The input must exist locally
(tmp_path / 'input').mkdir()
def run(self, tmp_path: Path, **kwargs: Any) -> tuple[int, list[str]]:
backup = connected_backup(**kwargs)
backup.inputs = [str(tmp_path / 'input')]
returncode = backup.run()
(args,) = RecordPopen.calls
return returncode, args
@staticmethod
def ssh_command(args: list[str]) -> str:
return args[args.index('-e') + 1]
def test_default(self, tmp_path: Path, remote_home: Path) -> None:
returncode, args = self.run(tmp_path)
assert returncode == 0
assert args[args.index('/') + 1].startswith(f'user@server:{remote_home}/backups/simple_backup/')
assert self.ssh_command(args) == 'ssh -o StrictHostKeyChecking=accept-new'
assert not any(a.startswith('--rsync-path') for a in args)
assert not any(a.startswith('--link-dest') for a in args)
def test_link_dest(self, tmp_path: Path, remote_home: Path) -> None:
base = make_backups(remote_home, '2026-10-01_10-00-00')
_, args = self.run(tmp_path)
assert f'--link-dest={base}/2026-10-01_10-00-00' in args
def test_keyfile_with_spaces(self, tmp_path: Path) -> None:
_, args = self.run(tmp_path, ssh_keyfile='/keys/my key')
# rsync splits the -e command like a shell
assert shlex.split(self.ssh_command(args)) == ['ssh', '-i', '/keys/my key', '-o',
'StrictHostKeyChecking=accept-new']
def test_remote_sudo(self, tmp_path: Path) -> None:
_, args = self.run(tmp_path, remote_sudo=True)
assert '--rsync-path=sudo rsync' in args
def test_password_auth(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(sb, 'which', lambda cmd: f'/usr/bin/{cmd}')
monkeypatch.setenv('SSHPASS', 'secret')
backup = connected_backup()
backup.inputs = [str(tmp_path / 'input')]
backup._password_auth = True
assert backup.run() == 0
(args,) = RecordPopen.calls
assert shlex.split(self.ssh_command(args)) == ['sshpass', '-e', 'ssh', '-l', 'user', '-o',
'StrictHostKeyChecking=accept-new']
# The password must not stay in the environment
assert 'SSHPASS' not in os.environ
def test_failure(self, tmp_path: Path, remote_home: Path) -> None:
base = make_backups(remote_home, '2026-10-01_10-00-00')
RecordPopen.returncode = 23
returncode, _ = self.run(tmp_path, keep=0)
assert returncode == 4
assert remote_dirs(base) == ['2026-10-01_10-00-00']
class FakeClient:
"""paramiko.SSHClient stand-in; connect() takes its outcomes from a list"""
def __init__(self, outcomes: list[Optional[BaseException]]) -> None:
self.outcomes = outcomes
self.connects: list[dict[str, Any]] = []
self.policies: list[str] = []
self.known_hosts: Optional[str] = None
self.closed = False
def load_host_keys(self, filename: str) -> None:
self.known_hosts = filename
if not os.path.isfile(filename):
raise FileNotFoundError(filename)
def set_missing_host_key_policy(self, policy: Any) -> None:
self.policies.append(type(policy).__name__)
def connect(self, hostname: str, **kwargs: Any) -> None:
self.connects.append(kwargs)
outcome = self.outcomes.pop(0)
if isinstance(outcome, UserWarning):
# Like WarningPolicy for an unknown host key
warnings.warn(outcome)
elif outcome is not None:
raise outcome
def close(self) -> None:
self.closed = True
@pytest.fixture
def client(monkeypatch: pytest.MonkeyPatch) -> FakeClient:
fake = FakeClient([])
monkeypatch.setattr(sb.paramiko, 'SSHClient', lambda: fake)
def no_terminal(prompt: str = '') -> str:
raise AssertionError(f'Unexpected prompt: {prompt}')
monkeypatch.setattr(sb, 'getpass', no_terminal)
monkeypatch.setattr('builtins.input', no_terminal)
return fake
def ssh_error() -> Exception:
return sb.paramiko.SSHException('Authentication failed')
@needs_paramiko
class TestLogin:
def test_agent(self, client: FakeClient, tmp_path: Path) -> None:
client.outcomes = [None]
assert remote_backup()._ssh_connect(str(tmp_path)) is client
assert client.known_hosts == f'{tmp_path}/.ssh/known_hosts'
assert client.connects[0]['timeout'] == sb.SSH_TIMEOUT
assert not client.closed
@pytest.mark.parametrize('answer, connected', [('y', True), (' Yes', True), ('n', False), ('', False)])
def test_unknown_host_key(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch,
answer: str, connected: bool) -> None:
client.outcomes = [UserWarning('Unknown host key'), None]
monkeypatch.setattr('builtins.input', lambda prompt: answer)
result = remote_backup()._ssh_connect()
assert (result is client) == connected
assert client.closed != connected
if connected:
assert client.policies == ['WarningPolicy', 'AutoAddPolicy']
def test_unknown_host_key_no_terminal(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch) -> None:
def eof(prompt: str) -> str:
raise EOFError
client.outcomes = [UserWarning('Unknown host key')]
monkeypatch.setattr('builtins.input', eof)
assert remote_backup()._ssh_connect() is None
assert client.closed
def test_bad_host_key(self, client: FakeClient, caplog: pytest.LogCaptureFixture) -> None:
class Key:
def get_base64(self) -> str:
return 'AAAA'
client.outcomes = [sb.paramiko.BadHostKeyException('server', Key(), Key())]
assert remote_backup()._ssh_connect() is None
assert client.closed
assert 'Can\'t connect to the server' in caplog.text
def test_password(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch) -> None:
client.outcomes = [ssh_error(), ssh_error(), None]
monkeypatch.setattr(sb, 'getpass', lambda prompt: 'secret')
monkeypatch.setenv('SSHPASS', '')
backup = remote_backup()
assert backup._ssh_connect() is client
assert client.connects[-1]['password'] == 'secret'
assert backup._password_auth
assert os.environ['SSHPASS'] == 'secret'
def test_wrong_password(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch) -> None:
client.outcomes = [ssh_error(), ssh_error(), ssh_error()]
monkeypatch.setattr(sb, 'getpass', lambda prompt: 'wrong')
assert remote_backup()._ssh_connect() is None
assert client.closed
def test_password_no_terminal(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture) -> None:
def no_tty(prompt: str) -> str:
warnings.warn('Can not control echo on the terminal.', GetPassWarning)
return ''
client.outcomes = [ssh_error(), ssh_error()]
monkeypatch.setattr(sb, 'getpass', no_tty)
assert remote_backup()._ssh_connect() is None
assert 'Unable to get password' in caplog.text
@pytest.mark.parametrize('outcome, connected', [(None, True), ('error', False)])
def test_keyfile(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch,
outcome: Optional[str], connected: bool) -> None:
pkey = object()
client.outcomes = [ssh_error(), ssh_error(), ssh_error() if outcome else None]
monkeypatch.setattr(sb.Backup, '_load_ssh_key', lambda self: pkey)
assert (remote_backup(ssh_keyfile='/keys/id')._ssh_connect() is client) == connected
assert client.connects[-1]['pkey'] is pkey
def test_keyfile_not_loaded(self, client: FakeClient, monkeypatch: pytest.MonkeyPatch) -> None:
client.outcomes = [ssh_error(), ssh_error()]
monkeypatch.setattr(sb.Backup, '_load_ssh_key', lambda self: None)
assert remote_backup(ssh_keyfile='/keys/id')._ssh_connect() is None
assert len(client.connects) == 2
assert client.closed
@pytest.fixture(scope='module')
def keys(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Real keys made by ssh-keygen; encrypted ones use the passphrase 'secret'"""
key_dir = tmp_path_factory.mktemp('keys')
key_types = {'ed25519': ['-t', 'ed25519'], 'ecdsa': ['-t', 'ecdsa'],
'rsa': ['-t', 'rsa', '-b', '2048'], 'rsa_pem': ['-t', 'rsa', '-b', '2048', '-m', 'PEM']}
for name, args in key_types.items():
for suffix, passphrase in [('', ''), ('_enc', 'secret')]:
subprocess.run(['ssh-keygen', '-q', *args, '-N', passphrase, '-f', str(key_dir / f'{name}{suffix}')],
check=True)
return key_dir
@needs_paramiko
@needs_ssh_keygen
class TestLoadKey:
@pytest.fixture(autouse=True)
def prompts(self, monkeypatch: pytest.MonkeyPatch) -> list[str]:
asked: list[str] = []
def answer(prompt: str) -> str:
asked.append(prompt)
return self.passphrase
self.passphrase = 'secret'
monkeypatch.setattr(sb, 'getpass', answer)
return asked
@pytest.mark.parametrize('name', ['ed25519', 'ecdsa', 'rsa', 'rsa_pem'])
def test_unencrypted(self, keys: Path, prompts: list[str], name: str) -> None:
key = remote_backup(ssh_keyfile=str(keys / name))._load_ssh_key()
assert isinstance(key, sb.paramiko.PKey)
assert prompts == []
@pytest.mark.parametrize('name', ['ed25519_enc', 'ecdsa_enc', 'rsa_enc', 'rsa_pem_enc'])
def test_encrypted(self, keys: Path, prompts: list[str], name: str) -> None:
key = remote_backup(ssh_keyfile=str(keys / name))._load_ssh_key()
assert isinstance(key, sb.paramiko.PKey)
assert len(prompts) == 1
@pytest.mark.parametrize('name', ['ed25519_enc', 'rsa_pem_enc'])
def test_wrong_passphrase(self, keys: Path, caplog: pytest.LogCaptureFixture, name: str) -> None:
self.passphrase = 'wrong'
assert remote_backup(ssh_keyfile=str(keys / name))._load_ssh_key() is None
assert 'wrong passphrase?' in caplog.text
def test_no_terminal(self, keys: Path, monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture) -> None:
def no_tty(prompt: str) -> str:
warnings.warn('Can not control echo on the terminal.', GetPassWarning)
return ''
monkeypatch.setattr(sb, 'getpass', no_tty)
assert remote_backup(ssh_keyfile=str(keys / 'ed25519_enc'))._load_ssh_key() is None
assert 'Unable to get passphrase' in caplog.text
@pytest.mark.parametrize('content', [None, 'not a key\n'])
def test_bad_file(self, tmp_path: Path, caplog: pytest.LogCaptureFixture, content: Optional[str]) -> None:
keyfile = tmp_path / 'id'
if content is not None:
keyfile.write_text(content)
assert remote_backup(ssh_keyfile=str(keyfile))._load_ssh_key() is None
assert 'Unable to load SSH key' in caplog.text