Bump version to 4.2.0

Use git.shouldnt.work as the clone URL, don't install wheel for
building, as setuptools no longer needs it, list paramiko among the
optional dependencies, and use the correct names of the remote
backup options (--ssh-user, --ssh-host).

PKGBUILD

@@ -4,7 +4,7 @@
 
 pkgname=simple_backup
 pkgdesc='Simple backup script that uses rsync to copy files'
-pkgver=4.1.6
+pkgver=4.1.7
 pkgrel=1
 url="https://git.shouldnt.work/fuxino/${pkgname}"
 arch=('any')
@@ -12,18 +12,16 @@ license=('GPL-3.0-or-later')
 makedepends=('git'
              'python-setuptools'
              'python-build'
-             'python-installer'
-             'python-wheel')
+             'python-installer')
 depends=('python>=3.10'
-         'rsync'
-         'python-dotenv')
+         'rsync')
 optdepends=('python-systemd: use systemd log'
             'python-dbus: for desktop notifications'
             'python-paramiko: for remote backup through ssh')
 conflicts=('simple_backup-git')
 source=(git+${url}?signed#tag=${pkgver})
 validpgpkeys=('7E12BC1FF3B6EDB2CD8053EB981A2B2A3BBF5514')
-sha256sums=('b3b29d9e2e1b7b949e95674d9a401e8eeb0d5f898e8450473dce94f799ee9df3')
+sha256sums=('011cd5546745e413ac7603b64531def2688c2bf6e4ea19876a67547a1db208ce')
 
 build()
 {

README.md

@@ -20,21 +20,19 @@ The script uses rsync to actually run the backup, so you will have to install it
 sudo pacman -Syu rsync
 ```
 
-It's also required to have python-dotenv
-
-Optional dependencies are systemd-python to enable using systemd journal for logging, and dbus-python for desktop notifications.
+Optional dependencies are systemd-python to enable using systemd journal for logging, dbus-python for desktop notifications, and paramiko for remote backup.
 
 ## Install
 To install the program, first clone the repository:
 
 ```bash
-git clone https://github.com/Fuxino/simple_backup.git
+git clone https://git.shouldnt.work/fuxino/simple_backup.git
 ```
 
 Then install the tools required to build the package:
 
 ```bash
-pip install --upgrade build wheel
+pip install --upgrade build
 ```
 
 Finally, run:
@@ -49,11 +47,16 @@ For Arch Linux and Arch-based distros, two packages are available in the AUR (au
 - **simple_backup** for the release version
 - **simple_backup-git** for the git version
 
+## Old backups
+The number of old backups to keep can be set with the --keep (or -k) argument (or in the configuration file).
+Old backups are only removed if rsync completes successfully, so that they are not deleted when the new backup may be incomplete.
+The only rsync error that is ignored is return code 24 (source files vanished during the transfer), which is common when backing up a live system.
+
 ## Remote backup
 > **Warning**
 > This feature is experimental
 
-It's possible to use a remote server as destination for the backup. Just use the --username (or -u) and --host arguments (or set them in the configuration file).
+It's possible to use a remote server as destination for the backup. Just use the --ssh-user and --ssh-host arguments (or set them in the configuration file).
 For this to work, rsync must be installed on the server too.
 
 ### Server authentication

man/simple_backup.1

@@ -1,4 +1,4 @@
-.TH SIMPLE_BACKUP 1 2025-03-30 SIMPLE_BACKUP 4.1.6
+.TH SIMPLE_BACKUP 1 2026-10-08 SIMPLE_BACKUP 4.2.0
 .SH NAME
 simple_backup \- Backup files and folders using rsync
 .SH SYNOPSIS
@@ -52,7 +52,15 @@ or to use single or double quotes around them.
 .B \-o, \-\-output DIR
 Specify the directory where the files will be copied. The program will automatically
 create a subdirectory called \(aqsimple_backup\(aq (if it does not already exist) and
-inside this directory the actual backup directory (using the current date and time).
+inside this directory the actual backup directory (using the current date and time, for example
+\(aq2026\-10\-08_17\-34\-47\(aq).
+.RS
+.P
+Backups are best stored on a Linux filesystem (e.g. ext4 or btrfs). FAT and exFAT filesystems
+are supported, but they don\(aqt support hard links, so each backup will be a full copy instead
+of only storing files that changed since the previous one. File ownership and permissions are
+not preserved either.
+.RE
 .TP
 .B \-e, \-\-exclude FILE|DIR|PATTERN [FILE|...]]
 Specify files, directories or patterns to exclude from the backup. Matching files and directories
@@ -61,6 +69,10 @@ will not be copied. Multiple elements can be specified, in the same way as for t
 .B \-k, \-\-keep N
 Specify how many old backups (so excluding the current one) will be kept. The default behavior
 is to keep them all (same as N=\-1).
+Old backups are only removed if rsync completes successfully, to avoid deleting them when the
+new backup may be incomplete. rsync return code 24 (source files vanished during the transfer)
+is treated as success, while any other error (including return code 23, partial transfer due to error)
+will prevent old backups from being removed. This does not apply when using \-\-remove\-before\-backup.
 .TP
 .B \-u, \-\-user USERNAME
 Explicitly specify the user running the backup (in case it is needed for home directory expansion).
@@ -86,6 +98,9 @@ if saving bandwith is needed.
 Remove old backups (if necessary) before creating the new backup. Useful to free some space
 before performing the backup.
 Default behavior is to remove old backups after successfully completing the backup.
+Note that if the backup fails after old backups have been removed, only the backups kept
+according to \-\-keep will be left, and the new backup may be incomplete. In particular, with
+\-\-keep 0 there will be no complete backup available until the next successful one.
 .TP
 .B \-\-no\-syslog
 Don't use systemd journal for logging.
@@ -100,7 +115,7 @@ Using \-\-rsync\-options it is possible to manually select which options to use.
 .P
     \-a, \-l, \-p, \-t, \-g, \-o, \-c, \-h, \-D, \-H, \-X, \-s
 .P
-Options \-r and \-v are used in any case. Not that options must be specified without dash (\-), for example:
+Options \-r, \-v, \-\-ignore\-missing\-args and \-\-mkpath are used in any case. Note that options must be specified without dash (\-), for example:
 .P
 .EX
     simple_backup \-\-rsync\-options a l p

pyproject.toml

@@ -1,3 +1,48 @@
 [build-system]
-requires = ['setuptools']
+requires = ['setuptools>=77']
 build-backend = 'setuptools.build_meta'
+
+[project]
+name = "simple_backup"
+dynamic = ["version"]
+description = "Simple backup script that uses rsync to copy files"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "GPL-3.0-or-later"
+license-files = ["LICENSE"]
+authors = [
+  { name = "Daniele Fucini", email = "dfucini@gmail.com" }
+]
+maintainers = [
+  { name = "Daniele Fucini", email = "dfucini@gmail.com" }
+]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Environment :: Console",
+  "Natural Language :: English",
+  "Operating System :: POSIX :: Linux",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",
+  "Topic :: System :: Archiving :: Backup",
+]
+
+[project.optional-dependencies]
+JOURNAL = ["systemd-python"]
+NOTIFICATIONS = ["dbus-python"]
+REMOTE = ["paramiko"]
+
+[project.urls]
+Homepage = "https://git.shouldnt.work/fuxino/simple_backup"
+
+[project.scripts]
+simple_backup = "simple_backup.simple_backup:simple_backup"
+
+[tool.setuptools]
+packages = ["simple_backup"]
+
+[tool.setuptools.dynamic]
+version = { attr = "simple_backup.__version__" }

setup.cfg

@@ -1,39 +0,0 @@
-[metadata]
-name = simple_backup
-version = attr: simple_backup.__version__
-description = Simple backup script using rsync
-long_description = file: README.md
-author = Daniele Fucini
-author_email = dfucini@gmail.com
-license = GPL3
-url = https://git.shouldnt.work/fuxino
-classifiers =
-   Development Status :: 4 - Beta
-   Environment :: Console
-   License :: OSI Approved :: GNU General Public License v3 (GPLv3)
-   Natural Language :: English
-   Operating System :: POSIX :: Linux
-   Programming Language :: Python :: 3.10
-   Programming Language :: Python :: 3.11
-   Programming Language :: Python :: 3.12
-   Programming Language :: Python :: 3.13
-   Topic :: System :: Archiving :: Backup
-
-[options]
-packages = simple_backup
-python_requires = >=3.10
-install_requires =
-   python-dotenv
-
-[options.extras_require]
-JOURNAL =
-    systemd-python
-NOTIFICATIONS =
-    dbus-python
-REMOTE =
-    paramiko
-
-[options.entry_points]
-console_scripts =
-   simple_backup = simple_backup.simple_backup:simple_backup
-

simple_backup/__init__.py

@@ -1,3 +1,3 @@
 """Init."""
 
-__version__ = '4.1.6'
+__version__ = '4.2.0'

simple_backup/simple_backup.py

@@ -11,8 +11,10 @@ Classes:
     Backup
 """
 
+# Avoid evaluating annotations at definition time, as they reference optional modules (paramiko)
+from __future__ import annotations
+
 # Import libraries
-import traceback
 import sys
 import os
 import pwd
@@ -32,10 +34,6 @@ from tempfile import mkstemp
 from getpass import GetPassWarning, getpass
 from glob import glob
 
-from dotenv import load_dotenv
-
-warnings.filterwarnings('error')
-
 try:
     import paramiko
     from paramiko import RSAKey, Ed25519Key, ECDSAKey, DSSKey
@@ -50,10 +48,9 @@ except ImportError:
 try:
     import dbus
 except ImportError:
-    pass
+    dbus = None
 
 
-load_dotenv()
 logging.getLogger().setLevel(logging.DEBUG)
 logger = logging.getLogger(os.path.basename(__file__))
 c_handler = StreamHandler()
@@ -160,6 +157,7 @@ class Backup:
         self._ssh = None
         self._password_auth = False
         self._password = None
+        self._removed_count = 0
 
     def check_params(self, homedir: str = '') -> int:
         """Check if parameters for the backup are valid"""
@@ -210,7 +208,9 @@ class Backup:
     # Function to create the actual backup directory
     def define_backup_dir(self) -> None:
         """Define the actual backup dir"""
-        now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
+        # Avoid ':', which is not allowed on FAT/exFAT filesystems.
+        # Backups created with the old format ('%Y-%m-%d %H:%M:%S') still sort correctly
+        now = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
         self._output_dir = f'{self.output}/simple_backup/{now}'
 
         if self._remote:
@@ -282,6 +282,8 @@ class Backup:
                         logger.error(
                             'Error while removing backup %s. Permission denied', dirs[i])
 
+        self._removed_count = count
+
         if count == 1:
             logger.info('Removed %d backup', count)
         elif count > 1:
@@ -315,10 +317,7 @@ class Backup:
                 logger.critical(
                     'Cannot access the backup directory. Permission denied')
 
-                try:
                 _notify('Backup failed (check log for details)')
-                except NameError:
-                    pass
 
                 sys.exit(3)
 
@@ -327,7 +326,7 @@ class Backup:
             except IndexError:
                 logger.info('No previous backups available')
 
-    def _ssh_connect(self, homedir: str = '') -> paramiko.client.SSHClient:
+    def _ssh_connect(self, homedir: str = '') -> Optional[paramiko.client.SSHClient]:
         try:
             ssh = paramiko.SSHClient()
         except NameError:
@@ -343,6 +342,9 @@ class Backup:
         ssh.set_missing_host_key_policy(paramiko.WarningPolicy())
 
         try:
+            # WarningPolicy emits a UserWarning for unknown host keys
+            with warnings.catch_warnings():
+                warnings.simplefilter('error', UserWarning)
                 ssh.connect(self.ssh_host, username=self.ssh_user)
 
             return ssh
@@ -371,8 +373,12 @@ class Backup:
 
         if self.ssh_keyfile is None:
             try:
+                # Fail instead of reading the password with echo if no terminal is available
+                with warnings.catch_warnings():
+                    warnings.simplefilter('error', GetPassWarning)
                     password = getpass(
                         f'{self.ssh_user}@{self.ssh_host}\'s password: ')
+
                 ssh.connect(self.ssh_host, username=self.ssh_user,
                             password=password)
 
@@ -490,25 +496,36 @@ class Backup:
     def run(self) -> int:
         """Perform the backup"""
 
-        logger.info('Starting backup...')
-
         try:
-            _notify('Starting backup...')
-        except NameError:
+            return self._run()
+        finally:
+            self._remove_temp_files()
+
+    def _remove_temp_files(self) -> None:
+        for path in [self._inputs_path, self._exclude_path]:
+            if path != '':
+                try:
+                    os.remove(path)
+                except FileNotFoundError:
                     pass
 
-        self.define_backup_dir()
-        self.find_last_backup()
+    def _run(self) -> int:
+        logger.info('Starting backup...')
 
-        _, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True)
+        _notify('Starting backup...')
+
+        self.define_backup_dir()
+
+        fd, self._inputs_path = mkstemp(prefix='tmp_inputs', text=True)
         count = 0
 
-        with open(self._inputs_path, 'w', encoding='utf-8') as fp:
+        with os.fdopen(fd, 'w', encoding='utf-8') as fp:
             for i in self.inputs:
                 if not os.path.exists(i):
                     logger.warning('Input %s not found. Skipping', i)
                 else:
-                    fp.write(i)
+                    # rsync reads --files-from paths relative to the source ('/')
+                    fp.write(os.path.abspath(i))
                     fp.write('\n')
                     count += 1
 
@@ -516,16 +533,13 @@ class Backup:
             logger.info(
                 'No existing files or directories specified for backup. Nothing to do')
 
-            try:
             _notify('Backup finished. No files copied')
-            except NameError:
-                pass
 
             return 1
 
-        _, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True)
+        fd, self._exclude_path = mkstemp(prefix='tmp_exclude', text=True)
 
-        with open(self._exclude_path, 'w', encoding='utf-8') as fp:
+        with os.fdopen(fd, 'w', encoding='utf-8') as fp:
             if self.exclude is not None:
                 for e in self.exclude:
                     fp.write(e)
@@ -534,6 +548,9 @@ class Backup:
         if self.keep != -1 and self._remove_before:
             self.remove_old_backups()
 
+        # Only look for the last backup now, as it may have been removed above
+        self.find_last_backup()
+
         logger.info('Copying files. This may take a long time...')
 
         if self._last_backup == '':
@@ -585,11 +602,23 @@ class Backup:
                 else:
                     logger.error(output)
 
-        if self.keep != -1 and not self._remove_before:
-            self.remove_old_backups()
+        if self._removed_count > 0 and returncode not in [0, 24]:
+            # With --remove-before-backup, old backups are already gone if the new one failed
+            if self.keep == 0:
+                message = 'All old backups were removed before the backup, and the new backup may be ' +\
+                    'incomplete. No complete backup is available'
+            else:
+                message = 'Old backups were removed before the backup, and the new backup may be incomplete'
 
-        os.remove(self._inputs_path)
-        os.remove(self._exclude_path)
+            logger.error(message)
+            _notify(message)
+
+        if self.keep != -1 and not self._remove_before:
+            # Don't delete old backups if the new one may be incomplete
+            if returncode in [0, 24]:
+                self.remove_old_backups()
+            else:
+                logger.warning('Backup not completed successfully. Old backups will not be removed')
 
         if self._remote:
             assert self._ssh is not None
@@ -602,17 +631,11 @@ class Backup:
             if output == 'ok':
                 logger.info('Backup completed')
 
-                try:
                 _notify('Backup completed')
-                except NameError:
-                    pass
             else:
                 logger.error('Backup failed')
 
-                try:
                 _notify('Backup failed (check log for details)')
-                except NameError:
-                    pass
 
             if self._ssh:
                 self._ssh.close()
@@ -621,34 +644,26 @@ class Backup:
                 logger.error(
                     'Some errors occurred while performing the backup')
 
-                try:
                 _notify(
                     'Some errors occurred while performing the backup. Check log for details')
-                except NameError:
-                    pass
 
                 return 4
 
             logger.info('Backup completed')
 
-            try:
             _notify('Backup completed')
-            except NameError:
-                pass
 
             return 0
 
 
+def _detect_user() -> Optional[str]:
+    if os.geteuid() == 0:
+        return os.getenv('SUDO_USER') or os.getenv('DOAS_USER')
+
+    return os.getenv('USER')
+
+
 def _parse_arguments() -> argparse.Namespace:
-    euid = os.geteuid()
-
-    if euid == 0:
-        user = os.getenv('SUDO_USER')
-    else:
-        user = os.getenv('USER')
-
-    homedir = os.path.expanduser(f'~{user}')
-
     parser = argparse.ArgumentParser(prog='simple_backup',
                                      description='Simple backup script written in Python that uses rsync to copy files',
                                      epilog='See simple_backup(1) manpage for full documentation',
@@ -656,7 +671,7 @@ def _parse_arguments() -> argparse.Namespace:
 
     parser.add_argument('-v', '--verbose', action='store_true',
                         help='More verbose output')
-    parser.add_argument('-c', '--config', default=f'{homedir}/.config/simple_backup/simple_backup.conf',
+    parser.add_argument('-c', '--config', default='~/.config/simple_backup/simple_backup.conf',
                         help='Specify location of configuration file')
     parser.add_argument('-i', '--inputs', nargs='+',
                         help='Paths/files to backup')
@@ -695,6 +710,15 @@ def _parse_arguments() -> argparse.Namespace:
     return args
 
 
+def _expand_home(path: str, user: str) -> str:
+    """Expand a leading '~' to the home directory of user (other '~' are left as they are)"""
+
+    if path == '~' or path.startswith('~/'):
+        path = f'~{user}{path[1:]}'
+
+    return os.path.expanduser(path)
+
+
 def _expand_inputs(inputs, user: Optional[str] = None):
     expanded_inputs = []
 
@@ -703,11 +727,11 @@ def _expand_inputs(inputs, user: Optional[str] = None):
             continue
 
         if user is not None:
-            i_ex = glob(os.path.expanduser(i.replace('~', f'~{user}')))
+            i_ex = glob(_expand_home(i, user))
         else:
             i_ex = glob(i)
 
-            if '~' in i:
+            if i.startswith('~'):
                 logger.warning('Cannot expand \'~\'. No user specified')
 
         if len(i_ex) == 0:
@@ -739,7 +763,8 @@ def _read_config(config_file, user: Optional[str] = None):
 
         return config_args
 
-    config = configparser.ConfigParser()
+    # Disable interpolation, so that '%' can be used in paths and patterns
+    config = configparser.ConfigParser(interpolation=None)
     config.read(config_file)
 
     section = 'backup'
@@ -751,7 +776,7 @@ def _read_config(config_file, user: Optional[str] = None):
         section = 'default'
         inputs = config.get(section, 'inputs')
 
-    inputs = inputs.split(',')
+    inputs = [i.strip() for i in inputs.split(',')]
     inputs = _expand_inputs(inputs, user)
     inputs = list(set(inputs))
 
@@ -760,15 +785,15 @@ def _read_config(config_file, user: Optional[str] = None):
     output = config.get(section, 'backup_dir')
 
     if user is not None:
-        output = os.path.expanduser(output.replace('~', f'~{user}'))
-    elif user is None and '~' in output:
+        output = _expand_home(output, user)
+    elif user is None and output.startswith('~'):
         logger.warning('Cannot expand \'~\', no user specified')
 
     config_args['output'] = output
 
     try:
         exclude = config.get(section, 'exclude')
-        exclude = exclude.split(',')
+        exclude = [e.strip() for e in exclude.split(',') if e.strip() != '']
     except configparser.NoOptionError:
         exclude = []
 
@@ -833,12 +858,20 @@ def _target_uid() -> int | None:
     return None
 
 
+_notify_failed = False
+
+
 def _notify(text: str) -> None:
+    global _notify_failed
+
+    if dbus is None or _notify_failed:
+        return
+
     euid = os.geteuid()
     uid = _target_uid()
 
     if uid is None:
-        print('notify: no target uid (neither SUDO_UID nor DOAS_USER set)', file=sys.stderr)
+        logger.debug('Cannot send desktop notification: user not found')
 
         return
 
@@ -851,8 +884,10 @@ def _notify(text: str) -> None:
                                            '/org/freedesktop/Notifications')
         obj = dbus.Interface(obj, 'org.freedesktop.Notifications')
         obj.Notify('simple_backup', 0, '', 'simple_backup', text, [], {'urgency': dbus.Byte(1)}, 10000)
-    except Exception:
-        traceback.print_exc()
+    except Exception as e:
+        # Report the failure only once (e.g. no session bus available), then stop trying
+        _notify_failed = True
+        logger.warning('Cannot send desktop notification: %s', e)
     finally:
         os.seteuid(euid)
 
@@ -869,7 +904,7 @@ def simple_backup() -> int:
         euid = os.geteuid()
 
         if euid == 0:
-            user = os.getenv('SUDO_USER')
+            user = _detect_user()
 
             if user is not None:
                 homedir = os.path.expanduser(f'~{user}')
@@ -890,10 +925,19 @@ def simple_backup() -> int:
         except NameError:
             pass
 
+    config_file = args.config
+
+    # Expand '~' to the home of the user running the backup, not root's
+    if config_file.startswith('~/') and user is not None:
+        config_file = f'~{user}{config_file[1:]}'
+
+    config_file = os.path.expanduser(config_file)
+
     try:
-        config_args = _read_config(args.config, user)
-    except (configparser.NoSectionError, configparser.NoOptionError):
-        logger.critical('Bad configuration file')
+        config_args = _read_config(config_file, user)
+    except (configparser.Error, ValueError) as e:
+        # configparser.Error: missing sections/options or bad syntax, ValueError: invalid values (e.g. keep=abc)
+        logger.critical('Bad configuration file: %s', e)
 
         return 6
 
@@ -906,12 +950,12 @@ def simple_backup() -> int:
     ssh_keyfile = args.keyfile if args.keyfile is not None else config_args['ssh_keyfile']
     remote_sudo = args.remote_sudo or config_args['remote_sudo']
 
-    if args.rsync_options is None:
-        rsync_options = ['-a', '-r', '-v', '-h', '-H',
-                         '-X', '-s', '--ignore-missing-args', '--mkpath']
-    else:
-        rsync_options = ['-r', '-v']
+    # Options needed in any case (--mkpath creates the backup directory on the first run)
+    rsync_options = ['-r', '-v', '--ignore-missing-args', '--mkpath']
 
+    if args.rsync_options is None:
+        rsync_options.extend(['-a', '-h', '-H', '-X', '-s'])
+    else:
         for ro in args.rsync_options:
             rsync_options.append(f'-{ro}')