Author: | The Borg Collective |
---|---|
Date: | 2017-02-05 |
Manual section: | 1 |
Manual group: | borg backup tool |
borg [common options] <command> [options] [arguments]
BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption.
The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets.
Borg stores a set of files in an archive. A repository is a collection of archives. The format of repositories is Borg-specific. Borg does not distinguish archives from each other in any way other than their name, it does not matter when or where archives were created (e.g. different hosts).
Before a backup can be made a repository has to be initialized:
$ borg init --encryption=repokey /path/to/repo
Backup the ~/src
and ~/Documents
directories into an archive called
Monday:
$ borg create /path/to/repo::Monday ~/src ~/Documents
The next day create a new archive called Tuesday:
$ borg create --stats /path/to/repo::Tuesday ~/src ~/Documents
This backup will be a lot quicker and a lot smaller since only new never
before seen data is stored. The --stats
option causes Borg to
output statistics about the newly created archive such as the amount of unique
data (not shared with other archives):
------------------------------------------------------------------------------
Archive name: Tuesday
Archive fingerprint: bd31004d58f51ea06ff735d2e5ac49376901b21d58035f8fb05dbf866566e3c2
Time (start): Tue, 2016-02-16 18:15:11
Time (end): Tue, 2016-02-16 18:15:11
Duration: 0.19 seconds
Number of files: 127
------------------------------------------------------------------------------
Original size Compressed size Deduplicated size
This archive: 4.16 MB 4.17 MB 26.78 kB
All archives: 8.33 MB 8.34 MB 4.19 MB
Unique chunks Total chunks
Chunk index: 132 261
------------------------------------------------------------------------------
List all archives in the repository:
$ borg list /path/to/repo
Monday Mon, 2016-02-15 19:14:44
Tuesday Tue, 2016-02-16 19:15:11
List the contents of the Monday archive:
$ borg list /path/to/repo::Monday
drwxr-xr-x user group 0 Mon, 2016-02-15 18:22:30 home/user/Documents
-rw-r--r-- user group 7961 Mon, 2016-02-15 18:22:30 home/user/Documents/Important.doc
...
Restore the Monday archive by extracting the files relative to the current directory:
$ borg extract /path/to/repo::Monday
Recover disk space by manually deleting the Monday archive:
$ borg delete /path/to/repo::Monday
Note
Borg is quiet by default (it works on WARNING log level).
You can use options like --progress
or --list
to get specific
reports during command execution. You can also add the -v
(or
--verbose
or --info
) option to adjust the log level to INFO to
get other informational messages.
Borg only supports taking options (-s
and --progress
in the example)
to the left or right of all positional arguments (repo::archive
and path
in the example), but not in between them:
borg create -s --progress repo::archive path # good and preferred
borg create repo::archive path -s --progress # also works
borg create -s repo::archive path --progress # works, but ugly
borg create repo::archive -s --progress path # BAD
This is due to a problem in the argparse module: http://bugs.python.org/issue15112
Local filesystem (or locally mounted network filesystem):
/path/to/repo
- filesystem path to repo directory, absolute path
path/to/repo
- filesystem path to repo directory, relative path
Also, stuff like ~/path/to/repo
or ~other/path/to/repo
works (this is
expanded by your shell).
Note: you may also prepend a file://
to a filesystem path to get URL style.
Remote repositories accessed via ssh user@host:
user@host:/path/to/repo
- remote repo, absolute path
ssh://user@host:port/path/to/repo
- same, alternative syntax, port can be given
Remote repositories with relative paths can be given using this syntax:
user@host:path/to/repo
- path relative to current directory
user@host:~/path/to/repo
- path relative to user’s home directory
user@host:~other/path/to/repo
- path relative to other’s home directory
Note: giving user@host:/./path/to/repo
or user@host:/~/path/to/repo
or
user@host:/~other/path/to/repo
is also supported, but not required here.
Remote repositories with relative paths, alternative syntax with port:
ssh://user@host:port/./path/to/repo
- path relative to current directory
ssh://user@host:port/~/path/to/repo
- path relative to user’s home directory
ssh://user@host:port/~other/path/to/repo
- path relative to other’s home directory
If you frequently need the same repo URL, it is a good idea to set the
BORG_REPO
environment variable to set a default for the repo URL:
export BORG_REPO='ssh://user@host:port/path/to/repo'
Then just leave away the repo URL if only a repo URL is needed and you want to use the default - it will be read from BORG_REPO then.
Use ::
syntax to give the repo URL when syntax requires giving a positional
argument for the repo (e.g. borg mount :: /mnt
).
Many commands want either a repository (just give the repo URL, see above) or
an archive location, which is a repo URL followed by ::archive_name
.
Archive names must not contain the /
(slash) character. For simplicity,
maybe also avoid blanks or other characters that have special meaning on the
shell or in a filesystem (borg mount will use the archive name as directory
name).
If you have set BORG_REPO (see above) and an archive location is needed, use
::archive_name
- the repo URL part is then read from BORG_REPO.
Borg writes all log output to stderr by default. But please note that something showing up on stderr does not indicate an error condition just because it is on stderr. Please check the log levels of the messages and the return code of borg for determining error, warning or success conditions.
If you want to capture the log output to a file, just redirect it:
borg create repo::archive myfiles 2>> logfile
Custom logging configurations can be implemented via BORG_LOGGING_CONF.
The log level of the builtin logging configuration defaults to WARNING.
This is because we want Borg to be mostly silent and only output
warnings, errors and critical messages, unless output has been requested
by supplying an option that implies output (e.g. --list
or --progress
).
Log levels: DEBUG < INFO < WARNING < ERROR < CRITICAL
Use --debug
to set DEBUG log level -
to get debug, info, warning, error and critical level output.
Use --info
(or -v
or --verbose
) to set INFO log level -
to get info, warning, error and critical level output.
Use --warning
(default) to set WARNING log level -
to get warning, error and critical level output.
Use --error
to set ERROR log level -
to get error and critical level output.
Use --critical
to set CRITICAL log level -
to get critical level output.
While you can set misc. log levels, do not expect that every command will give different output on different log levels - it’s just a possibility.
Warning
Options --critical
and --error
are provided for completeness,
their usage is not recommended as you might miss important information.
Borg can exit with the following return codes (rc):
Return code | Meaning |
---|---|
0 | success (logged as INFO) |
1 | warning (operation reached its normal end, but there were warnings – you should check the log, logged as WARNING) |
2 | error (like a fatal error, a local or remote exception, the operation did not reach its normal end, logged as ERROR) |
128+N | killed by signal N (e.g. 137 == kill -9) |
If you use --show-rc
, the return code is also logged at the indicated
level as the last log entry.
Borg uses some environment variables for automation:
::archive
. If a command needs a repository parameter, you
can either leave it away or abbreviate as ::
, if a positional parameter is required.borg change-passphrase
.borg debug info
).
If this is not the case or you do not want Borg to automatically remove stale locks,
set this to no.ssh
. This can be used to specify ssh options, such as
a custom identity file ssh -i /path/to/private/key
. See man ssh
for other options.--remote-path PATH
commandline option overrides the environment variable.Note: answers are case sensitive. setting an invalid answer value might either give the default answer or ask you interactively, depending on whether retries are allowed (they by default are allowed). So please test your scripts interactively before making them a non-interactive script.
Please note:
We strongly recommend against using Borg (or any other database-like software) on non-journaling file systems like FAT, since it is not possible to assume any consistency in case of power failures (or a sudden disconnect of an external drive or similar failures).
While Borg uses a data store that is resilient against these failures when used on journaling file systems, it is not possible to guarantee this with some hardware – independent of the software used. We don’t know a list of affected hardware.
If you are suspicious whether your Borg repository is still consistent
and readable after one of the failures mentioned above occurred, run
borg check --verify-data
to make sure it is consistent.
Requirements for Borg repository file systems
--inplace
To display quantities, Borg takes care of respecting the
usual conventions of scale. Disk sizes are displayed in decimal, using powers of ten (so
kB
means 1000 bytes). For memory usage, binary prefixes are used, and are
indicated using the IEC binary prefixes,
using powers of two (so KiB
means 1024 bytes).
We format date and time conforming to ISO-8601, that is: YYYY-MM-DD and HH:MM:SS (24h clock).
For more information about that, see: https://xkcd.com/1179/
Unless otherwise noted, we display local date and time. Internally, we store and process date and time as UTC.
Borg might use a lot of resources depending on the size of the data set it is dealing with.
If one uses Borg in a client/server way (with a ssh: repository), the resource usage occurs in part on the client and in another part on the server.
If one uses Borg as a single process (with a filesystem repo), all the resource usage occurs in that one process, so just add up client + server to get the approximate resource usage.
It won’t go beyond 100% of 1 core as the code is currently single-threaded. Especially higher zlib and lzma compression levels use significant amounts of CPU cycles. Crypto might be cheap on the CPU (if hardware accelerated) or expensive (if not).
It usually doesn’t need much CPU, it just deals with the key/value store (repository) and uses the repository index for that.
borg check: the repository check computes the checksums of all chunks (medium CPU usage) borg delete repo: low CPU usage
ssh://
repo url).
If you use a locally mounted network filesystem, additionally some copy
operations used for transaction support also go over the connection. If
you backup multiple sources to one target repository, additional traffic
happens for cache resynchronization.Besides regular file and directory structures, Borg can preserve
--read-special
mode.
By default the metadata to create them with mknod(2), mkfifo(2) etc. is stored.On some platforms additional features are supported:
Platform | ACLs [5] | xattr [6] | Flags [7] |
---|---|---|---|
Linux | Yes | Yes | Yes [1] |
Mac OS X | Yes | Yes | Yes (all) |
FreeBSD | Yes | Yes | |
OpenBSD | n/a | n/a | |
NetBSD | n/a | No [2] | |
Solaris 11 | No [3] | n/a | |
OpenIndiana | |||
Windows (cygwin) | No [4] | No | No |
Other Unix-like operating systems may work as well, but have not been tested at all.
Note that most of the platform-dependent features also depend on the file system. For example, ntfs-3g on Linux isn’t able to convey NTFS ACLs.
[1] | (1, 2) Only “nodump”, “immutable”, “compressed” and “append” are supported. Feature request #618 for more flags. |
[2] | Feature request #1332 |
[3] | Feature request #1337 |
[4] | Cygwin tries to map NTFS ACLs to permissions with varying degress of success. |
[5] | The native access control list mechanism of the OS. This normally limits access to non-native ACLs. For example, NTFS ACLs aren’t completely accessible on Linux with ntfs-3g. |
[6] | extended attributes; key-value pairs attached to a file, mainly used by the OS. This includes resource forks on Mac OS X. |
[7] | aka BSD flags. The Linux set of flags [1] is portable across platforms. The BSDs define additional flags. |
borg-common(1) for common command line options
borg-init(1), borg-create(1), borg-mount(1), borg-extract(1), borg-list(1), borg-info(1), borg-delete(1), borg-prune(1), borg-recreate(1)
borg-compression(1), borg-patterns(1), borg-placeholders(1)