Postgres Pro Backup Enterprise : Documentation: 3.0: pg

pg_probackup3
Prev	Up	Chapter 4. Reference	Home	Next

pg_probackup3

pg_probackup3 — utility to manage backup and recovery of Postgres Pro Enterprise database clusters

Synopsis

pg_probackup3 version

pg_probackup3 help [command]

pg_probackup3 init -B backup_dir --skip-if-exists

pg_probackup3 add-instance -B backup_dir -D data_dir --instance instance_name --skip-if-exists

pg_probackup3 del-instance -B backup_dir --instance instance_name

pg_probackup3 set-config -B backup_dir --instance instance_name [option...]

pg_probackup3 set-backup -B backup_dir --instance instance_name -i backup_id [option...]

pg_probackup3 show-config -B backup_dir --instance instance_name [option...]

pg_probackup3 show -B backup_dir [option...]

pg_probackup3 backup -B backup_dir --instance instance_name -b backup_mode [option...]

pg_probackup3 restore -B backup_dir --instance instance_name [option...]

pg_probackup3 validate -B backup_dir [option...]

pg_probackup3 merge -B backup_dir --instance instance_name -i backup_id [option...]

pg_probackup3 delete -B backup_dir --instance instance_name -i backup_id

pg_probackup3 archive-push -B backup_dir --instance instance_name --wal-file-path wal_file_path --wal-file-name wal_file_name [option...]

pg_probackup3 fuse -B backup_dir --mnt-path mnt_path --instance instance_name -i backup_id --cache-swap-size cache_swap_size --cache-dir cache_dir [option...]

pg_probackup3 file-map -B backup_dir --instance instance_name -i backup_id [option...]

pg_probackup3 archive-get -B backup_dir --instance instance_name --wal-file-path wal_file_path --wal-file-name wal_file_name [option...]

pg_probackup3 retention -B backup_dir --instance instance_name { --delete-wal | --delete-expired | --merge-expired } [option...]

Command-Line Reference #

Commands #

This section describes pg_probackup3 commands. Optional parameters are enclosed in square brackets. For detailed parameter descriptions, see the section Options.

version #

pg_probackup3 version

Prints pg_probackup3 version.

If --format=json is specified, the output is printed in the JSON format. This may be needed for native integration with JSON-based applications, such as PPEM. Example of a JSON output:

        pg_probackup3 version
        {
            "pg_probackup3":
            {
                "version": "3.0.0",
            },
            "compressions": [zlib, lz4, zstd]
        }

help #

pg_probackup3 help [command]

Displays the synopsis of pg_probackup3 commands. If one of the pg_probackup3 commands is specified, shows detailed information about the options that can be used with this command.

init #

pg_probackup3 init -B backup_dir [--skip-if-exists] [s3_options] [--help]
[ssh_options] [logging_options] [buffer_options]

Initializes the backup catalog in backup_dir that will store backup copies, WAL archive, and meta information for the backed up database clusters. If the specified backup_dir already exists, it must be empty. Otherwise, pg_probackup3 displays a corresponding error message. You can ignore this error by specifying the --skip-if-exists option. Although the backup will not be initialized, the application will return 0 code.

For more details of the process, refer to the section Initializing a Backup Catalog. For more details of the command settings, see the section Common Options.

add-instance #

pg_probackup3 add-instance -B backup_dir -D data_dir --instance=instance_name
[--skip-if-exists] [s3_options] [ssh_options] [--help] [logging_options]
[connection_options] [compression_options] [retention_options]
[buffer_options]

Initializes a new backup instance inside the backup catalog backup_dir and generates the pg_probackup3.conf configuration file that controls pg_probackup3 settings for the cluster with the specified data_dir data directory. If the catalog was already initialized, you can ignore the error by specifying --skip-if-exists.

For more details of the command settings, see sections Common Options and Adding a New Backup Instance.

del-instance #

pg_probackup3 del-instance -B backup_dir --instance=instance_name [s3_options] [--help]
[ssh_options] [logging_options] [buffer_options]

Deletes all backups and WAL files associated with the specified instance.

For more details of the command settings, see the section Common Options.

set-config #

pg_probackup3 set-config -B backup_dir --instance=instance_name
[--help] [--pgdata=pgdata-path]
[--retention-redundancy=redundancy][--retention-window=window]
[compress_options] [connection_options]
[--archive-timeout=wait_time] [--external-dirs=external_directory_path]
[logging_options] [ssh_options] [buffer_options]

Adds the specified connection, compression, retention, logging, and external directory settings into the pg_probackup3.conf configuration file, or modifies the previously defined values.

For all available settings, see the Options section.

It is not recommended to edit pg_probackup3.conf manually.

set-backup #

pg_probackup3 set-backup -B backup_dir --instance=instance_name -i backup_id
{--ttl=ttl | --expire-time=time}
[--note=backup_note] [ssh_options]
[s3_options] [--help] [logging_options] [buffer_options]

Sets the provided backup-specific settings into the backup.control configuration file, or modifies the previously defined values.

--note=backup_note: Sets the text note for backup copy. If backup_note contains newline characters, then only the substring before the first newline character will be saved. The maximum size of a text note is 1 KB. The 'none' value removes the current note.

For more details of the command settings, see sections Common Options and Pinning Options.

show-config #

pg_probackup3 show-config -B backup_dir --instance instance_name
[--format=plain|json] [s3_options] [ssh_options]
[logging_options] [buffer_options]

Displays all the current pg_probackup3 configuration settings, including those that are specified in the pg_probackup3.conf configuration file located in the backup_dir/backups/instance_name directory and those that were provided on a command line. The configuration settings are shown as plain text.

To edit pg_probackup3.conf, use the set-config command.

show #

pg_probackup3 show -B backup_dir
[--help] [--instance=instance_name [-i backup_id | --archive]]
[--show-log] [--format=plain|json] [--no-color] [--format=plain|json|tree]
[s3_options] [ssh_options]
[logging_options] [buffer_options]

Shows the contents of the backup catalog. If instance_name and backup_id are specified, shows detailed information about this backup. If the --archive option is specified, shows the contents of WAL archive of the backup catalog.

By default, the contents of the backup catalog is shown as plain text. You can specify the --format=json option to get the result in the JSON format. If --no-color flag is used, then the output is not colored. You can also use the --format=tree option to see the list of backups as a tree.

For details on usage, see the sections Managing the Backup Catalog and Viewing WAL Archive Information.

backup #

pg_probackup3 backup -B backup_dir --instance=instance_name -b backup_mode -s backup_source -i backup_id
[--with-file-map] [--help] [--progress] [-j num_threads]
[--num-write-threads num_threads] [--num-validate-threads num_threads]
[--num-segments] [--create-slot] [--transfer-mode]
[--no-validate] [--skip-block-validation]
[--archive-timeout=wait_time] [--external-dirs=external_directory_path]
[--no-sync] [--note=backup_note]
[connection_options] [compression_options] [ssh_options]
[pinning_options] [logging_options] [s3_options] [buffer_options]

Creates a backup copy of the Postgres Pro instance.

-b mode --backup-mode=mode

Specifies the backup mode to use. Possible values are: FULL, DELTA, and PTRACK.

-s backup_source --backup-source=backup_source

Specifies the backup data source. Possible values are: DIRECT, BASE, and PRO.

--num-segments num_segments

Specifies the number of the backup segments during the backup creation or merge. Must be a positive integer.

Note

If the specified value exceeds the system limit for simultaneously open files, the process will fail with the error message “too many open files”.

--num-write-threads num_threads

Specifies the number of threads for copying files. Overrides the j/--threads option for file copying.

--num-validate-threads num_threads

Specifies the number of threads for the backup validation. Overrides the j/--threads option for the backup validation.

-C --smooth-checkpoint

Spreads out the checkpoint over a period of time. By default, pg_probackup3 tries to complete the checkpoint as soon as possible.

--stream

Makes a STREAM backup, which includes all the necessary WAL files by streaming them from the database server via replication protocol.

--temp-slot[=true|false|on|off]

Creates a temporary physical replication slot for streaming WAL from the backed up Postgres Pro instance. --temp-slot is enabled by default. It ensures that all the required WAL segments remain available if WAL is rotated while the backup is in progress. This flag can only be used together with the --stream flag. The default slot name is pg_probackup_slot. To change it, use the --slot/-S option and explicitly specify --temp-slot or --temp-slot=true|on.

-S slot_name --slot=slot_name

Specifies the replication slot to connect to for WAL streaming. This option can only be used together with the --stream flag.

--backup-pg-log

Includes the log directory into the backup. This directory usually contains log messages. By default, log directory is excluded.

-E external_directory_path --external-dirs=external_directory_path

Includes the specified directory into the backup by recursively copying its contents into a separate subdirectory in the backup catalog. This option is useful to back up scripts, SQL dump files, and configuration files located outside of the data directory. If you would like to back up several external directories, separate their paths by a colon on Unix and a semicolon on Windows.

--archive-timeout=wait_time

Sets the timeout for WAL segment archiving and streaming, in seconds. By default, pg_probackup3 waits 300 seconds.

--skip-block-validation

Disables block-level checksum verification to speed up the backup process.

--no-validate

Skips automatic validation after the backup is taken. You can use this flag if you validate backups regularly and would like to save time when running backup operations.

It is recommended to use this flag when creating a backup to an S3 storage. Due to some features of S3 storages, automatic validation may appear incorrect in this case. Skip automatic validation and then perform validation using a separate validate command.

--no-sync

Do not sync backed up files to disk. You can use this flag to speed up the backup process. Using this flag can result in data corruption in case of operating system or hardware crash. If you use this option, it is recommended to run the validate command once the backup is complete to detect possible issues.

--note=backup_note

Sets the text note for backup copy. If backup_note contains newline characters, then only substring before first newline character will be saved. Max size of text note is 1 KB. The 'none' value removes current note.

--with-file-map

Enables file map generation. Required for the fuse command.

For more details of the command settings, see sections Common Options, Connection Options, Pinning Options, Remote Mode Options, Compression Options, and Logging Options.

For details on usage, see the section Creating a Backup.

restore #

pg_probackup3 restore -B backup_dir --instance=instance_name
[--help] [-D data_dir] [-i backup_id]
[--progress] [-T OLDDIR=NEWDIR]
[--external-mapping=OLDDIR=NEWDIR] [--skip-external-dirs]
[-j num_threads] [--num-validate-threads num_threads]
[--no-validate] [--skip-block-validation]
[--no-sync] [--restore-command=cmdline]
[--primary-conninfo=primary_conninfo]
[--primary-slot-name=slot_name]
[recovery_target_options] [logging_options]
[ssh_options] [s3_options] [buffer_options]

Restores the Postgres Pro instance from a backup located in the backup_dir backup catalog.

Note

While backup files for restore can be retrieved from different sources (the file system, S3, or SSH SFTP), pg_probackup3 can only restore the Postgres Pro server PGDATA to a local file system.

Note

The restore command does not support the --threads option yet. The number of threads will match the number of segments in the backup.

--primary-conninfo=primary_conninfo

Sets the primary_conninfo parameter to the specified value. This option will be ignored unless the -R flag is specified.

Example: --primary-conninfo="host=192.168.1.50 port=5432 user=foo password=foopass"

--primary-slot-name=slot_name

Sets the primary_slot_name parameter to the specified value. This option will be ignored unless the -R flag is specified.

-T OLDDIR=NEWDIR --tablespace-mapping=OLDDIR=NEWDIR

Relocates the tablespace from the OLDDIR to the NEWDIR directory at the time of recovery. Both OLDDIR and NEWDIR must be absolute paths. If the path contains the equals sign (=), escape it with a backslash. This option can be specified multiple times for multiple tablespaces.

--external-mapping=OLDDIR=NEWDIR

Relocates an external directory included into the backup from the OLDDIR to the NEWDIR directory at the time of recovery. Both OLDDIR and NEWDIR must be absolute paths. If the path contains the equals sign (=), escape it with a backslash. This option can be specified multiple times for multiple directories.

--skip-external-dirs

Skip external directories included into the backup with the --external-dirs option. The contents of these directories will not be restored.

--skip-block-validation

Disables block-level checksum verification to speed up validation. During automatic validation before the restore only file-level checksums will be verified.

--no-validate

Skips backup validation. You can use this flag if you validate backups regularly and would like to save time when running restore operations.

--restore-command=cmdline

Sets the restore_command parameter to the specified command. For example: --restore-command='cp /mnt/server/archivedir/%f "%p"'

--no-sync

Do not sync restored files to disk. You can use this flag to speed up restore process. Using this flag can result in data corruption in case of operating system or hardware crash. If it happens, you have to run the restore command again.

For more details of the command settings, see sections Common Options, Recovery Target Options, Remote Mode Options, Remote WAL Archive Options, Logging Options.

For details on usage, see the section Restoring a Cluster.

validate #

pg_probackup3 validate -B backup_dir
[--help] [--instance=instance_name] [-i backup_id]
[-j num_threads] [--progress]
[--skip-block-validation] [buffer_options]
[logging_options] [ssh_options] [s3_options]

Verifies that all the files required to restore the cluster are present and are not corrupt. If you specify the instance_name without any additional options, pg_probackup3 validates all the backups available for this backup instance.

If the --progress option is specified, a list of the backup files and directories will be displayed during the validation process.

For details, see the section Validating a Backup.

merge #

pg_probackup3 merge -B backup_dir --instance=instance_name -i backup_id --merge-from-id=merge_from --merge-interval=merge_interval
[-t | --target-backup-id=backup_id] [-j num_threads] [--progress] [--no-validate] [--no-sync]
[--with-file-map] [--keep-backups] [--dry-run] [--help] [logging_options] [ssh_options] [s3_options] [buffer_options]

Merges backups that belong to a common incremental backup chain. If you specify a full backup, it will be merged with its first incremental backup. If you specify an incremental backup, it will be merged to its parent full backup, together with all incremental backups between them. Once the merge is complete, the full backup takes in all the merged data, and the incremental backups are removed as redundant. You can also merge chains of incremental backups by specifying the first and the last incremental backup or the time interval (in hours) after the first backup.

--no-validate: Skips automatic validation before and after merge.
--no-sync: Do not sync merged files to disk. You can use this flag to speed up the merge process. Using this flag can result in data corruption in case of operating system or hardware crash.
-t --target-backup-id: Specifies an ID of the merged backups.
--keep-backups: Preserves original backups after merging.
--merge-from-id: Specifies an ID of the first incremental backup from the backup chain for merge.
--merge-interval: Specifies a time period (in hours) before merging a chain of incremental backups.
--with-file-map: Enables file map generation. Required for the fuse command.

For more details of the command settings, see sections Common Options and Merging Backups.

delete #

pg_probackup3 delete -B backup_dir --instance=instance_name
[--help] [--progress] [--status=backup_status]
[--dry-run] [logging_options] [ssh_options]
[s3_options] [buffer_options]

Deletes backups with specified backup_id.

--dry-run: Initiates a trial run of the delete command, which does not actually make any changes, that is, it does not delete files on disk. This flag allows you to check that all the command options are correct and the command is ready to run.
--status: Allows deleting all backups with a specific status.

For details, see the section Deleting Backups.

archive-push #

pg_probackup3 archive-push -B backup_dir --instance=instance_name
--wal-file-name=wal_file_name [--wal-file-path=wal_file_path]
[--help] [--no-sync]
[--archive-timeout=wait_time]
[--compress-algorithm=compression_algorithm]
[--compress-level=compression_level]
[-j num_threads]
[ssh_options] [logging_options]
[s3_options] [buffer_options]

Copies WAL files into the corresponding subdirectory of the backup catalog and validates the backup instance by instance_name and system-identifier. If parameters of the backup instance and the cluster do not match, this command fails with the following error message: Refuse to push WAL segment segment_name into archive. Instance parameters mismatch.

If the files to be copied already exists in the backup catalog, pg_probackup3 computes and compares their checksums. If the checksums match, archive-push skips the corresponding file and returns a successful execution code. Otherwise, archive-push fails with an error.

Each file is copied to a temporary file with the .part suffix. If the temporary file already exists, pg_probackup3 will wait archive_timeout seconds before discarding it. After the copy is done, atomic rename is performed. This algorithm ensures that a failed archive-push will not stall continuous archiving and that concurrent archiving from multiple sources into a single WAL archive has no risk of archive corruption.

WAL segments copied to the archive are synced to disk unless the --no-sync flag is used.

You can use archive-push in the archive_command Postgres Pro parameter to set up continuous WAL archiving.

For more details of the command settings, see sections Common Options, Archiving Options, and Compression Options.

fuse #

pg_probackup3 fuse -B backup_dir --mnt-path=mnt_path --instance=instance_name
-i backup_id [--cache-swap-size=cache_swap_size] [--cache-dir=cache_dir] [--help]
[ssh_options] [logging_options] [s3_options] [buffer_options]

Mounts a backup directory as a virtual file system and allows the Postgres Pro server to run on top of it.

--cache-swap-size: Specifies the amount of data (in MB) stored in memory. The default value is 128 MB. When the cache exceeds this size, changes are flushed to the nearby disk. This allows working with a database snapshot without modifying the actual backup. The cache is cleared when the Postgres Pro server is stopped.
--cache-dir=cache_dir: Specifies the path to the FUSE cache directory. If omitted, the system temporary directory is used.

file-map #

pg_probackup3 file-map -B backup_dir --instance=instance_name -i backup_id

Enables file map generation for an existing backup chain.

If file maps already exist for the specified backups, file-map overwrites them with newly generated versions.

archive-get #

pg_probackup3 archive-get -B backup_dir --instance=instance_name --wal-file-path=wal_file_path --wal-file-name=wal_file_name
[--help] [ssh_options] [logging_options]
[s3_options] [buffer_options]

Copies WAL files from the corresponding subdirectory of the backup catalog to the cluster's write-ahead log location. This command is automatically set by pg_probackup3 as part of the restore_command when restoring backups using a WAL archive. You do not need to set it manually if you use local storage for backups or remote mode.

If you use S3 interface, to ensure that the Postgres Pro server has access to S3 storage to fetch WAL files during restore, you can specify the --config-file option that defines the S3 configuration file with appropriate configuration settings, as described in the section called “Common Options”.

For more details of the command settings, see sections Common Options, Archiving Options, and Compression Options.

retention #

pg_probackup3 retention -B backup_dir --instance=instance_name
[--retention-redundancy] [--retention-window] [--dry-run] [--merge-expired]
[--delete-expired] [--delete-wal] [pinning_options]
[ssh_options] [s3_options] [buffer_options]

Sets the backup retention policy for an instance or directory and launches backup merge or purge according to the specified parameters.

For more details of the command settings, see the section Retention Options.

Options #

This section describes command-line options for pg_probackup3 commands. If the option value can be derived from an environment variable, this variable is specified below the command-line option, in the uppercase. Some values can be taken from the pg_probackup3.conf configuration file located in the backup catalog.

For details, see Section 2.4.

If an option is specified using more than one method, command-line input has the highest priority, while the pg_probackup3.conf settings have the lowest priority.

Common Options #

The list of general options.

--dry-run

Initiates a trial run of the appropriate command, which does not actually do any changes, that is, it does not create, delete or move files on disk. This flag allows you to check that all the command options are correct and the command is ready to run. WAL streaming is skipped with --dry-run.

-B directory --backup-path=directory BACKUP_PATH

Specifies the absolute path to the backup catalog. Backup catalog is a directory where all backup files and meta information are stored. Since this option is required for most of the pg_probackup3 commands, you are recommended to specify it once in the BACKUP_PATH environment variable. In this case, you do not need to use this option each time on the command line.

-D directory --pgdata=directory PGDATA

Specifies the absolute path to the data directory of the database cluster. This option is mandatory only for the add-instance command. Other commands can take its value from the PGDATA environment variable, or from the pg_probackup3.conf configuration file.

-i backup_id --backup-id=backup_id

Specifies the unique identifier of the backup.

--parent-backup-id=parent_backup_id

Specifies the unique identifier of the parent backup (used for incremental backups).

--from-full

Creates an incremental backup from the latest parent FULL backup.

-j num_threads --threads=num_threads

Sets the number of parallel threads for backup, restore, merge, validate, and archive-push processes. Defaults to the number of processor cores.

--num-validate-threads num_threads

Sets the number of parallel threads during the backup validation, for example, when running the backup or restore command.

--no-validate disables --num-validate-threads.

--progress

Shows the progress of operations.

--help

Shows detailed information about the options that can be used with this command.

-v version --version=version

Shows pg_probackup3 version.

--config-file=file_name

Specifies the S3 or SSH configuration file. Settings in the configuration file override the environment variables.

To generate the configuration file, run the set-config command with the --config-file option.

The generated file will include all explicitly specified S3 or SSH parameters, while passwords will be omitted and displayed as asterisks.

An example of the S3 configuration file:

access-key=admin
s3=on
s3-bucket=test
s3-host=127.0.0.1
s3-port=9000
s3-region=us-west-2
secret-key=***

If the configuration file contains both S3 and SSH options, S3 options will be used.

If the --config-file option is not specified, pg_probackup3 will first look for S3 and SSH configuration files at /etc/pg_probackup/s3.config or /etc/pg_probackup/ssh.config and then at ~postgres/.pg_probackup/s3.config or ~postgres/.pg_probackup/ssh.config, respectively.

Recovery Target Options #

If continuous WAL archiving is configured, you can use one of these options with restore command to specify the moment up to which the database cluster must be restored.

--recovery-target-stop=immediate|latest

Defines when to stop the recovery:

The immediate value stops the recovery after reaching the consistent state of the specified backup. This is the default behavior for STREAM backups.
The latest value continues the recovery until all WAL segments available in the archive are applied. Setting this value of --recovery-target also sets --recovery-target-timeline to latest.

--recovery-target-timeline=timeline

Specifies a particular timeline to be used for recovery:

current — the timeline of the specified backup, default.

latest — the timeline of the latest available backup.
A numeric value.

--recovery-target-lsn=lsn

Specifies the LSN of the write-ahead log location up to which recovery will proceed.

--recovery-target-name=recovery_target_name

Specifies a named savepoint up to which to restore the cluster.

--recovery-target-time=time|current|latest

Specifies the timestamp up to which recovery will proceed. If the time zone offset is not specified, the local time zone is used.

Example: --recovery-target-time="2027-04-09 18:21:32+00"

--recovery-target-xid=xid

Specifies the transaction ID up to which recovery will proceed.

--recovery-target-inclusive=boolean

Specifies whether to stop just after the specified recovery target (true), or just before the recovery target (false). This option can only be used together with --recovery-target-time, --recovery-target-lsn or --recovery-target-xid options. The default depends on the recovery_target_inclusive parameter.

--recovery-target-action=pause|promote|shutdown

Specifies the action the server should take when the recovery target is reached.

Default: pause

Retention Options #

These options are used with the retention command.

For details on configuring retention policy, see the section Configuring Retention Policy.

--retention-redundancy=redundancy

Specifies the number of full backup copies to keep in the data directory. Must be a non-negative integer. The zero value disables this setting.

Default: 0

--retention-window=window

Specifies the number of days of recoverability. Must be a non-negative integer. The zero value disables this setting.

Default: 0

--delete-wal

Deletes WAL files that are no longer required to restore the cluster from any of the existing backups.

--delete-expired

Deletes backups that do not conform to the retention policy defined in the pg_probackup3.conf configuration file.

--merge-expired

Merges the oldest incremental backup that satisfies the requirements of retention policy with its parent backups that have already expired.

Pinning Options #

You can use these options together with backup, set-backup, and retention commands.

For details on backup pinning, see the section Backup Pinning.

--ttl=ttl

Specifies the amount of time the backup should be pinned. Must be a non-negative integer. The zero value unpins the already pinned backup. Supported units: ms, s, min, h, d (s by default).

Example: --ttl=30d

--expire-time=time

Specifies the timestamp up to which the backup will stay pinned. Must be an ISO-8601 complaint timestamp. If the time zone offset is not specified, the local time zone is used.

Example: --expire-time="2027-04-09 18:21:32+00"

Logging Options #

You can use these options with any command.

--no-color

Disable coloring for console log messages of warning and error levels.

--log-level-console=log_level

Controls which message levels are sent to the console log. Valid values are trace, debug, info, warning, error and off. Each level includes all the levels that follow it. The later the level, the fewer messages are sent. The off level disables console logging.

Default: info

Note

All console log messages are going to stderr, so the output of show and show-config commands does not mingle with log messages.

--log-level-file=log_level

Controls which message levels are sent to a log file. Valid values are trace, debug, info, warning, error, and off. Each level includes all the levels that follow it. The later the level, the fewer messages are sent. The off level disables file logging.

Default: off

--log-backup=log_level

Controls which message levels are sent to a backup log file created in the backup directory when running the backup command. Valid values are trace, debug, info, warning, error, and off. Each level includes all the levels that follow it. The later the level, the fewer messages are sent. The off level disables file logging.

Default: info

--log-filename=log_filename

Defines the filenames of the created log files. The filenames are treated as a strftime pattern, so you can use %-escapes to specify time-varying filenames.

Default: pg_probackup.log

For example, if you specify the pg_probackup-%u.log pattern, pg_probackup3 generates a separate log file for each day of the week, with %u replaced by the corresponding decimal number: pg_probackup-1.log for Monday, pg_probackup-2.log for Tuesday, and so on.

This option takes effect if file logging is enabled by the --log-level-file option.

--error-log-filename=error_log_filename

Defines the filenames of log files for error messages only. The filenames are treated as a strftime pattern, so you can use %-escapes to specify time-varying filenames.

Default: none

For example, if you specify the error-pg_probackup-%u.log pattern, pg_probackup3 generates a separate log file for each day of the week, with %u replaced by the corresponding decimal number: error-pg_probackup-1.log for Monday, error-pg_probackup-2.log for Tuesday, and so on.

This option is useful for troubleshooting and monitoring.

--log-directory=log_directory

Defines the directory in which log files will be created. You must specify the absolute path. This directory is created lazily, when the first log message is written.

Note that the directory for log files is always created locally even if backups are created in the S3 storage. So be sure to pass a local path in log_directory when needed.

Default: $BACKUP_PATH/log/

--log-format-console=log_format

Defines the format of the console log. Only set from the command line. Note that you cannot specify this option in the pg_probackup3.conf configuration file through the set-config command and that the backup command also treats this option specified in the configuration file as an error. Possible values are:

plain — sets the plain-text format of the console log.
json — sets the JSON format of the console log.

Default: plain

--log-format-file=log_format

Defines the format of log files used. Possible values are:

plain — sets the plain-text format of log files.
json — sets the JSON format of log files.

Default: plain

--log-rotation-size=log_rotation_size

Maximum size of an individual log file. If this value is reached, the log file is rotated once a pg_probackup3 command is launched, except help and version commands. The zero value disables size-based rotation. Supported units: kB, MB, GB, TB (kB by default).

Default: 0

--log-rotation-age=log_rotation_age

Maximum lifetime of an individual log file. If this value is reached, the log file is rotated once a pg_probackup3 command is launched, except help and version commands. The time of the last log file creation is stored in $BACKUP_PATH/log/log_rotation. The zero value disables time-based rotation. Supported units: ms, s, min, h, d (min by default).

Default: 0

Connection Options #

You can use these options together with the backup command.

All libpq environment variables are supported.

-d dbname --pgdatabase=dbname PGDATABASE

Specifies the name of the database to connect to. The connection is used only for managing backup process, so you can connect to any existing database. If this option is not provided on the command line, PGDATABASE environment variable, or the pg_probackup3.conf configuration file, pg_probackup3 tries to take this value from the PGUSER environment variable, or from the current user name if PGUSER variable is not set.

-h host --pghost=host PGHOST

Specifies the host name of the system on which the server is running. If the value begins with a slash, it is used as a directory for the Unix domain socket.

Default: localhost

-p port --pgport=port PGPORT

Specifies the TCP port or the local Unix domain socket file extension on which the server is listening for connections.

Default: 5432

-U username --pguser=username PGUSER

User name to connect as.

-w --no-password

Disables a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file or PGPASSWORD environment variable, the connection attempt will fail. This flag can be useful in batch jobs and scripts where no user is present to enter a password.

-W --password

Forces a password prompt. (Deprecated)

Compression Options #

You can use these options together with backup and archive-push commands.

--compress-algorithm=compression_algorithm

Defines the algorithm to use for compressing data files. Possible values are zlib, lz4, zstd, and none. If set to any value but none, this option enables compression that uses the corresponding algorithm. Both data files and WAL files are compressed. By default, compression is disabled.

Default: none

Warning

Option value lz4 for --compress-algorithm isn't currently supported in archive-push and archive-get commands.

--compress-level=compression_level

Defines the compression level.

Note

This option must be used together with the --compress-algorithm option.

Possible values depend on the compression algorithm specified:

0 — 9 for zlib
0 — 12 for lz4
0 — 22 for zstd

The value of 0 sets the default compression level for the specified algorithm:

6 for zlib
9 for lz4
3 for zstd

Note

The pure lz4 algorithm has only one compression level — 1. So, if the specified compression algorithm is lz4 and --compress-level is greater than 1, the lz4hc algorithm is actually used, which is much slower although does better compression.

Default: 1

Archiving Options #

These options can be used with the archive-push command in the archive_command setting and the archive-get command in the restore_command setting.

Additionally, remote mode options and logging options can be used.

--wal-file-path=wal_file_path: Provides the path to the WAL file in archive_command and restore_command. Use the %p variable as the value for this option or explicitly specify the path to a file outside of the data directory. If you skip this option, the path specified in pg_probackup3.conf will be used.
--wal-file-name=wal_file_name: Provides the name of the WAL file in archive_command and restore_command. Use the %f variable as the value for this option for correct processing. If the value of --wal-file-path is a path outside of the data directory, explicitly specify the filename.
--archive-timeout=wait_time: Sets the timeout for considering existing .part files to be stale. By default, pg_probackup3 waits 300 seconds. This option can be used only with the archive-push command.
--no-sync: Do not sync copied WAL files to disk. You can use this flag to speed up archiving process. Using this flag can result in WAL archive corruption in case of operating system or hardware crash. This option can be used only with archive-push command.

Buffer Options #

You can use these options with all commands.

--buffer-size=size: Specifies the buffer size for read and write operations. Must be a non-negative integer. The zero value disables this setting. The default value is 0.
--buffer-read-size=size: Specifies a separate buffer size for read operations. Must be a non-negative integer. The zero value disables this setting. The default value is 0.
--buffer-write-size=size: Specifies a separate extended buffer size for write operations. Must be a non-negative integer. The zero value disables this setting. The default value is 0.

You can explicitly specify units for any of the buffer options:

B — bytes
KB — kilobytes
MB — megabytes
GB — gigabytes
TB — terabytes

If no unit is specified, the value defaults to bytes.

Remote Mode Options #

This section describes the options related to running pg_probackup3 operations remotely via SSH. These options can be used with all commands.

For details on configuring and using the remote mode, see Section 2.11 and Section 3.5.

--remote-host=destination

Specifies the remote host IP address or hostname to connect to.

--remote-port=port

Specifies the remote host port to connect to.

Default: 22

--remote-user=username

Specifies remote host user for SSH connection. If you omit this option, the current user initiating the SSH connection is used.

--remote-path=path

Specifies pg_probackup3 installation directory on the remote system.

--ssh-options=ssh_options

Provides a string of SSH command-line options. For example, the following options can be used to set keep-alive for SSH connections opened by pg_probackup3: --ssh-options="-o ServerAliveCountMax=5 -o ServerAliveInterval=60". For the full list of possible options, see ssh_config manual page.

--ssh-password=password

Specifies the password for SSH connection.

Remote WAL Archive Options #

This section describes the options used to provide the arguments for remote mode options.

--archive-host=destination

Provides the argument for the --remote-host option in the archive-get command.

--archive-port=port

Provides the argument for the --remote-port option in the archive-get command.

Default: 22

--archive-user=username

Provides the argument for the --remote-user option in the archive-get command. If you omit this option, the user that has started the Postgres Pro cluster is used.

Default: Postgres Pro user

Incremental Restore Options #

This section describes the options for incremental cluster restore. These options can be used with the restore command.

--I incremental_mode --incremental-mode=incremental_mode

Specifies the incremental mode to be used. Possible values are:

CHECKSUM — replace only pages with mismatched checksum and LSN.
LSN — replace only pages with LSN greater than point of divergence.
NONE — regular restore.

Partial Restore Options #

This section describes the options for partial cluster restore. These options can be used with the restore command.

--db-exclude-oid=dboid: Specifies the OID of the database to exclude from restore. All other databases in the cluster will be restored as usual, including template0 and template1. This option can be specified multiple times for multiple databases.
--db-include-oid=dboid: Specifies the OID of the database to restore from a backup. All other databases in the cluster will not be restored, with the exception of template0 and template1. This option can be specified multiple times for multiple databases.
--db-exclude-name=dbname: Specifies the name of the database to exclude from restore. All other databases in the cluster will be restored as usual, including template0 and template1. This option can be specified multiple times for multiple databases.
--db-include-name=dbname: Specifies the name of the database to restore from a backup. All other databases in the cluster will not be restored, with the exception of template0 and template1. This option can be specified multiple times for multiple databases.

Warning

Options --db-exclude-oid and --db-include-oid cannot be used together, as well as --db-exclude-name and --db-include-name.

S3 Options #

This section describes the options needed to store backups in private clouds. These options can be used with any command that pg_probackup3 runs using S3 interface.

For more details, refer to the section Configuring S3 Connection section.

The S3 connection options are described below.

--s3=s3_interface_provider

Specifies the S3 interface provider. Possible values are:

minio — MinIO object storage, compatible with S3 cloud storage service. With this provider, custom S3 server settings can be specified. The HTTP protocol, port 9000, and region us-east-1 are used by default.
off — explicitly disables S3 mode. This is the default value for --s3 option.

With --s3=minio, pg_probackup3 will work fine for a VK Cloud storage if the S3 host address, port and protocol are properly specified (host address is hb.vkcs.cloud or the one specified in the appropriate section of the VK Cloud profile, port 443, and HTTPS protocol). Do not specify --s3=minio for the Amazon S3 storage.

--s3-host=hostname

Specifies the address of the S3 server. Can include the port number, separated by a colon. If the port number is not specified in a host string, the value of --s3-port is assumed. Do not add a colon if the port number is not specified.

--s3-port=port_number

Specifies the port of the S3 server.

--s3-region=region

Specifies the region of the S3 server. The default is us-east-1.

--s3-bucket=bucket

Specifies the name of the bucket on the S3 server.

--access-key=access_key

Specifies the access key from S3 storage.

--secret-key=password

Specifies the secret access key from S3 storage.

--s3-secure=protocol

Specifies the protocol to be used. Possible values:

ON or HTTPS — HTTPS is used.
HTTP — HTTP is used. This is the default value.

--s3-retries=retry_count

Sets the maximum number of attempts to execute an S3 request in case of failures. The default is 3.

--s3-timeout=timeout

Sets the maximum amount of time to execute an HTTP request to the S3 server, in seconds. The default is 300.

--s3-ignore-cert-ver=ON|OFF

Allows to skip the certificate host and peer verification. The default is OFF.

--s3-ca-certificate=ca_certificate

Specifies the path to the file with a trust Certificate Authority (CA) bundle.

--s3-ca-path=ca_path

Specifies the directory with trust CA certificates.

--s3-client-cert=client_cert

Sets the SSL client certificate.

--s3-client-key=client_key

Sets the private key file for the TLS and SSL client certificates.

--s3-versioning=enabled|suspended|off

Sets the S3 bucket support for object versioning. The default is off.

--s3-http-compression=true|false

Sets the "Accept-Encoding" HTTP header and decompresses received contents. The default is false.

The S3 performance options are described below.

--s3-buffer-size=size [unit]

Specifies the size of the read/write buffer for communicating with S3. You can explicitly specify units:

B — bytes
KB — kilobytes
MB — megabytes
GB — gigabytes
TB — terabytes

If no unit is specified, the value defaults to bytes.

Note

The --s3-buffer-size value must not be less than 5 MB. Smaller values will trigger a warning and will be automatically switched to 5MB.

Testing and Debugging Options #

This section describes options useful only in a test or development environment.

--cfs-nondatafile-mode: Instructs backup command to backup CFS in a legacy mode. This allows fine-tuning compatibility with pg_probackup3 versions earlier than 2.6.0. This option is mainly designed for testing.
PGPROBACKUP_TESTS_SKIP_HIDDEN: Instructs pg_probackup3 to ignore backups marked as hidden. Note that pg_probackup3 can never mark a backup as hidden. It can only be done by directly editing the backup.control file. This option can only be set with environment variables.
--destroy-all-other-dbs: By default, pg_probackup3 exits with an error if an attempt is made to perform a partial incremental restore since this destroys databases not included in the restore set. This flag allows you to suppress the error and proceed with the partial incremental restore (e.g., to keep a development database snapshot up-to-date with a production one). This option can be used with the restore command.
Important
Never use this flag in a production cluster.
PGPROBACKUP_TESTS_SKIP_EMPTY_COMMIT: Instructs pg_probackup3 to skip empty commits after pg_backup_stop.

Authors #

Postgres Professional, Moscow, Russia.

Prev	Up	Next
Chapter 4. Reference	Home	Chapter 5. For Developers

pdf pdf

pg_probackup3

Synopsis

Command-Line Reference #

Commands #

version #

help #

init #

add-instance #

del-instance #

set-config #

set-backup #

show-config #

show #

backup #

Note

restore #

Note

Note

validate #

merge #

delete #

archive-push #

fuse #

file-map #

archive-get #

retention #

Options #

Common Options #

Recovery Target Options #

Retention Options #

Pinning Options #

Logging Options #

Note

Connection Options #

Compression Options #

Warning

Note

Note

Archiving Options #

Buffer Options #

Remote Mode Options #

Remote WAL Archive Options #

Incremental Restore Options #

Partial Restore Options #

Warning

S3 Options #

Note

Testing and Debugging Options #

Important

Authors #