diff options
author | Robert Haas | 2020-04-14 17:41:32 +0000 |
---|---|---|
committer | Robert Haas | 2020-04-14 17:41:32 +0000 |
commit | 149f2ae88ab0510be3239caf45699d7373f2c6e8 (patch) | |
tree | f7902957b5f84ebaee4a5de5e80b7e0f901a153a | |
parent | 80634e3b181562015762b5846583ff67f8037a80 (diff) |
Document the backup manifest file format.
Patch by me, at the request of Andres Freund. Reviewed by
Justin Pryzby, Erik Rijkers, Álvaro Herrera, and Andrew
Dunstan.
Discussion: http://postgr.es/m/[email protected]
-rw-r--r-- | doc/src/sgml/backup-manifest.sgml | 212 | ||||
-rw-r--r-- | doc/src/sgml/filelist.sgml | 1 | ||||
-rw-r--r-- | doc/src/sgml/postgres.sgml | 1 |
3 files changed, 214 insertions, 0 deletions
diff --git a/doc/src/sgml/backup-manifest.sgml b/doc/src/sgml/backup-manifest.sgml new file mode 100644 index 0000000000..376aff0d6d --- /dev/null +++ b/doc/src/sgml/backup-manifest.sgml @@ -0,0 +1,212 @@ +<!-- doc/src/sgml/backupmanifest.sgml --> + +<chapter id="backup-manifest-format"> + <title>Backup Manifest Format</title> + + <para> + The backup manifest generated by <xref linkend="app-pgbasebackup" /> is + primarily intended to permit the backup to be verified using + <xref linkend="app-pgverifybackup" />. However, it is + also possible for other tools to read the backup manifest file and use + the information contained therein for their own purposes. To that end, + this chapter describes the format of the backup manifest file. + </para> + + <para> + A backup manifest is a JSON document encoded as UTF-8. (Although in + general JSON documents are required to be Unicode, PostgreSQL permits + the <type>json</type> and <type>jsonb</type> data types to be used with any + supported server encoding. There is no similar exception for backup + manifests.) The JSON document is always an object; the keys that are present + in this object are described in the next section. + </para> + + <sect1 id="backup-manifest-toplevel"> + <title>Backup Manifest Toplevel Object</title> + + <para> + The backup manifest JSON document contains the following keys. + </para> + + <variablelist> + <varlistentry> + <term><literal>PostgreSQL-Backup-Manifest-Version</literal></term> + <listitem> + <para> + The associated value is always the integer 1. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Files</literal></term> + <listitem> + <para> + The associated value is always a list of objects, each describing one + file that is present in the backup. No entries are present in this + list for the WAL files that are needed in order to use the backup, + or for the backup manifest itself. The structure of each object in the + list is described in <xref linkend="backup-manifest-files" />. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>WAL-Ranges</literal></term> + <listitem> + <para> + The associated value is always a list of objects, each describing a + range of WAL records that must be readable from a particular timeline + in order to make use of the backup. The structure of these objects is + further described in <xref linkend="backup-manifest-wal-ranges" />. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Manifest-Checksum</literal></term> + <listitem> + <para> + This key is always present on the last line of the backup manifest file. + The associated value is a SHA256 checksum of all the preceding lines. + We use a fixed checksum method here to make it possible for clients + to do incremental parsing of the manifest. While a SHA256 checksum + is significantly more expensive than a CRC32C checksum, the manifest + should normally be small enough that the extra computation won't matter + very much. + </para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + + <sect1 id="backup-manifest-files"> + <title>Backup Manifest File Object</title> + + <para> + The object which describes a single file contains either a + <literal>Path</literal> key or an <literal>Encoded-Path</literal> key. + Normally, the <literal>Path</literal> key will be present. The + associated string value is the path of the file relative to the root + of the backup directory. Files located in a user-defined tablespace + will have paths whose first two components are pg_tblspc and the OID + of the tablespace. If the path is not a string that is legal in UTF-8, + or if the user requests that encoded paths be used for all files, then + the <literal>Encoded-Path</literal> key will be present instead. This + stores the same data, but it is encoded as a string of hexadecimal + digits. Each pair of hexadecimal digits in the string represents a + single octet. + </para> + + <para> + The following two keys are always present: + </para> + + <variablelist> + <varlistentry> + <term><literal>Size</literal></term> + <listitem> + <para> + The expected size of this file, as an integer. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Last-Modified</literal></term> + <listitem> + <para> + The last modification time of the file as reported by the server at + the time of the backup. Unlike the other fields stored in the backup, + this field is not used by <xref linkend="app-pgverifybackup" />. + It is included only for informational purposes. + </para> + </listitem> + </varlistentry> + </variablelist> + + <para> + If the backup was taken with file checksums enabled, the following + keys will be present: + </para> + + <variablelist> + <varlistentry> + <term><literal>Checksum-Algorithm</literal></term> + <listitem> + <para> + The checksum algorithm used to compute a checksum for this file. + Currently, this will be the same for every file in the backup + manifest, but this may change in future releases. At present, the + supported checksum algorithms are <literal>CRC32C</literal>, + <literal>SHA224</literal>, + <literal>SHA256</literal>, + <literal>SHA384</literal>, and + <literal>SHA512</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Checksum</literal></term> + <listitem> + <para> + The checksum computed for this file, stored as a series of + hexadecimal characters, two for each byte of the checksum. + </para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + + <sect1 id="backup-manifest-wal-ranges"> + <title>Backup Manifest WAL Range Object</title> + + <para> + The object which describes a WAL range always has three keys: + </para> + + <variablelist> + <varlistentry> + <term><literal>Timeline</literal></term> + <listitem> + <para> + The timeline for this range of WAL records, as an integer. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Start-LSN</literal></term> + <listitem> + <para> + The LSN at which replay must begin on the indicated timeline in order to + make use of this backup. The LSN is stored in the format normally used + by <productname>PostgreSQL</productname>; that is, it is a string + consisting of two strings of hexadecimal characters, each with a length + of between 1 and 8, separated by a slash. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>End-LSN</literal></term> + <listitem> + <para> + The earliest LSN at which replay on the indicated timeline may end when + making use of this backup. This is stored in the same format as + <literal>Start-LSN</literal>. + </para> + </listitem> + </varlistentry> + </variablelist> + + <para> + Ordinarily, there will be only a single WAL range. However, if a backup is + taken from a standby which switches timelines during the backup due to an + upstream promotion, it is possible for multiple ranges to be present, each + with a different timeline. There will never be multiple WAL ranges present + for the same timeline. + </para> + </sect1> +</chapter> diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index cf21ef857e..68179f71cd 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -102,6 +102,7 @@ <!ENTITY storage SYSTEM "storage.sgml"> <!ENTITY tablesample-method SYSTEM "tablesample-method.sgml"> <!ENTITY generic-wal SYSTEM "generic-wal.sgml"> +<!ENTITY backup-manifest SYSTEM "backup-manifest.sgml"> <!-- contrib information --> <!ENTITY contrib SYSTEM "contrib.sgml"> diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml index f4a3c7e5e7..c41ce9499b 100644 --- a/doc/src/sgml/postgres.sgml +++ b/doc/src/sgml/postgres.sgml @@ -268,6 +268,7 @@ break is not needed in a wider output rendering. &storage; &bki; &planstats; + &backup-manifest; </part> |