summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorRobert Haas2020-04-14 17:41:32 +0000
committerRobert Haas2020-04-14 17:41:32 +0000
commit149f2ae88ab0510be3239caf45699d7373f2c6e8 (patch)
treef7902957b5f84ebaee4a5de5e80b7e0f901a153a
parent80634e3b181562015762b5846583ff67f8037a80 (diff)
Document the backup manifest file format.
Patch by me, at the request of Andres Freund. Reviewed by Justin Pryzby, Erik Rijkers, Álvaro Herrera, and Andrew Dunstan. Discussion: http://postgr.es/m/[email protected]
-rw-r--r--doc/src/sgml/backup-manifest.sgml212
-rw-r--r--doc/src/sgml/filelist.sgml1
-rw-r--r--doc/src/sgml/postgres.sgml1
3 files changed, 214 insertions, 0 deletions
diff --git a/doc/src/sgml/backup-manifest.sgml b/doc/src/sgml/backup-manifest.sgml
new file mode 100644
index 0000000000..376aff0d6d
--- /dev/null
+++ b/doc/src/sgml/backup-manifest.sgml
@@ -0,0 +1,212 @@
+<!-- doc/src/sgml/backupmanifest.sgml -->
+
+<chapter id="backup-manifest-format">
+ <title>Backup Manifest Format</title>
+
+ <para>
+ The backup manifest generated by <xref linkend="app-pgbasebackup" /> is
+ primarily intended to permit the backup to be verified using
+ <xref linkend="app-pgverifybackup" />. However, it is
+ also possible for other tools to read the backup manifest file and use
+ the information contained therein for their own purposes. To that end,
+ this chapter describes the format of the backup manifest file.
+ </para>
+
+ <para>
+ A backup manifest is a JSON document encoded as UTF-8. (Although in
+ general JSON documents are required to be Unicode, PostgreSQL permits
+ the <type>json</type> and <type>jsonb</type> data types to be used with any
+ supported server encoding. There is no similar exception for backup
+ manifests.) The JSON document is always an object; the keys that are present
+ in this object are described in the next section.
+ </para>
+
+ <sect1 id="backup-manifest-toplevel">
+ <title>Backup Manifest Toplevel Object</title>
+
+ <para>
+ The backup manifest JSON document contains the following keys.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>PostgreSQL-Backup-Manifest-Version</literal></term>
+ <listitem>
+ <para>
+ The associated value is always the integer 1.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>Files</literal></term>
+ <listitem>
+ <para>
+ The associated value is always a list of objects, each describing one
+ file that is present in the backup. No entries are present in this
+ list for the WAL files that are needed in order to use the backup,
+ or for the backup manifest itself. The structure of each object in the
+ list is described in <xref linkend="backup-manifest-files" />.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>WAL-Ranges</literal></term>
+ <listitem>
+ <para>
+ The associated value is always a list of objects, each describing a
+ range of WAL records that must be readable from a particular timeline
+ in order to make use of the backup. The structure of these objects is
+ further described in <xref linkend="backup-manifest-wal-ranges" />.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>Manifest-Checksum</literal></term>
+ <listitem>
+ <para>
+ This key is always present on the last line of the backup manifest file.
+ The associated value is a SHA256 checksum of all the preceding lines.
+ We use a fixed checksum method here to make it possible for clients
+ to do incremental parsing of the manifest. While a SHA256 checksum
+ is significantly more expensive than a CRC32C checksum, the manifest
+ should normally be small enough that the extra computation won't matter
+ very much.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect1>
+
+ <sect1 id="backup-manifest-files">
+ <title>Backup Manifest File Object</title>
+
+ <para>
+ The object which describes a single file contains either a
+ <literal>Path</literal> key or an <literal>Encoded-Path</literal> key.
+ Normally, the <literal>Path</literal> key will be present. The
+ associated string value is the path of the file relative to the root
+ of the backup directory. Files located in a user-defined tablespace
+ will have paths whose first two components are pg_tblspc and the OID
+ of the tablespace. If the path is not a string that is legal in UTF-8,
+ or if the user requests that encoded paths be used for all files, then
+ the <literal>Encoded-Path</literal> key will be present instead. This
+ stores the same data, but it is encoded as a string of hexadecimal
+ digits. Each pair of hexadecimal digits in the string represents a
+ single octet.
+ </para>
+
+ <para>
+ The following two keys are always present:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>Size</literal></term>
+ <listitem>
+ <para>
+ The expected size of this file, as an integer.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>Last-Modified</literal></term>
+ <listitem>
+ <para>
+ The last modification time of the file as reported by the server at
+ the time of the backup. Unlike the other fields stored in the backup,
+ this field is not used by <xref linkend="app-pgverifybackup" />.
+ It is included only for informational purposes.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If the backup was taken with file checksums enabled, the following
+ keys will be present:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>Checksum-Algorithm</literal></term>
+ <listitem>
+ <para>
+ The checksum algorithm used to compute a checksum for this file.
+ Currently, this will be the same for every file in the backup
+ manifest, but this may change in future releases. At present, the
+ supported checksum algorithms are <literal>CRC32C</literal>,
+ <literal>SHA224</literal>,
+ <literal>SHA256</literal>,
+ <literal>SHA384</literal>, and
+ <literal>SHA512</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>Checksum</literal></term>
+ <listitem>
+ <para>
+ The checksum computed for this file, stored as a series of
+ hexadecimal characters, two for each byte of the checksum.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect1>
+
+ <sect1 id="backup-manifest-wal-ranges">
+ <title>Backup Manifest WAL Range Object</title>
+
+ <para>
+ The object which describes a WAL range always has three keys:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>Timeline</literal></term>
+ <listitem>
+ <para>
+ The timeline for this range of WAL records, as an integer.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>Start-LSN</literal></term>
+ <listitem>
+ <para>
+ The LSN at which replay must begin on the indicated timeline in order to
+ make use of this backup. The LSN is stored in the format normally used
+ by <productname>PostgreSQL</productname>; that is, it is a string
+ consisting of two strings of hexadecimal characters, each with a length
+ of between 1 and 8, separated by a slash.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>End-LSN</literal></term>
+ <listitem>
+ <para>
+ The earliest LSN at which replay on the indicated timeline may end when
+ making use of this backup. This is stored in the same format as
+ <literal>Start-LSN</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ Ordinarily, there will be only a single WAL range. However, if a backup is
+ taken from a standby which switches timelines during the backup due to an
+ upstream promotion, it is possible for multiple ranges to be present, each
+ with a different timeline. There will never be multiple WAL ranges present
+ for the same timeline.
+ </para>
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index cf21ef857e..68179f71cd 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -102,6 +102,7 @@
<!ENTITY storage SYSTEM "storage.sgml">
<!ENTITY tablesample-method SYSTEM "tablesample-method.sgml">
<!ENTITY generic-wal SYSTEM "generic-wal.sgml">
+<!ENTITY backup-manifest SYSTEM "backup-manifest.sgml">
<!-- contrib information -->
<!ENTITY contrib SYSTEM "contrib.sgml">
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index f4a3c7e5e7..c41ce9499b 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -268,6 +268,7 @@ break is not needed in a wider output rendering.
&storage;
&bki;
&planstats;
+ &backup-manifest;
</part>