pg_basebackup: Allow client-side LZ4 (de)compression.

LZ4 compression can now be performed on the client using
pg_basebackup -Ft --compress client-lz4, and LZ4 decompression of
a backup compressed on the server can be performed on the client
using pg_basebackup -Fp --compress server-lz4.

Dipesh Pandit, reviewed and tested by Jeevan Ladhe and Tushar Ahuja,
with a few corrections - and some documentation - by me.

Discussion: http://postgr.es/m/CAN1g5_FeDmiA9D8wdG2W6Lkq5CpubxOAqTmd2et9hsinTJtsMQ@mail.gmail.com

Author: robertmhaas

doc/src/sgml/ref/pg_basebackup.sgml

@@ -417,18 +417,14 @@ PostgreSQL documentation
         specify <literal>-Xfetch</literal>.
        </para>
        <para>
-        The compression method can be set to <literal>gzip</literal> for
-        compression with <application>gzip</application>, or
-        <literal>lz4</literal> for compression with
-        <application>lz4</application>, or <literal>none</literal> for no
-        compression. However, <literal>lz4</literal> can be currently only
-        used with <literal>server</literal>. A compression level can be
-        optionally specified, by appending the level number after a
-        colon (<literal>:</literal>). If no level is specified, the default
-        compression level will be used. If only a level is specified without
-        mentioning an algorithm, <literal>gzip</literal> compression will
-        be used if the level is greater than 0, and no compression will be
-        used if the level is 0.
+        The compression method can be set to <literal>gzip</literal> or
+        <literal>lz4</literal>, or <literal>none</literal> for no
+        compression. A compression level can be optionally specified, by
+        appending the level number after a colon (<literal>:</literal>). If no
+        level is specified, the default compression level will be used. If
+        only a level is specified without mentioning an algorithm,
+        <literal>gzip</literal> compression will be used if the level is
+        greater than 0, and no compression will be used if the level is 0.
        </para>
        <para>
         When the tar format is used with <literal>gzip</literal> or
@@ -439,6 +435,13 @@ PostgreSQL documentation
         compression. If this is done, the server will compress the backup for
         transmission, and the client will decompress and extract it.
        </para>
+       <para>
+        When this option is used in combination with
+        <literal>-Xstream</literal>, <literal>pg_wal.tar</literal> will
+        be compressed using <literal>gzip</literal> if client-side gzip
+        compression is selected, but will not be compressed if server-side
+        compresion or LZ4 compresion is selected.
+       </para>
       </listitem>
      </varlistentry>
     </variablelist>

src/bin/pg_basebackup/Makefile

@@ -43,6 +43,7 @@ BBOBJS = \
 	bbstreamer_file.o \
 	bbstreamer_gzip.o \
 	bbstreamer_inject.o \
+	bbstreamer_lz4.o \
 	bbstreamer_tar.o
 
 all: pg_basebackup pg_receivewal pg_recvlogical

src/bin/pg_basebackup/bbstreamer.h

@@ -206,6 +206,9 @@ extern bbstreamer *bbstreamer_extractor_new(const char *basepath,
 											void (*report_output_file) (const char *));
 
 extern bbstreamer *bbstreamer_gzip_decompressor_new(bbstreamer *next);
+extern bbstreamer *bbstreamer_lz4_compressor_new(bbstreamer *next,
+												 int compresslevel);
+extern bbstreamer *bbstreamer_lz4_decompressor_new(bbstreamer *next);
 extern bbstreamer *bbstreamer_tar_parser_new(bbstreamer *next);
 extern bbstreamer *bbstreamer_tar_terminator_new(bbstreamer *next);
 extern bbstreamer *bbstreamer_tar_archiver_new(bbstreamer *next);