diff options
| author | Pavan Deolasee | 2015-07-23 13:11:15 +0000 |
|---|---|---|
| committer | Pavan Deolasee | 2015-09-14 05:38:13 +0000 |
| commit | 57d93c30dc8f1b0deea9e6686a0dfa89854a44a3 (patch) | |
| tree | d232bd3ddeeb5c91f2f1b07d9c61ecbf04deefff | |
| parent | 35f4e1094bc03005f0f771fb20946ddc63c45f9e (diff) | |
Rewrite Postgres-XL docs to make changes in-place in the main PostgreSQL
documentation
Originally, Postgres-XC had chosen to create a copy of PostgreSQL documentation
(in doc-xc directory) and converted all SGML files into source SGML files. The
source SGML files are then preprocessed to add XC specific notes while removing
things which are not applicable in XC. Postgres-XL had borrowed the same
documentation style.
This approach has major problems because:
1. Its not easy to find what has changed in Postgres-XL with respect to
vanilla PostgreSQL
2. The merges from upstream PostgreSQL branches does not merge at all and a
manual merge is required for documentation
With in-place changes, now one can easily use git-diff tools to see the
changes. We also intent to write a separate documentation just highlighing
these changes.
The doc-xc directory is not yet removed. Also, we haven't changed the name
PostgreSQL to Postgres-XL at almost all places. But that's something we can do
as a separate commit, if at all needed.
134 files changed, 10913 insertions, 220 deletions
diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 38f111ef9d..6906e2a1ae 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -266,6 +266,15 @@ </varlistentry> <varlistentry> + <term><acronym>GTM</acronym></term> + <listitem> + <para> + Global Transaction Manager + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><acronym>GSSAPI</acronym></term> <listitem> <para> @@ -287,6 +296,15 @@ </varlistentry> <varlistentry> + <term><acronym>GXID</acronym></term> + <listitem> + <para> + Global Transaction Identifier + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><acronym>HBA</acronym></term> <listitem> <para> diff --git a/doc/src/sgml/add-node.sgml b/doc/src/sgml/add-node.sgml new file mode 100644 index 0000000000..b0bdb54a92 --- /dev/null +++ b/doc/src/sgml/add-node.sgml @@ -0,0 +1,271 @@ +<!-- doc/src/sgml/add-node.sgml --> + +<chapter id="add-node"> + <title>Adding a New Node</title> + + <indexterm zone="add-node"> + <primary>Add a new node</primary> + </indexterm> + + <para> + This chapter outlines steps to add a new Coordinator or a Datanode to a running cluster. + Note that an easier way to do this is to make use of the pgxc_ctl utility. + </para> + + <para> + + </para> + + <sect1 id="add-node-coordinator"> + <title>Adding a New Coordinator</title> + + <indexterm zone="add-node-coordinator"> + <primary>Add a new coordinator</primary> + </indexterm> + + <para> + The following steps should be performed to add a new coordinator to a running cluster: + </para> + + <para> + <orderedlist> + <listitem> + <para>Initialize the new coordinator. The following example initilizes a coordinator named coord_3.</para> + <programlisting> + /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_cord3 --nodename coord_3 + </programlisting> + </listitem> + + <listitem> + <para> + Make necessary changes in postgresql.conf of the new coordinator, + in particular specify new coordinator name and pooler port. + </para> + </listitem> + + <listitem> + <para> + Connect to any of the existing coordinators and lock the cluster for backup, do not close this session. + The following example assumes a coordinator is running on port 5432. Make sure the function call returns true. + The detailed description of the function <function>pgxc_lock_for_backup</> can be found + in <xref linkend="functions-pgxc-add-new-node"> + </para> + <programlisting> + ./psql postgres -p 5432 + select pgxc_lock_for_backup(); + </programlisting> + </listitem> + + <listitem> + <para> + Connect to any of the existing coordinators and take backup of the database. + Please note that only schema (i.e. no data) is to be dumped. + Also note the use of <option>--include-nodes</>, so that the <command>CREATE TABLE</> contains <command>TO NODE</> clause. + Similarly <option>--dump-nodes</> ensures that the dump does contain existing nodes and node groups. + </para> + <programlisting> + ./pg_dumpall -p 5432 -s --include-nodes --dump-nodes --file=/some/valid/path/some_file_name.sql + </programlisting> + </listitem> + + <listitem> + <para> + Start the new coordinator specifying <option>--restoremode</> while starting. + The following example starts the new coordinator on port 5455 + </para> + <programlisting> + ./postgres --restoremode -D ../data_cord3 -p 5455 + </programlisting> + <para> + You can use <literal>pg_ctl</literal> with <option>-Z restoremode</option> option. + </para> + <programlisting> + ./pg_ctl start -Z restoremode -D ../data_coord3 -p 5455 + </programlisting> + </listitem> + + <listitem> + <para> + Restore the backup (taken in step 4) by connecting to the new coordinator directly. + </para> + <programlisting> + ./psql -d postgres -f /some/valid/path/some_file_name.sql -p 5455 + </programlisting> + </listitem> + + <listitem> + <para> + Quit the new coordinator. + </para> + </listitem> + + <listitem> + <para> + Start the new coordinator specifying <option>--coordinator</> while starting. + The following example starts the new coordinator on port 5455 + </para> + <programlisting> + ./postgres --coordinator -D ../data_cord3 -p 5455 + </programlisting> + </listitem> + + <listitem> + <para> + Create the new coordinator on rest of the coordinators and reload configuration. + The following example creates coord_3, with host localhost and port 5455. + </para> + <programlisting> + CREATE NODE COORD_3 WITH (HOST = 'localhost', type = 'coordinator', PORT = 5455); + SELECT pgxc_pool_reload(); + </programlisting> + </listitem> + + <listitem> + <para> + Quit the session of step 3, this will unlock the cluster. The new coordinator is now ready. + </para> + </listitem> + + </orderedlist> + </para> + + </sect1> + + <sect1 id="add-node-datanode"> + <title>Adding a New Datanode</title> + + <indexterm zone="add-node-datanode"> + <primary>Add a new Datanode</primary> + </indexterm> + + <para> + Following steps should be performed to add a new datanode to a running cluster: + </para> + + <para> + <orderedlist> + + <listitem> + <para> + Initialize the new datanode. The following example initializes a new datanode named data_node_3. + </para> + <programlisting> + /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data3 --nodename data_node_3 + </programlisting> + </listitem> + + <listitem> + <para> + Make the necessary changes in postgresql.conf of the new datanode, in particular specify new datanode name + </para> + </listitem> + + <listitem> + <para> + Connect to any of the existing coordinators and lock the cluster for backup, do not close this session. + The following example assumes a coordinator is running on port 5432. Make sure the function call returns true. + The detailed description of the function <function>pgxc_lock_for_backup</> can be found + in <xref linkend="functions-pgxc-add-new-node"> + </para> + <programlisting> + ./psql postgres -p 5432 + select pgxc_lock_for_backup(); + </programlisting> + </listitem> + + <listitem> + <para> + Connect to any of the existing datanodes and take backup of the database. + Please note that only schema (i.e. no data) is to be dumped. + The following example assumes that a datanode is running on port 15432. + </para> + <programlisting> + ./pg_dumpall -p 15432 -s --file=/some/valid/path/some_file_name.sql + </programlisting> + </listitem> + + <listitem> + <para> + Start the new datanode specifying <option>--restoremode</> while starting the it. + The following example starts the new datanode on port 35432. + </para> + <programlisting> + ./postgres --restoremode -D ../data3 -p 35432 + </programlisting> + <para> + You can use <literal>pg_ctl</literal> with <option>-Z restoremode</option> option. + </para> + <programlisting> + ./pg_ctl start -Z restoremode -D ../data3 -p 5455 + </programlisting> + </listitem> + + <listitem> + <para> + Restore the backup (taken in step 4) by connecting to the new datanode directly. + </para> + <programlisting> + ./psql -d postgres -f /some/valid/path/some_file_name.sql -p 35432 + </programlisting> + </listitem> + + <listitem> + <para> + Quit the new datanode. + </para> + </listitem> + + <listitem> + <para> + Start the new datanode specifying --datanode while starting. + </para> + <programlisting> + ./postgres --datanode -D ../data3 -p 35432 + </programlisting> + </listitem> + + <listitem> + <para> + Create the new datanode on all the coordinators and reload configuration. + The following example creates data_node_3, with host localhost and port 35432. + </para> + <programlisting> + CREATE NODE DATA_NODE_3 WITH (HOST = 'localhost', type = 'datanode', PORT = 35432); + SELECT pgxc_pool_reload(); + </programlisting> + </listitem> + + <listitem> + <para> + Create the new datanode on all the other datanodes too and reload configuration. + The following example creates data_node_3, with host localhost and port 35432. + </para> + <programlisting> + EXECUTE DIRECT ON (DATA_NODE_1) 'CREATE NODE DATA_NODE_3 WITH (HOST = ''localhost'', type = ''datanode'', PORT = 35432)'; + EXECUTE DIRECT ON (DATA_NODE_2) 'CREATE NODE DATA_NODE_3 WITH (HOST = ''localhost'', type = ''datanode'', PORT = 35432)'; + EXECUTE DIRECT ON (DATA_NODE_3) 'ALTER NODE DATA_NODE_3 WITH (HOST = ''localhost'', type = ''datanode'', PORT = 35432)'; + EXECUTE DIRECT ON (DATA_NODE_1) 'SELECT pgxc_pool_reload()'; + EXECUTE DIRECT ON (DATA_NODE_2) 'SELECT pgxc_pool_reload()'; + EXECUTE DIRECT ON (DATA_NODE_3) 'SELECT pgxc_pool_reload()'; + </programlisting> + </listitem> + + <listitem> + <para> + Quit the session of step 3, this will unlock the cluster. + </para> + </listitem> + + <listitem> + <para> The new datanode is now ready. + Redistribute existing data by using <command>ALTER TABLE +<replaceable>my_table</replaceable> ADD NODE (DATA_NODE_3)</>. + </para> + </listitem> + + </orderedlist> + </para> + + </sect1> + +</chapter> diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 1e45511cc6..d66268e0ab 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -134,6 +134,20 @@ DETAIL: Key (city)=(Berkeley) is not present in table "cities". foreign keys will definitely improve the quality of your database applications, so you are strongly encouraged to learn about them. </para> + + <para> + Please note that primary key and reference key are both allowed + only when these columns are distribution keys when tables are + distributed. As a default, <productname>Postgres-XC</> + distributes each row of tables based upon the value of the first + column of the table. You can choose any column as a basis of + table distribution, or you can have copies of a table in all the + Datanodes. + </para> + + <para> + Please refer to <xref linkend="sql-select"> for details. + </para> </sect1> diff --git a/doc/src/sgml/arch-dev.sgml b/doc/src/sgml/arch-dev.sgml index c835e87215..a961634c40 100644 --- a/doc/src/sgml/arch-dev.sgml +++ b/doc/src/sgml/arch-dev.sgml @@ -553,3 +553,517 @@ </sect1> </chapter> + <chapter id="xc-overview"> + <title>Overview of <productname>Postgres-XL</productname> Internals</title> + + <para> + This chapter gives an overview of the internal structure + of <productname>Postgres-XL</productname>. + </para> + + <sect1 id="xc-overview-components"> + <title><productname>Postgres-XL</productname> Components</title> + <para> + As described + in <xref linkend="intro-whatis">, <productname>Postgres-XL</productname> + is a database cluster which consists of multiple database servers + based + upon <productname>PostgreSQL</productname>. <productname>Postgres-XL</productname> + provides global transparent transaction management to all the + database servers involved and provide both read and write + scalability. + </para> + + <para> + To achieve these features, <productname>Postgres-XL</productname> + is composed of three major components as follows: + + <variablelist> + <varlistentry> + <term>GTM</term> + <listitem> + <para> + GTM stands for Global Transaction Manager. It provides global + transaction IDs and snapshots for each transaction + in the <productname>Postgres-XL</productname> database cluster. + It also provide several global values such as sequences and + global timestamps. + </para> + <para> + To improve scalability itself, each server hardware or virtual + machine may have GTM-Proxy. GTM-Proxy groups commands and + response from/to GTM to reduce number of interaction and the + amount of data which GTM reads and writes. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term>Coordinator</term> + <listitem> + <para> + Coordinator is an entry point + for <productname>Postgres-XL</productname> from applications. + You can configure more than one Coordinators in the + same <productname>Postgres-XL</productname>. With the help + of GTM, they provide transparent concurrency and integrity of + transactions globally. Applications can choose any + Coordinator to connect to. Any Coordinator provides the + same view of the database. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term>Datanode</term> + <listitem> + <para> + Datanode stores user data. As described + in <xref linkend="whatis-postgres-xl-in-short"> + and <xref linkend="SQL-CREATETABLE">, more than one Datanodes + can be configured. Each table can be replicated or + distributed among Datanodes. A table is distributed, you can + choose a column as the distribute key, whose value is used to + determine which Datanode each row should be stored. + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + </sect1> + + <sect1 id="xc-overview-gtm"> + <title>GTM and Global Transaction Management</title> + <sect2 id="xc-overview-gtm-pgreview"> + <title>Review of <productname>PostgreSQL</productname> Transaction Management Internals</title> + <para> + In PostgreSQL, each transaction is given unique ID called + transaction ID (or XID). XID is given in ascending order to + distinguish which transaction is older/newer. + <footnote> + <para> + More precisely, XID is 32bit integer. When XID reaches the max + value, it wraps around to the lowest value (3, as to the latest + definition). PostgreSQL has a means to handle this, as well as + Postgres-XL. For simplicity, it will not be described in this + document. + </para> + </footnote> + When a transaction tries to read a tuple, + <footnote> + <para> + This description is somewhat simplified for explanation. You + will find the precise rule in <filename>tqual.c</filename> file + in PostgreSQL's source code. + </para> + </footnote> + each tuple has a set of XIDs to indicate transactions which + created and deleted the tuple. So if the target tuple is created + by an active transaction, it is not committed or aborted and the + transaction should ignore such tuple. In such way (in practice, + this is done by versup module in PostgreSQL core), if we give + each transaction a unique transaction Id throughout the system + and maintain snapshot what transaction is active, not only in a + single server but transaction in all the servers, we can maintain + global consistent visibility of each tuple even when a server + accepts new statement from other transactions running on the + other server. + </para> + <para> + These information is stored in "<varname>xmin</varname>" and + "<varname>xmax</varname>" fields of each row of table. When + we <command>INSERT</command> rows, <varname>XID</varname> of + inserting transaction is recorded at xmin field. When we update + rows of tables (with <command>UPDATE</command> + or <command>DELETE</command> statement), PostgreSQL does not + simply overwrite the old rows. Instead, PostgreSQL + "<emphasis>marks</emphasis>" the old rows as + "<emphasis>deleted</emphasis>" by writing updating + transaction's <varname>XID</varname> to xmax field. In the case + of <command>UPDATE</command> (just + like <command>INSERT</command>), new rows are created whose xmin + field is "<emphasis>marked</emphasis>" + with <varname>XID</varname>s of the creating transaction. + </para> + <para> + These "<varname>xmin</varname>" and "<varname>xmax</varname>" are + used to determine which row is visible to a transaction. To do + this, PostgreSQL needs a data to indicate what transactions are + running, which is called the "<emphasis>snapshot</emphasis>". + </para> + <para> + If the creating transaction is not running, visibility of each + row depends upon the fact if the creating transaction was + committed or aborted. Suppose a row of a table which was created + by some transaction and is not deleted yet. If the creating + transaction is running, such row is visible to the transaction + which created the row, but not visible to other transactions. If + the creating transaction is not running and was committed the row + is visible. If the transaction was aborted, this row is not + visible. + </para> + <para> + Therefore, PostgreSQL needs two kinds of information to determine + "which transaction is running" and "if an old transaction was + committed or aborted." + </para> + <para> + The former information is obtained as + "<emphasis>snapshot</emphasis>." PostgreSQL maintains the latter + information as "<filename>CLOG</filename>." + </para> + <para> + PostgreSQL uses all these information to determine which row is + visible to a given transaction. + </para> + </sect2> + + <sect2 id="xc-overview-global-mvcc"> + <title>Making Transaction Management Global</title> + <para> + In Postgres-XL, the following features of transaction management + and visibility checking extracted out from the nodes and pulled + into the GTM. + </para> + <itemizedlist> + <listitem> + <para> + Assigning XID globally to transactions (GXID, Global + Transaction ID). This can be done globally to identify each + Transactions in the system. + </para> + </listitem> + <listitem> + <para> + Providing snapshots. GTM collects all the transaction's status + (running, committed, aborted etc.) to provide snapshots globally + (global snapshot). Please note that each global snapshot + includes <varname>GXID</varname> initiated by other + Coordinators or Datanodes. This is needed because some older + transaction may visit new server after a while. In this case, + if <varname>GXID</varname> of such a transaction is not + included in the snapshot, this transaction may be regarded as + "old enough" and uncommitted rows may be + read. If <varname>GXID</varname> of such transaction is + included in the snapshot from the beginning, such inconsistency + does not take place. + </para> + </listitem> + </itemizedlist> + <para> + To do this, <productname>Postgres-XL</productname> introduced a dedicated component called + GTM (Global Transaction Manager). GTM runs on one of the servers + and provides unique and ordered transaction id to each transaction + running on <productname>Postgres-XL</productname> servers. Because this is a globally unique + ID, we call this <varname>GXID</varname> (Global Transaction Id). + </para> + <para> + GTM receives <varname>GXID</varname> request from transactions + and provide <varname>GXID</varname>. It also keeps track of all + the transactions when it started and finished to generate + snapshots used to control each tuple visibility. Because snapshots + here is also a global property, it is called <emphasis>Global + Snapshot</emphasis>. + </para> + <para> + As long as each transaction runs with a <varname>GXID</varname> and + a Global Snapshot, it can maintain consistent visibility throughout + the system and it is safe to run transactions in parallel in any + servers. On the other hand, a transaction, composed of multiple + statements, can be executed using multiple servers maintaining + database consistency. + </para> + <para> + GTM provides Global Transaction Id to each transaction and keeps + track of the status of all the transactions, whether it is + running, committed or aborted, to calculate global snapshots to + maintain tuple visibility. + </para> + <para> + For this purpose, each transaction reports when it starts and + ends, as well as when it issues <command>PREPARE</command> + command in two-phase commit protocol. + </para> + <para> + Each transaction requests snapshots according to the transaction + isolation level as done in PostgreSQL. If the transaction + isolation level is "<emphasis>read committed</emphasis>", then + transaction will request a snapshot for each statement. If it is + "<emphasis>serializable</emphasis>" transaction will request a + snapshot at the beginning of transaction and reuse it thought the + transaction. + </para> + </sect2> + + <sect2 id="xc-overview-gtm-proxy"> + <title>Improving GTM Performance</title> + <para> + Because GTM can be regarded as "serializing" all the transaction + processing, people may think that GTM can be a performance + bottleneck. + </para> + + <para> + In fact, GTM can limit the whole scalability. GTM should not be + used in very slow network environment such as wide area + network. GTM architecture is intended to be used with Gigabit + local network. It is encouraged to install Postgres-XL with a local + Gigabit network with minimum latency, that is, use as few + switches involved in the connection among GTM, Coordinator and + Datanodes. + In addition, consider putting all components on their own subnet + if you have multiple network ports in the systems. + </para> + + <sect3> + <title>Primitive GTM Implementation</title> + + <para> + Primitive GTM implementation can be done as follows: + </para> + + <procedure> + <step> + <para> + The Coordinator backend is provided with a GTM client library to + obtain GXID and snapshots and to report the transaction status. + </para> + </step> + + <step> + <para> + GTM opens a port to accept connections from each Coordinator and + Datanode backend. When GTM accepts a connection, it creates a + thread (GTM Thread) to handle requests to GTM from the connected + Coordinator backend. + </para> + </step> + + <step> + <para> + GTM Thread receives each request, records it and + sends <varname>GXID</varname>, <emphasis>snapshot</emphasis> + and other response to the Coordinator backend. + </para> + </step> + + <step> + <para> + They are repeated until the Coordinator backend requests + disconnect. + </para> + </step> + </procedure> + + </sect3> + + <sect3> + <title>GTM Proxy Implementation</title> + + <para> + Each transaction is issuing + requests to GTM frequently. We can collect them into single + block of requests in each Coordinator to reduce the amount of + interaction by using a <emphasis>GTM-Proxy</emphasis>. + </para> + + <para> + In this configuration, each Coordinator and Datanode backend + does not connect to GTM directly. Instead, we have GTM Proxy + between GTM and Coordinator backend to group multiple requests + and responses. GTM Proxy, like GTM explained in the previous + sections, accepts connections from the Coordinator + backend. However, it does not create new thread. The following + paragraphs explains how GTM Proxy is initialized and how it + handles requests from Coordinator backends. + </para> + + <para> + GTM Proxy, as well as GTM, is initialized as follows: + </para> + + <procedure> + <step> + <para> + GTM starts up normally, but now can accept connections from + GTM proxies. + </para> + </step> + + <step> + <para> + GTM Proxy starts up. GTM Proxy creates GTM Proxy Threads. Each + GTM Proxy Thread connects to the GTM in advance. The number of + GTM Proxy Threads can be specified at the startup. A typical + number of threads is one or two so it can save the number of + connections between GTM and Coordinators. + </para> + </step> + + <step> + <para> + GTM Main Thread waits for the request connection from each + backend. + </para> + </step> + + </procedure> + + <para> + When each Coordinator backend requests for connection, the Proxy + Main Thread assigns a GTM Proxy Thread to handle + request. Therefore, one GTM Proxy Thread handles multiple + Coordinator backends. If a Coordinator has one hundred + Coordinator backends and one GTM Proxy Thread, this thread takes + care of one hundred Coordinator backend. + </para> + + <para> + Then GTM Proxy Thread scans all the requests from Coordinator + backend. If Coordinator is busy, it is expected to capture + more requests in a single scan. Therefore, the proxy can group + many requests into single block of requests, to reduce the + number of interaction between GTM and the Coordinator. + </para> + + <para> + Furthermore, in a single scan, we may have multiple request for + snapshots. Because these requests can be regarded as received at + the same time, we can represent multiple snapshots with single + one. This will reduce the amount of data which GTM provides. + </para> + + </sect3> + </sect2> + + <sect2 id="xc-overview-Coordinator"> + <title>Coordinator</title> + <para> + Coordinator handles SQL statements from applications and + determines which Datanode should be involved and generates local + SQL statements for each Datanode. In the most simplest case, if + a single Datanode is involved, the Coordinator simply proxies + incoming statements to the Datanode. In more complicated cases, + for example, if the target Datanode cannot be determined, then + the Coordinator generates local statements for each Datanode, + collects the result to materialize at the Coordinator for further + handling. In this case, the Coordinator will try to optimize the + plan by + <itemizedlist> + <listitem> + <para> + Pushdown <command>WHERE</command> clause to Datanodes, + </para> + </listitem> + <listitem> + <para> + Pushdown <emphasis>joins</emphasis> to Datanodes, + </para> + </listitem> + <listitem> + <para> + Pushdown <emphasis>projection</emphasis> (column list in <command>SELECT</command> clause), + </para> + </listitem> + <listitem> + <para> + Pushdown <command>ORDER BY</command> clause, as well as other clauses. + </para> + </listitem> + </itemizedlist> + + If a transaction is involved by more than one Datanodes and/or + Coordinators, the Coordinator will handle the transaction with + two-phase commit protocol internally. + </para> + + <para> + In the case of aggregate + functions, <productname>Postgres-XL</productname> introduced new + function collection function between existing transition function + and finalize function. Collection function runs on the + Coordinator to collect all the intermediate results from involved + Datanodes. For details, see <xref linkend="xaggr"> + and <xref linkend="SQL-CREATEAGGREGATE">. + </para> + + <para> + In the case of reading replicated tables, the Coordinator can choose + any Datanode to read. The most efficient way is to select one + running in the same hardware or virtual machine. This is + called <emphasis>preferred Datanode</emphasis> and can be + specified by a GUC local to each Coordinator. + </para> + + <para> + On the other hand, in the case of writing replicated tables, all + the Coordinators choose the same Datanode to begin with to avoid + update conflicts. This is called <emphasis>primary + Datanode</emphasis>. + </para> + + <para> + Coordinators also take care of DDL statements. Because DDL + statements handles system catalogs, which are replicated in all + the Coordinators and Datanodes, they are proxied to all the + Coordinators and Datanodes. To synchronize the catalog update in + all the nodes, the Coordinator handles DDL with two-phase commit + protocol internally. + </para> + + </sect2> + + <sect2 id="xc-overview-Datanode"> + <title>Datanode</title> + <para> + While Coordinators handle cluster-wide SQL statements, Datanodes + take care of just local issues. In this sense, Datanodes are + essentially <productname>PostgreSQL</productname> servers except + that transaction management information is obtained from GTM, as + well as other global value. + </para> + + </sect2> + + + <sect2 id="xc-overview-pooler"> + <title>Coordinator And Datanode Connection</title> + + <para> + The number of connections between Coordinators and Datanodes may + increase from time to time. This may leave unused connection and + waste system resources. Repeating real connect and disconnect + requires Datanode backend initialization which increases latency + and also wastes system resources. + </para> + + <para> + For example, as in the case of GTM, if each Coordinator has one + hundred connections to applications and we have ten Coordinators, + after a while, each Coordinator may have connection to each data + node. It means that each Coordinator backend has ten connections + to Coordinators and each Coordinator has one thousand (10 x 10) + connections to Coordinators. + </para> + + <para> + Because we consume much more resources for locks and other + control information per backend and only a few of such connection + is active at a given time, it is not a good idea to hold such + unused connections between Coordinator and Datanode. + </para> + + <para> + To improve this, Postgres-XL is equipped with connection pooler + between Coordinator and Datanode. When a Coordinator backend + requires connection to a Datanode, the pooler looks for + appropriate connection from the pool. If there's an available + one, the pooler assigns it to the Coordinator backend. When the + connection is no longer needed, the Coordinator backend returns + the connection to the pooler. The pooler does not disconnect the + connection. It keeps the connection to the pool for later reuse, + keeping Datanode backend running. + </para> + + </sect2> + + </sect1> + </chapter> diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml index d527208271..54fcbe8b56 100644 --- a/doc/src/sgml/auto-explain.sgml +++ b/doc/src/sgml/auto-explain.sgml @@ -32,6 +32,14 @@ LOAD 'auto_explain'; that. </para> + <para> + To log plans on Datanodes, you must preload this module in each + Datanode. This module will log local plans of each node. For + example, a Coordinator log will include the plan for Coordinator only. + the Corresponding plan in Datanodes will be found in each Datanode's + log. + </para> + <sect2> <title>Configuration Parameters</title> diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index 75dabe9f29..e46cc83eba 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -101,6 +101,17 @@ pg_dump <replaceable class="parameter">dbname</replaceable> > <replaceable cl exclusive lock, such as most forms of <command>ALTER TABLE</command>.) </para> + <important> + <para> + In <productname>Postgres-XL</>, <application>pg_dump</> + and <application>pg_dumpall</> backs up all the information stored + both in Coordinators and Datanodes. + That is, it dumps the entire database just like in regular PostgreSQL + and outputs similarly. The output will contain additional table + distribution information. + </para> + </important> + <sect2 id="backup-dump-restore"> <title>Restoring the Dump</title> @@ -343,6 +354,12 @@ pg_dump -j <replaceable class="parameter">num</replaceable> -F d -f <replaceable <title>File System Level Backup</title> <para> + In <productname>Postgres-XL</productname>, file system level backup covers + only each Coordinator or Datanode. To make file system level backup, you + should backup each Coordinator and Datanode manually. + </para> + + <para> An alternative backup strategy is to directly copy the files that <productname>PostgreSQL</> uses to store the data in the database; <xref linkend="creating-cluster"> explains where these files diff --git a/doc/src/sgml/biblio.sgml b/doc/src/sgml/biblio.sgml index e2cd69d278..7968372ae5 100644 --- a/doc/src/sgml/biblio.sgml +++ b/doc/src/sgml/biblio.sgml @@ -537,5 +537,30 @@ [email protected] </confgroup> </biblioentry> + <biblioentry id="OZS91"> + <title>Principles of Distributed Database Systems</title> + <titleabbrev>Ozsu and Valduriez, 1991</titleabbrev> + <edition>Second Edition</edition> + <authorgroup> + <author> + <firstname>M. Tamer</firstname> + <surname>Ozsu</surname> + </author> + <author> + <firstname>Patrick</firstname> + <surname>Valduriez</surname> + </author> + </authorgroup> + <isbn>0-13-659707-6</isbn> + <pubdate>1991</pubdate> + <publisher> + <publishername>Prentice Hall</publishername> + </publisher> + <copyright> + <year>1991, 1999</year> + <holder>Prentice-Hall, Inc.</holder> + </copyright> + </biblioentry> + </bibliodiv> </bibliography> diff --git a/doc/src/sgml/btree-gist.sgml b/doc/src/sgml/btree-gist.sgml index f4afc09546..e251a1d0a8 100644 --- a/doc/src/sgml/btree-gist.sgml +++ b/doc/src/sgml/btree-gist.sgml @@ -39,6 +39,13 @@ as described below. </para> + <important> + <para> + <productname>Postgres-XL</productname> does not support exclusion constraint +currently. + </para> + </important> + <para> Also, for data types for which there is a natural distance metric, <filename>btree_gist</> defines a distance operator <literal><-></>, diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 2c2190f13d..106b12fbdb 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -332,6 +332,22 @@ <entry><link linkend="catalog-pg-user-mapping"><structname>pg_user_mapping</structname></link></entry> <entry>mappings of users to foreign servers</entry> </row> + + <row> + <entry><link linkend="catalog-pgxc-class"><structname>pgxc_class</structname></link></entry> + <entry>replication or distribution information of tables (Postgres-XL only)</entry> + </row> + + <row> + <entry><link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link></entry> + <entry>Postgres-XL cluster nodes (Postgres-XC only)</entry> + </row> + + <row> + <entry><link linkend="catalog-pgxc-group"><structname>pgxc_group</structname></link></entry> + <entry>Postgres-XL cluster node groups (Postgres-XC only)</entry> + </row> + </tbody> </tgroup> </table> @@ -406,6 +422,12 @@ <entry>Transition function</entry> </row> <row> + <entry><structfield>aggcollectfn</structfield></entry> + <entry><type>regproc</type></entry> + <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry> + <entry>Collection function (Only for Postgres-XL>)</entry> + </row> + <row> <entry><structfield>aggfinalfn</structfield></entry> <entry><type>regproc</type></entry> <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry> @@ -451,7 +473,7 @@ <entry><structfield>aggtranstype</structfield></entry> <entry><type>oid</type></entry> <entry><literal><link linkend="catalog-pg-type"><structname>pg_type</structname></link>.oid</literal></entry> - <entry>Data type of the aggregate function's internal transition (state) data</entry> + <entry>Data type of the aggregate function's internal transition and collection (state) data (Revised for Postgres-XL)</entry> </row> <row> <entry><structfield>aggtransspace</structfield></entry> @@ -486,6 +508,17 @@ </entry> </row> <row> + <entry><structfield>agginitcollect</structfield></entry> + <entry><type>text</type></entry> + <entry></entry> + <entry> + The initial value of the collection state. This is a text + field containing the initial value in its external string + representation. If this field is null, the collection state + value starts out null. (Only for Postgres-XL) + </entry> + </row> + <row> <entry><structfield>aggminitval</structfield></entry> <entry><type>text</type></entry> <entry></entry> @@ -7565,6 +7598,228 @@ </table> </sect1> + <sect1 id="catalog-pgxc-class"> + <title><structname>pgxc_class</structname></title> + + <indexterm zone="catalog-pgxc-class"> + <primary>pgxc_class</primary> + </indexterm> + + <para> + The catalog <structname>pgxc_class</structname> stores information + whether each table is replicated or distributed, as well as + distribution method and the distribution column. + </para> + + <table> + <title><structname>pgxc_class</> Columns</title> + + <tgroup cols="4"> + <thead> + <row> + <entry>Name</entry> + <entry>Type</entry> + <entry>References</entry> + <entry>Description</entry> + </row> + </thead> + + <tbody> + <row> + <entry><structfield>pcrelid</structfield></entry> + <entry><type>oid</type></entry> + <entry><link linkend="catalog-pg-class"><structname>pg_class</structname></link><literal>.oid</literal></entry> + <entry>OID of the table</entry> + </row> + + <row> + <entry><structfield>pclocatortype</structfield></entry> + <entry><type>char</type></entry> + <entry></entry> + <entry> + Type of locator. + </entry> + </row> + + <row> + <entry><structfield>pcattnum</structfield></entry> + <entry><type>int2</type></entry> + <entry></entry> + <entry> + Column number of used as distribution key. + </entry> + </row> + + <row> + <entry><structfield>pchashalgorithm</structfield></entry> + <entry><type>int2</type></entry> + <entry></entry> + <entry> + Indicates hashing algorithm used to distribute the tuples. + </entry> + </row> + + <row> + <entry><structfield>pchashbuckets</structfield></entry> + <entry><type>int2</type></entry> + <entry></entry> + <entry> + Indicates the number of hash buckets used to distribute duple. + </entry> + </row> + + <row> + <entry><structfield>nodeoids</structfield></entry> + <entry><type>oidvector</type></entry> + <entry><literal><link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>.oid</literal></entry> + <entry> + List of node OIDs where table is located. + This list is ordered by <literal><link linkend="catalog-pgxc-node"><structname> + pgxc_node</structname></link>.node_name</literal>. This list is then + indexed in information in user session cache and reused as a node target list + when doing SQL operations on cluster tables. + </entry> + </row> + + </tbody> + </tgroup> + </table> + </sect1> + + <sect1 id="catalog-pgxc-node"> + <title><structname>pgxc_node</structname></title> + + <indexterm zone="catalog-pgxc-node"> + <primary>pgxc_node</primary> + </indexterm> + + <para> + The catalog <structname>pgxc_node</structname> stores information + about cluster node information, such as connection information with + node port and host, node name, primary and preferred nodes. + </para> + + <table> + <title><structname>pgxc_node</> Columns</title> + + <tgroup cols="4"> + <thead> + <row> + <entry>Name</entry> + <entry>Type</entry> + <entry>References</entry> + <entry>Description</entry> + </row> + </thead> + + <tbody> + <row> + <entry><structfield>node_name</structfield></entry> + <entry><type>name</type></entry> + <entry></entry> + <entry>Name of cluster node</entry> + </row> + + <row> + <entry><structfield>node_type</structfield></entry> + <entry><type>char</type></entry> + <entry></entry> + <entry>Type of cluster node. + It is <literal>C</literal> for a Coordinator. + It is <literal>D</literal> for a Datanode. + </entry> + </row> + + <row> + <entry><structfield>node_port</structfield></entry> + <entry><type>int4</type></entry> + <entry></entry> + <entry>Port number of cluster node</entry> + </row> + + <row> + <entry><structfield>node_host</structfield></entry> + <entry><type>name</type></entry> + <entry></entry> + <entry>Host name or IP number of cluster node</entry> + </row> + + <row> + <entry><structfield>nodeis_primary</structfield></entry> + <entry><type>bool</type></entry> + <entry></entry> + <entry>Defines if node is a primary node. + Only Datanode can be a primary node. + </entry> + </row> + + <row> + <entry><structfield>nodeis_preferred</structfield></entry> + <entry><type>bool</type></entry> + <entry></entry> + <entry>Defines if node is a preferred node. + Only a Datanode can be a preferred node. + </entry> + </row> + + <row> + <entry><structfield>node_id</structfield></entry> + <entry><type>int4</type></entry> + <entry></entry> + <entry>Integer node identifier of node. + It is generated when node is created. + </entry> + </row> + </tbody> + </tgroup> + </table> + </sect1> + + <sect1 id="catalog-pgxc-group"> + <title><structname>pgxc_group</structname></title> + + <indexterm zone="catalog-pgxc-group"> + <primary>pgxc_group</primary> + </indexterm> + + <para> + The catalog <structname>pgxc_group</structname> stores information + about cluster node group information, which is a list of cluster node + OIDs. Groups can be used as aliases. + </para> + + <table> + <title><structname>pgxc_group</> Columns</title> + + <tgroup cols="4"> + <thead> + <row> + <entry>Name</entry> + <entry>Type</entry> + <entry>References</entry> + <entry>Description</entry> + </row> + </thead> + + <tbody> + <row> + <entry><structfield>group_name</structfield></entry> + <entry><type>name</type></entry> + <entry></entry> + <entry>Name of cluster node group</entry> + </row> + + <row> + <entry><structfield>group_members</structfield></entry> + <entry><type>oidvector</type></entry> + <entry><literal><link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>.oid</literal></entry> + <entry>List of node OIDs of the node group</entry> + </row> + + </tbody> + </tgroup> + </table> + </sect1> <sect1 id="views-overview"> <title>System Views</title> diff --git a/doc/src/sgml/citext.sgml b/doc/src/sgml/citext.sgml index 7fdf30252a..949e22876e 100644 --- a/doc/src/sgml/citext.sgml +++ b/doc/src/sgml/citext.sgml @@ -76,15 +76,16 @@ SELECT * FROM tab WHERE lower(col) = LOWER(?); <programlisting> CREATE TABLE users ( - nick CITEXT PRIMARY KEY, + id int PRIMARY KEY, + nick CITEXT, pass TEXT NOT NULL ); -INSERT INTO users VALUES ( 'larry', md5(random()::text) ); -INSERT INTO users VALUES ( 'Tom', md5(random()::text) ); -INSERT INTO users VALUES ( 'Damian', md5(random()::text) ); -INSERT INTO users VALUES ( 'NEAL', md5(random()::text) ); -INSERT INTO users VALUES ( 'Bjørn', md5(random()::text) ); +INSERT INTO users VALUES (1, 'larry', md5(random()::text) ); +INSERT INTO users VALUES (2, 'Tom', md5(random()::text) ); +INSERT INTO users VALUES (3, 'Damian', md5(random()::text) ); +INSERT INTO users VALUES (4, 'NEAL', md5(random()::text) ); +INSERT INTO users VALUES (5, 'Bjørn', md5(random()::text) ); SELECT * FROM users WHERE nick = 'Larry'; </programlisting> @@ -93,6 +94,10 @@ SELECT * FROM users WHERE nick = 'Larry'; the <structfield>nick</> column was set to <literal>larry</> and the query was for <literal>Larry</>. </para> + <para> + Please note that the column of the type <literal>CITEXT</literal> + cannot be the distribution column; + </para> </sect2> <sect2> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 4b7bd8a86e..ddf81ba0af 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -1,7 +1,7 @@ <!-- doc/src/sgml/config.sgml --> <chapter id="runtime-config"> - <title>Server Configuration</title> + <title>Coordinator and Datanode Configuration</title> <indexterm> <primary>configuration</primary> @@ -10,9 +10,10 @@ <para> There are many configuration parameters that affect the behavior of - the database system. In the first section of this chapter we - describe how to interact with configuration parameters. The subsequent sections - discuss each parameter in detail. + the database system. In the first section of this chapter, we + describe how to set configuration parameters for Coordinator and + Datanodes. The subsequent sections discuss each parameter in + detail. </para> <sect1 id="config-setting"> @@ -153,7 +154,7 @@ shared_buffers = 128MB receives a <systemitem>SIGHUP</> signal; this signal is most easily sent by running <literal>pg_ctl reload</> from the command line or by calling the SQL function <function>pg_reload_conf()</function>. The main - server process also propagates this signal to all currently running + Coordinator/Datanode process also propagates this signal to all currently running server processes, so that existing sessions also adopt the new values (this will happen after they complete any currently-executing client command). Alternatively, you can @@ -300,7 +301,7 @@ UPDATE pg_settings SET setting = reset_val WHERE name = 'configuration_parameter passed to the <command>postgres</command> command via the <option>-c</> command-line parameter. For example, <programlisting> -postgres -c log_connections=yes -c log_destination='syslog' +postgres --coordinator -c log_connections=yes -c log_destination='syslog' </programlisting> Settings provided in this way override those set via <filename>postgresql.conf</> or <command>ALTER SYSTEM</>, @@ -670,6 +671,18 @@ include_dir 'conf.d' same or higher value than on the master server. Otherwise, queries will not be allowed in the standby server. </para> + + <para> + In the case of the Coordinator, this parameter determines how many + connections can each Coordinator accept. + </para> + + <para> + In the case of the Datanode, number of connection to each + Datanode may become as large as <varname>max_connections</> + multiplied by the number of Coordinators. + </para> + </listitem> </varlistentry> @@ -1444,6 +1457,14 @@ include_dir 'conf.d' same or higher value than on the master server. Otherwise, queries will not be allowed in the standby server. </para> + + <para> + Even though your application does not issue <command>PREPARE + TRANSACTION</> explicitly, Coordinator may generate this + command when an updating transaction involves more than one + Coordinators and Datanodes. For Datanodes, you should specify + this value as the same value as <varname>max_connections</>. + </para> </listitem> </varlistentry> @@ -2637,6 +2658,11 @@ include_dir 'conf.d' <sect1 id="runtime-config-replication"> <title>Replication</title> + <para> + Streaming replication only been thoroughly tested in asynchronous mode + with <productname>Postgres-XL</productname>. + </para> + <para> These settings control the behavior of the built-in <firstterm>streaming replication</> feature (see @@ -5703,6 +5729,14 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; to problems such as forward references. Set this parameter to <literal>off</> before loading functions on behalf of other users; <application>pg_dump</> does so automatically. + <note> + <para> + In <productname>Postgres-XL</> setting <varname>check_function_bodies</varname> + to <literal>off</> is not recommended. If SQL functions are created after setting + <varname>check_function_bodies</varname> to <literal>off</>, the behaviour of the + function when executed is un-defined. + </para> + </note> </para> </listitem> </varlistentry> @@ -6618,6 +6652,15 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir' to investigate locking delays you might want to set a shorter than normal <varname>deadlock_timeout</varname>. </para> + <para> + <productname>Postgres-XL</> does not detect global deadlocks + where multiple node (Coordinators and/or Datanodes) are + involved. To get around this it is recommended to set + <varname>statement_timeout</varname> to cause those statements to + fail in a normal processing environment. If however, it is a reporting + environment, you may want to set this higher globally, or just for + the session temporarily. + </para> </listitem> </varlistentry> @@ -7617,6 +7660,11 @@ LOG: CleanUpLock: deleting: lock(0xb7acd844) id(24688,24696,0,0,0,1) macro was defined when <productname>PostgreSQL</productname> was compiled. </para> + <para> + <productname>Postgres-XL</> does not detect global deadlocks + where multiple nodes (Coordinators and/or Datanodes) are + involved. + </para> </listitem> </varlistentry> @@ -7819,4 +7867,283 @@ LOG: CleanUpLock: deleting: lock(0xb7acd844) id(24688,24696,0,0,0,1) </table> </sect1> + <sect1 id="pg-xc-specifics"> + <title>Postgres-XL Specific Parameters</title> + <para> + Because <productname>Postgres-XL</> distributes data into multiple + Datanodes and multiple Coordinators can accept transactions in + parallel, the Coordinators must know what Coordinators and + Datanodes to connect to. The Coordinators and Datanodes also must know + where they can request transaction information. The following + describes these additional GUC parameters. + </para> + + <variablelist> + + <varlistentry id="guc-max-pool-size" xreflabel="max_pool_size"> + <term><varname>max_pool_size</varname> (<type>integer</type>) + <indexterm> + <primary><varname>max_pool_size</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specify the maximum connection pool of the Coordinator to Datanodes. + Because each transaction can be involved by all the Datanodes, + this parameter should at least be <varname>max_connections</> + multiplied by number of Datanodes. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-min-pool-size" xreflabel="min_pool_size"> + <term><varname>min_pool_size</varname> (<type>integer</type>) + <indexterm> + <primary><varname>min_pool_size</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Minumum number of the connection from the Coordinator to Datanodes. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-pool-maintenance-timeout" xreflabel="pool_maintenance_timeout"> + <term><varname>pool_maintenance_timeout</varname> (<type>integer</type>) + <indexterm> + <primary><varname>pool_maintenance_timeout</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + This parameter specifies how long to wait until pooler + maintenance is performed. During such maintenance, + old idle connections are discarded. + This parameter is useful in multi-tenant environments where + many connections to many different databases may be used, + so that idle connections may cleaned up. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-remote-query-cost" xreflabel="remote_query_cost"> + <term><varname>remote_query_cost</varname> (<type>integer</type>) + <indexterm> + <primary><varname>remote_query_cost</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + This parameter specifies the cost overhead of setting up + a remote query to obtain remote data. It is used by + the planner in costing queries. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-network-byte-cost" xreflabel="network_byte_cost"> + <term><varname>network_byte_cost</varname> (<type>integer</type>) + <indexterm> + <primary><varname>network_byte_cost</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + This parameter is used in query cost planning to + estimate the cost involved in row shipping and obtaining + remote data based on the expected data size. + Row shipping is expensive and adds latency, so this + setting helps to favor plans that minimizes row shipping. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-sequence-range" xreflabel="sequence_range"> + <term><varname>sequence_range</varname> (<type>integer</type>) + <indexterm> + <primary><varname>sequence_range</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + This parameter is used to get several sequence values + at once from GTM. This greatly speeds up COPY and INSERT SELECT + operations where the target table uses sequences. + <productname>Postgres-XL</productname> will not use this entire + amount at once, but will increase the request size over + time if many requests are done in a short time frame in + the same session. + After a short time without any sequence requests, decreases back down to 1. + Note that any settings here are overriden if the CACHE clause was + used in <xref linkend='sql-createsequence'> or <xref linkend='sql-altersequence'>. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-max-coordinators" xreflabel="max_coordinators"> + <term><varname>max_coordinators</varname> (<type>integer</type>) + <indexterm> + <primary><varname>max_coordinators</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Maximum number of Coordinators that can be configured in the cluster. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-max-datanodes" xreflabel="max_datanodes"> + <term><varname>max_datanodes</varname> (<type>integer</type>) + <indexterm> + <primary><varname>max_datanodes</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Maximum number of Datanodes that can be configured in the cluster. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-pgxc-node-name" xreflabel="pgxc_node_name"> + <term><varname>pgxc_node_name</varname> (<type>integer</type>) + <indexterm> + <primary><varname>pgxc_node_name</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the node name. A node uses this parameter to identify itself + with <link linkend="catalog-pgxc-node"><structname>pgxc_node</structname> + </link>.nodename + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-pooler-port" xreflabel="pooler_port"> + <term><varname>pooler_port</varname> (<type>integer</type>) + <indexterm> + <primary><varname>pooler_port</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the port number assigned to the connection pooler process. + This is used in both the Coordinator and Datanode processes to + connect to other nodes. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-gtm-host" xreflabel="gtm_host"> + <term><varname>gtm_host</varname> (<type>string</type>) + <indexterm> + <primary><varname>gtm_host</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the host name or IP address of the <filename>GTM</>. If you use <filename>GTM-Proxy</>, specify the <filename>GTM-Proxy</>'s one. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-gtm-port" xreflabel="gtm_port"> + <term><varname>gtm_port</varname> (<type>integer</type>) + <indexterm> + <primary><varname>gtm_port</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifieis the port number of GTM. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-xc-maintenance-mode" xreflabel="xc_maintenance_mode"> + <term><varname>xc_maintenance_mode</varname> (<type>bool</type>) + <indexterm> + <primary><varname>xc_maintenance_mode</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specify to enter into maintenance mode. Turning this parameter to on will + change behavior of several statements. Each statement + behavior may not be compatible in future releases, so users + must be extremely careful when they attempt to set this + value to <literal>on</literal>. + </para> + <para> + Behavior of <xref linkend="sql-commit-prepared"> and + <xref linkend="sql-rollback-prepared"> statements are affected by + this parameter. + </para> + <para> + If this is set to ON, you can do maintenance work to update + Datanode locally through <literal>EXECUTE DIRECT</literal> or connecting + directly to Datanodes. As a DBA, you are totally responsible to any side + effects. You must be extremely careful not to bring in any inconsistencies + to Postgres-XL database cluster. + </para> + <para> + This parameter can only be set by superuser + with <literal>SET</literal> command. Otherwise, the setting + will be ignored. + </para> + <para> + As a session parameter, this parameter is shared among all + the connections of the session. It affects originating Coordinator as + well as remote nodes involved in session. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-shared-queues" xreflabel="shared_queues"> + <term><varname>shared_queues</varname> (<type>integer</type>) + <indexterm> + <primary><varname>shared_queues</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Datanode Only + </para> + <para> + For some joins that occur in queries, data from one Datanode may + need to be joined with data from another Datanode. + <productname>Postgres-XL</productname> uses shared queues for this purpose. + During execution each Datanode knows if it needs to produce or consume + tuples, or both. + </para> + <para> + Note that there may be mulitple shared_queues used even for a single + query. So a value should be set taking into account the number of + connections it can accept and expected number of such joins occurring + simultaneously. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-shared-queue-size" xreflabel="shared_queue_size"> + <term><varname>shared_queue_size</varname> (<type>integer</type>) + <indexterm> + <primary><varname>shared_queue_size</> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Datanode Only + </para> + <para> + This parameter sets the size of each each shared queue allocated. + </para> + </listitem> + </varlistentry> + </variablelist> + + </sect1> </chapter> diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml index 1b3d2d93c7..2752ac850f 100644 --- a/doc/src/sgml/contrib.sgml +++ b/doc/src/sgml/contrib.sgml @@ -132,6 +132,10 @@ CREATE EXTENSION <replaceable>module_name</> FROM unpackaged; &pgstatstatements; &pgstattuple; &pgtrgm; + &pgxcclean; + &pgxcctl; + &pgxcddl; + &pgxcmonitor; &postgres-fdw; &seg; &sepgsql; diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml index 9d5ce953f1..d96c6e4974 100644 --- a/doc/src/sgml/datatype.sgml +++ b/doc/src/sgml/datatype.sgml @@ -4522,6 +4522,11 @@ SELECT * FROM pg_attribute (The system columns are further explained in <xref linkend="ddl-system-columns">.) </para> + <para> + Please note that <productname>Postgres-XL</> enforces OID identity only locally. + Different object at different Coordinator or Datanode may be assigned the same + OID value. + </para> </sect1> <sect1 id="datatype-pg-lsn"> diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml index b07ac48c00..053af5b608 100644 --- a/doc/src/sgml/dblink.sgml +++ b/doc/src/sgml/dblink.sgml @@ -8,6 +8,14 @@ </indexterm> <para> + <filename>dblink</> module has not been tested + with <productname>Postges-XL</> yet. Although there're no reason + that <filename>dblink</> does not run + in <productname>Postgres-XL</>, the development team leaves the test + and the use of this module entirely to users. + </para> + + <para> <filename>dblink</> is a module that supports connections to other <productname>PostgreSQL</> databases from within a database session. diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml index 0aa0c13c5c..4ffa09bf30 100644 --- a/doc/src/sgml/ddl.sgml +++ b/doc/src/sgml/ddl.sgml @@ -133,6 +133,78 @@ CREATE TABLE products ( <indexterm> <primary>table</primary> + <secondary>replication</secondary> + </indexterm> + + + <para> + In <productname>Postgres-XL</>, each table can be distributed or replicated + among Datanodes. By distributing tables each query + has the potential to be handled by a single or small + subset of the Datanodes and more transactions can be handled in parallel. + If you replicate tables, they can be read from just a single location, though + all writes will need to take place on all nodes. + </para> + + <para> + When you distribute a table, you can choose almost any column of a fundamental data type as distribution + column. + For details, please refer to <xref linkend="SQL-CREATETABLE">. + The Datanode for a particular row is determined based upon the value of the distribution + column, chosen if DISTRIBUTE BY HASH was used. + If not used, the first column of a primary key or unique constraint is used + if present in the CREATE TABLE clause. + If neither of those are present, the first column of a foreign key constraint + is used, the idea being that child data can be col-located on the same node + as the parent table. + If no such constraint is defined, Postgres-XL will choose the first reasonable + column it can find, that is the first column that has a deterministically + distributable data type. + + You may also choose another distribution method such as <type>MODULO</> and + <type>ROUNDROBIN</>. + To specify what column to choose as the distribution column and what value test to + choose, you can do as follows: +<programlisting> +CREATE TABLE products ( + product_no integer, + name text, + price numeric +) DISTRIBUTE BY HASH(product_no); +</programlisting> + In this case, the column <type>product_no</> is chosen as the distribute column and + the target Datanode of each row is determined based upon the hash value of the column. + You can use <type>MODULO</> to specify modulo to test and determine the target Datanode. + You can also specify <type>ROUNDROBIN</> to determine the Datanode by the order each + row is inserted. + </para> + + <para> + Please note that with <type>HASH</> and <type>MODULO</>, the Coordinator may be able to determine the + exact location of target row from incoming statement. This reduces the number of involved Datanodes + and can increase the number of transaction handled in parallel. + </para> + + <para> + On the other hand, if exact location cannot be obtained from incoming statement, the + Coordinator will have to involve all of the Datanodes on which the involved table + is distributed. + </para> + + <para> + To replicate a table into all the Datanodes, specify <type>DISTRIBUTE BY REPLICATION</> as + follows: +<programlisting> +CREATE TABLE products ( + product_no integer, + name text, + price numeric +) DISTRIBUTE BY REPLICATION; +</programlisting> + + </para> + <indexterm> + <primary>table</primary> <secondary>removing</secondary> </indexterm> @@ -569,6 +641,27 @@ CREATE TABLE products ( careful when developing applications that are intended to be portable. </para> + + <para> + In <productname>Postgres-XL</>, in distributed + tables, <literal>UNIQUE</literal> constraints must include the + distribution column of the table. + This is because Postgres-XL currently only allows that it can + push down to the Datanodes to be enforced locally. If we include + the distribution column in unique constraints, it stands to + reason that it can be enforced locally. + + If a table is distributed + by <type>ROUNDROBIN</type>, we cannot enforce <type>UNIQUE</> + constraints because it does not have a distribution column; + it is possible that the same value for a column exists on + multiple nodes. + + There's no restriction in <type>UNIQUE</> constraint in replicated + tables. When an expression is used on a <type>UNIQUE</> constraint, + this expression must contain the distribution column of its parent + table. It cannot use other columns as well. + </para> </sect2> <sect2 id="ddl-constraints-primary-keys"> @@ -643,6 +736,15 @@ CREATE TABLE example ( not enforced by <productname>PostgreSQL</productname>, but it is usually best to follow it. </para> + + <para> + As mentioned when discussing <type>UNIQUE</> constraint, the distribution column + must be included in <type>PRIMARY KEY</type>. Other restrictions + apply to the <type>PRIMARY KEY</> as well. When an expression is used on + a <type>PRIMARY KEY</> constraint, this expression must contain + the distribution column of its parent table. It cannot use other + columns as well. + </para> </sect2> <sect2 id="ddl-constraints-fk"> @@ -678,18 +780,27 @@ CREATE TABLE products ( price numeric ); </programlisting> + Let's also assume you have a table storing orders of those products. We want to ensure that the orders table only contains orders of products that actually exist. So we define a foreign key constraint in the orders table that references the products - table: + table. <programlisting> CREATE TABLE orders ( - order_id integer PRIMARY KEY, + order_id integer, product_no integer <emphasis>REFERENCES products (product_no)</emphasis>, quantity integer -); +) DISTRIBUTE BY HASH(product_no); </programlisting> + </para> + <para> + Please note that column with <type>REFERENCES</> should be the distribution column. + In this case, we cannot add <type>PRIMARY KEY</> to <type>order_id</> + because <type>PRIMARY KEY</type> must be the distribution column as well. + This limitation is introduced because constraints are enforced only locally in each Datanode. + </para> + <para> Now it is impossible to create orders with non-NULL <structfield>product_no</structfield> entries that do not appear in the products table. @@ -713,6 +824,13 @@ CREATE TABLE orders ( </programlisting> because in absence of a column list the primary key of the referenced table is used as the referenced column(s). + + <note> + <para> + In <productname>Postgres-XL</productname>, you cannot omit the column name + in REFERENCES clause. + </para> + </note> </para> <para> @@ -729,6 +847,12 @@ CREATE TABLE t1 ( </programlisting> Of course, the number and type of the constrained columns need to match the number and type of the referenced columns. + <note> + <para> + In <productname>Postgres-XL</productname>, you cannot specify both PRIMARY + KEY and REFERENCES key for different columns. + </para> + </note> </para> <para> @@ -764,6 +888,13 @@ CREATE TABLE order_items ( </programlisting> Notice that the primary key overlaps with the foreign keys in the last table. + + <note> + <para> + In <productname>Postgres-XL</productname>, you cannot specifiy more than +one foreign key constraints. + </para> + </note> </para> <indexterm> @@ -913,6 +1044,12 @@ CREATE TABLE circles ( Adding an exclusion constraint will automatically create an index of the type specified in the constraint declaration. </para> + <note> + <para> + <productname>Postgres-XL</productname> does not support exclusion + constraints. + </para> + </note> </sect2> </sect1> @@ -950,6 +1087,13 @@ CREATE TABLE circles ( <type>oid</type> (same name as the column); see <xref linkend="datatype-oid"> for more information about the type. </para> + <para> + Please note that <productname>Postgres-XL</> does not enforce + OID integrity among the cluster. OIDs are assigned locally in + each Coordinator and Datanode. You can use this in expressions + but you should not expect OID values are the same throughout the + <productname>XL</> cluster. + </para> </listitem> </varlistentry> @@ -969,6 +1113,13 @@ CREATE TABLE circles ( <structfield>oid</structfield> column of <structname>pg_class</structname> to obtain the table name. </para> + <para> + Please note that <productname>Postgres-XL</> does not enforce + OID integrity among the cluster. OIDs are assigned locally in + each Coordinator and Datanode. You can use this in expressions, + but you should not expect OID values are the same throughout the + <productname>XL</> cluster. + </para> </listitem> </varlistentry> @@ -1049,6 +1200,12 @@ CREATE TABLE circles ( identifier. The OID, or even better a user-defined serial number, should be used to identify logical rows. </para> + <para> + In <productname>Postgres-XL</>, ctid is local to Coordinators + and Datanodes. It is not good practice to use this value in + SQL statements and can be very dangerous when using it to update + data. + </para> </listitem> </varlistentry> </variablelist> @@ -1169,6 +1326,33 @@ CREATE TABLE circles ( here. </para> + <para> + In addition to these actions, in <productname>Postgres-XL</productname> you + can also use <command>ALTER TABLE</> command to: + <itemizedlist spacing="compact"> + <listitem> + <para>Change distribution strategy</para> + </listitem> + <listitem> + <para>Add a node to existing distribution target</para> + </listitem> + <listitem> + <para>Remove a node from existing distribution target</para> + </listitem> + </itemizedlist> + </para> + <para> + In <productname>Postgres-XL</> the following are not allowed: + <itemizedlist spacing="compact"> + <listitem> + <para>Modifying distribution columns definition</para> + </listitem> + <listitem> + <para>Modifying distribution column values</para> + </listitem> + </itemizedlist> + </para> + <sect2 id="ddl-alter-adding-a-column"> <title>Adding a Column</title> @@ -1266,6 +1450,9 @@ ALTER TABLE products ALTER COLUMN product_no SET NOT NULL; The constraint will be checked immediately, so the table data must satisfy the constraint before it can be added. </para> + <para> + Please remember that the distribution column has to be included in UNIQUE and REFERENCE constraints. + </para> </sect2> <sect2 id="ddl-alter-removing-a-constraint"> @@ -1398,6 +1585,76 @@ ALTER TABLE products RENAME TO items; </programlisting> </para> </sect2> + <sect2> + <title>Adding a Node to Distribution Target</title> + + <indexterm> + <primary>table</primary> + <secondary>distribution</secondary> + </indexterm> + <para> + To add a new node to existing distribution: +<programlisting> +ALTER TABLE products ADD NODE (datanode_3); +</programlisting> + The datanode must be a valid datanode name. If its a new datanode, then it +must have been added to the cluster by appropriate mechanism. +<!-- To learn how to +add a node to the cluster, please see <xref linkend="add-node">. --> + </para> + <para> + This command will redistribute the existing table data so as to include the + new node. For example, in case of <type>DISTRIBUTE BY HASH</> strategy, hash + value for each row must be recomputed and the row must be moved to the selected + node based on the new count of datanodes in the distribution list. This + operation requires an exclusive lock on the table and hence all read and write + access to the table is temporarily blocked. This can be important especially if + the table is large and requires considerable time for + redistribution. + </para> + </sect2> + <sect2> + <title>Removing a Node from Distribution Target</title> + + <indexterm> + <primary>table</primary> + <secondary>distribution</secondary> + </indexterm> + <para> + To remove a node from existing distribution: +<programlisting> +ALTER TABLE products DELETE NODE (datanode_3); +</programlisting> + It must be noted that if you want to remove a node from the cluster, its + important that you first remove such node from all existing tables using that + node and then only remove the node from the cluster. + </para> + <para> + Like the previous command, this also takes an exclusive lock on the table, + blocking all read and write access to the table. + </para> + </sect2> + <sect2> + <title>Changing Distribution Strategy</title> + + <indexterm> + <primary>table</primary> + <secondary>distribution</secondary> + </indexterm> + <para> + To change distribution strategy for a table: +<programlisting> +ALTER TABLE products DISSTRIBUTE BY REPLICATION; +</programlisting> +You can change distribution strategy for a table with this command. For +example, a table which was previously distributed by <type>HASH</> can now be +distributed by <type>REPLICATION</type>. + </para> + <para> + Like the previous command, this also takes an exclusive lock on the table, + blocking all read and write access to the table. + </para> + </sect2> </sect1> <sect1 id="ddl-priv"> diff --git a/doc/src/sgml/dml.sgml b/doc/src/sgml/dml.sgml index cd36a73811..8ee5cc08b8 100644 --- a/doc/src/sgml/dml.sgml +++ b/doc/src/sgml/dml.sgml @@ -207,6 +207,10 @@ UPDATE products SET price = price * 1.10; UPDATE mytable SET a = 5, b = 3, c = 1 WHERE a > 0; </programlisting> </para> + <para> + Please remember that <productname>Postgres-XL</> does not allow to + modify the value of distribution column. + </para> </sect1> <sect1 id="dml-delete"> diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index a12fee73db..440fda497e 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -49,6 +49,8 @@ <!ENTITY config SYSTEM "config.sgml"> <!ENTITY user-manag SYSTEM "user-manag.sgml"> <!ENTITY wal SYSTEM "wal.sgml"> +<!ENTITY add-node SYSTEM "add-node.sgml"> +<!ENTITY remove-node SYSTEM "remove-node.sgml"> <!-- programmer's guide --> <!ENTITY bgworker SYSTEM "bgworker.sgml"> @@ -137,6 +139,10 @@ <!ENTITY pgstattuple SYSTEM "pgstattuple.sgml"> <!ENTITY pgtrgm SYSTEM "pgtrgm.sgml"> <!ENTITY postgres-fdw SYSTEM "postgres-fdw.sgml"> +<!ENTITY pgxcclean SYSTEM "pgxcclean.sgml"> +<!ENTITY pgxcctl SYSTEM "pgxc_ctl-ref.sgml"> +<!ENTITY pgxcddl SYSTEM "pgxcddl.sgml"> +<!ENTITY pgxcmonitor SYSTEM "pgxcmonitor.sgml"> <!ENTITY seg SYSTEM "seg.sgml"> <!ENTITY contrib-spi SYSTEM "contrib-spi.sgml"> <!ENTITY sepgsql SYSTEM "sepgsql.sgml"> @@ -164,6 +170,7 @@ <!ENTITY sourcerepo SYSTEM "sourcerepo.sgml"> <!ENTITY release SYSTEM "release.sgml"> +<!ENTITY release-xl-9.5 SYSTEM "release-xl-9.5.sgml"> <!ENTITY release-9.5 SYSTEM "release-9.5.sgml"> <!ENTITY release-9.4 SYSTEM "release-9.4.sgml"> <!ENTITY release-9.3 SYSTEM "release-9.3.sgml"> diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index d49cd43428..c275f811e7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14841,6 +14841,11 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n); <entry><type>text</type></entry> <entry><productname>PostgreSQL</> version information. See also <xref linkend="guc-server-version-num"> for a machine-readable version.</entry> </row> + <row> + <entry><literal><function>pgxc_version()</function></literal></entry> + <entry><type>text</type></entry> + <entry><productname>Postgres-XL Cluster</> version information</entry> + </row> </tbody> </tgroup> </table> @@ -15604,6 +15609,13 @@ SELECT pg_type_is_visible('myschema.widget'::regtype); type name in this way — if the name can be recognized at all, it must be visible. </para> + <para> + Because value of OID is enforced unique only in each Coordinator + or Datanode in <productname>Postgres-XL</>, you should use these + functions locally, typically through <type>EXECUTE DIRECT</> + statement. + </para> + <indexterm> <primary>format_type</primary> </indexterm> @@ -16014,6 +16026,13 @@ SELECT typlen FROM pg_type WHERE oid = pg_typeof(33); </para> <para> + Please note that OID is valid locally only in each Coordinator and + Datanode. You should use specific OID value in statements + targeted to specific Coordinator or Datanode by <type>EXECUTE + DIRECT</> statement. + </para> + + <para> The expression <literal>collation for</literal> returns the collation of the value that is passed to it. Example: <programlisting> @@ -16237,6 +16256,12 @@ SELECT collation for ('foo' COLLATE "de_DE"); are stored globally as well. </para> + <para> + In <productname>Postgres-XL</>, OID is maintained locally in each Coordinator and Datanode. + If you specify specific OID value, you should do it in SQL statements targeted to a specific + Coordinator or Datanode by <type>EXECUTE DIRECT</> statement. + </para> + <indexterm> <primary>txid_current</primary> </indexterm> @@ -16277,6 +16302,11 @@ SELECT collation for ('foo' COLLATE "de_DE"); <tbody> <row> + <entry><literal><function>pgxc_is_committed(transaction_id)</function></literal></entry> + <entry><type>bool</type></entry> + <entry>If given xid (gxid) is committed or aborted. NULL indicates the status is unknown (running, not yet started, prepared, frozen, etc).</entry> + </row> + <row> <entry><literal><function>txid_current()</function></literal></entry> <entry><type>bigint</type></entry> <entry>get current transaction ID, assigning a new one if the current transaction does not have one</entry> @@ -16513,6 +16543,13 @@ SELECT set_config('log_statement_stats', 'off', false); </programlisting> </para> + <para> + Similar to the SET command, the set_config() function makes sure + that as long as the session/transaction is active, the new parameter + value is set across all the nodes on which the SQL queries are being run + as part of that particular session or transaction. + </para> + </sect2> <sect2 id="functions-admin-signal"> @@ -16623,6 +16660,11 @@ SELECT set_config('log_statement_stats', 'off', false); subprocess. </para> + <para> + Please note that these functions works just locally. To issue + these functions to another Coordinators or Datanodes, you should + issue these functions through <type>EXECUTE DIRECT</> statement. + </para> </sect2> <sect2 id="functions-admin-backup"> @@ -16854,6 +16896,12 @@ postgres=# SELECT * FROM pg_xlogfile_name_offset(pg_stop_backup()); </para> <para> + Please note that these functions works just locally. To issue + these functions to another Coordinators or Datanodes, you should + issue these functions through <type>EXECUTE DIRECT</> statement. + </para> + + <para> <function>pg_xlog_location_diff</> calculates the difference in bytes between two transaction log locations. It can be used with <structname>pg_stat_replication</structname> or some functions shown in @@ -17008,6 +17056,12 @@ postgres=# SELECT * FROM pg_xlogfile_name_offset(pg_stop_backup()); </table> <para> + Please note that these functions works just locally. To issue + these functions to another Coordinators or Datanodes, you should + issue these functions through <type>EXECUTE DIRECT</> statement. + </para> + + <para> While recovery is paused no further database changes are applied. If in hot standby, all new queries will see the same consistent snapshot of the database, and no further query conflicts will be generated until @@ -17705,6 +17759,19 @@ postgres=# SELECT * FROM pg_xlogfile_name_offset(pg_stop_backup()); </para> <para> + The object size functions pg_database_size, pg_indexes_size, pg_relation_size, + pg_table_size, and pg_total_relation_size return the cumulative size + from all the Datanodes. For e.g., pg_relation_size returns the sum of disk + space used up by the specified fork at all the Datanodes where the table is + distributed or replicated. If the table is replicated on 3 tables, the size + will be 3 times that of individual nodes. If you need to retrieve the local + results from a particular Coordinator or Datanode, you should issue these + function calls explicitly through <type>EXECUTE DIRECT</> statement. All other + system functions run locally at the Coordinator, unless explicitly specified + otherwise in this document. + </para> + + <para> The functions shown in <xref linkend="functions-admin-dblocation"> assist in identifying the specific disk files associated with database objects. </para> @@ -17779,6 +17846,12 @@ postgres=# SELECT * FROM pg_xlogfile_name_offset(pg_stop_backup()); </para> <para> + Please note that these functions works just locally. To issue + these functions to another Coordinators or Datanodes, you should + issue these functions through <type>EXECUTE DIRECT</> statement. + </para> + + <para> <function>pg_filenode_relation</> is the reverse of <function>pg_relation_filenode</>. Given a <quote>tablespace</> OID and a <quote>filenode</>, it returns the associated relation's OID. For a table @@ -18157,6 +18230,392 @@ SELECT (pg_stat_file('filename')).modification; at session end, even if the client disconnects ungracefully.) </para> + <para> + The advisory lock functions are aware of the Postgres XL cluster. Hence, + if you use a function like pg_advisory_lock() from a particular Coordinator, + the resource will be locked across the complete cluster, so another + application calling the same function from a different Coordinator will see + this lock, and will wait on the resource until the lock is released. + This applies to both transaction and session level advisory locks. + </para> + + <para> + The functions shown in <xref linkend="functions-pgxc-pooler"> manage + Postgres-XL pooler. For details about the Postgres-XL pooler, see + <xref linkend="xc-overview-pooler">. + </para> + <table id="functions-pgxc-pooler"> + <title>Postgres-XL pooler functions</title> + <tgroup cols="3"> + <thead> + <row><entry>Name</entry> <entry>Return Type</entry> <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry> + <literal><function>pgxc_pool_check()</function></literal> + </entry> + <entry><type>boolean</type></entry> + <entry>Check if connection data cached in pooler is consistent with + <link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>. + </entry> + </row> + <row> + <entry> + <literal><function>pgxc_pool_reload()</function></literal> + </entry> + <entry><type>boolean</type></entry> + <entry>Reload connection data cached in pooler and reload sessions in server</entry> + </row> + </tbody> + </tgroup> + </table> + + <indexterm> + <primary>pgxc_pool_check</primary> + </indexterm> + <para> + <function>pgxc_pool_check</> verifies if connection data cached in Postgres-XL pooler + is consistent with <link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link> + catalog. Data checked for consistency is Node Oid (<literal><link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>.oid</literal>), + node port (<literal><link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>.node_port</literal>) + and node host (<literal><link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>.node_host</literal>). + </para> + <indexterm> + <primary>pgxc_pool_reload</primary> + </indexterm> + <para> + <function>pgxc_pool_reload</> reloads connection data cached in pooler from + <link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link> catalog + and reloads all the information info cached in pooler. All the active transactions + are aborted and all existing pooler connections are dropped. This results in having + all the temporary and prepared objects dropped on remote and local node for session. + </para> + + <para> + The functions shown in <xref linkend="functions-pgxc-add-new-node"> manage + addition of a new node to Postgres-XL cluster. + </para> + <table id="functions-pgxc-add-new-node"> + <title>Postgres-XL functions to manage addition of a new node</title> + <tgroup cols="3"> + <thead> + <row><entry>Name</entry> <entry>Return Type</entry> <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry> + <literal><function>pgxc_lock_for_backup()</function></literal> + </entry> + <entry><type>boolean</type></entry> + <entry>Locks the cluster for taking backup that would be restored on the new node to be added. + </entry> + </row> + </tbody> + </tgroup> + </table> + + <indexterm> + <primary>pgxc_lock_for_backup</primary> + </indexterm> + <para> + <function>pgxc_lock_for_backup</> locks the cluster for taking backup using <application>pg_dump/pg_dumpall</application>. + Locking means that we disallow the statemets that change the portions of the catalog which are + backed up by <application>pg_dump/pg_dumpall</application>. When locked only the following utility statemets are allowed. + Please note that this function does not impact SELECTs or DMLs. + </para> + + <table id="utility-statements-allowed-while-locked"> + <title>Utility statements allowed while the cluster is locked for backup</title> + <tgroup cols="1"> + <thead> + <row><entry>Statement</entry> + </row> + </thead> + + <tbody> + + + <row> + <entry> + <literal><command>EXECUTE</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>CREATE NODE</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>START TRANSACTION</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>BEGIN</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>COMMIT</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>ROLLBACK</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>PREPARE TRANSACTION</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>COMMIT PREPARED</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>ROLLBACK PREPARED</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>DECLARE CURSOR</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>CLOSE CURSOR</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>FETCH</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>TRUNCATE</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>COPY</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>PREPARE</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>DEALLOCATE</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>DO</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>NOTIFY</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>LISTEN</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>UNLISTEN</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>LOAD</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>CLUSTER</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>VACUUM</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>EXPLAIN</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>SET</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>SHOW</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>DISCARD</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>LOCK</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>SET CONSTRAINTS</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>CHECKPOINT</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>CREATE BARRIER</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>REINDEX</command></literal> + </entry> + </row> + + + <row> + <entry> + <literal><command>CLEAN CONNECTION</command></literal> + </entry> + </row> + + + </tbody> + </tgroup> + </table> + + <para> + To lock the cluster for backup while adding a new node <productname>Postgres-XL</productname> uses advisory locks. + Every time a disallowed statement is issued the system tries to acquire a transaction + level advisory lock in shared mode and the lock is released when + the DDL or the transaction issuing the DDL ends. The function <function>pgxc_lock_for_backup</> + tries to acquire the same advisory lock in exclusive mode at session level. It is therefore necessary + to keep the session issuing <function>pgxc_lock_for_backup</> alive as long as the issuer wants + the system to keep the lock. + <productname>Postgres-XL</productname> uses the key pair (0xFFFF, 0xFFFF) as object ID while using + advisory lock for backup. + The function fails to acquire the lock if any one of the following is true: + + <itemizedlist> + <listitem> + <para> + The caller of the function is not a superuser. + </para> + </listitem> + + <listitem> + <para> + There are some uncommitted prepared transactions, because they might + contain any utility statement belonging to the disallowed group. + </para> + </listitem> + + <listitem> + <para> + The lock is already held by an uncommitted transaction that has issued + a utility statement belonging to the disallowed group. + </para> + </listitem> + + <listitem> + <para> + The lock is already held by a previous call to the same function + and current request is being issued from a different session. + </para> + </listitem> + + </itemizedlist> + + </para> <indexterm> <primary>pg_advisory_xact_lock</primary> </indexterm> @@ -18206,6 +18665,12 @@ SELECT (pg_stat_file('filename')).modification; <primary>suppress_redundant_updates_trigger</primary> </indexterm> + <note> + <para> + Triggers are currently not supported in <productname>Postgres-XL</productname>. + </para> + </note> + <para> Currently <productname>PostgreSQL</> provides one built in trigger function, <function>suppress_redundant_updates_trigger</>, @@ -18271,6 +18736,11 @@ FOR EACH ROW EXECUTE PROCEDURE suppress_redundant_updates_trigger(); <primary>pg_event_trigger_ddl_commands</primary> </indexterm> + <note> + <para> + Event triggers are currently not supported in <productname>Postgres-XL</productname>. + </para> + </note> <para> <function>pg_event_trigger_ddl_commands</> returns a list of <acronym>DDL</acronym> commands executed by each user action, diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml index d2f7fec523..e6cfdc5171 100644 --- a/doc/src/sgml/high-availability.sgml +++ b/doc/src/sgml/high-availability.sgml @@ -1554,6 +1554,11 @@ if (!triggered) explained below. </para> + <para> + Hot Standby is not supported by the current version + of <productname>Postgres-XL</productname>. + </para> + <sect2 id="hot-standby-users"> <title>User's Overview</title> diff --git a/doc/src/sgml/history.sgml b/doc/src/sgml/history.sgml index a7f4b701ea..2a0f774f4b 100644 --- a/doc/src/sgml/history.sgml +++ b/doc/src/sgml/history.sgml @@ -219,4 +219,29 @@ then can be found in <xref linkend="release">. </para> </sect2> +<sect2 id="pgxc-history"> + <title><productname>Postgres-XL</productname></title> + + <indexterm zone="pgxc-history"> + <primary>history</primary> + <secondary>of Postgres-XL</secondary> + </indexterm> + + <para> + In 2010, NTT's Open Source Software Center approached EnterpriseDB + to build off of NTT OSSC's experience with a project called RitaDB + and EnterpriseDB's experience with a project called GridSQL, + and the result was a new project, Postgres-XC. + </para> + <para> + In 2012, a company called StormDB was formed with some of the original key Postgres-XC developers. StormDB added enhancements, including MPP parallelism for performance and multi-tenant security. + </para> + <para> + In 2013, TransLattice acquired StormDB, and in 2014, open sourced it as Postgres-XL. + </para> + <para> + In 2015, 2ndQuandrant started further work on Postgres-XL. + </para> + +</sect2> </sect1> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 309fd1269b..d0ffa95fb0 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -15,6 +15,12 @@ sensibly. </para> + <para> + Each index is maintained locally in Coordinator and Datanode + in <productname>Postgres-XL</>. + Cross validation of index entries among Coordinators and Datanodes is + not performed in the current implementation. + </para> <sect1 id="indexes-intro"> <title>Introduction</title> @@ -630,6 +636,12 @@ CREATE INDEX test3_desc_index ON test3 (id DESC NULLS LAST); </indexterm> <para> + In <productname>Postgres-XL</>, unique indexes on distributed tables + must contain the distribution column. For replicated + tables, there's no such restriction. + </para> + + <para> Indexes can also be used to enforce uniqueness of a column's value, or the uniqueness of the combined values of more than one column. <synopsis> diff --git a/doc/src/sgml/info.sgml b/doc/src/sgml/info.sgml index a59d0c8a43..b5da113e1c 100644 --- a/doc/src/sgml/info.sgml +++ b/doc/src/sgml/info.sgml @@ -13,12 +13,9 @@ <term>Wiki</term> <listitem> <para> - The <productname>PostgreSQL</productname> <ulink - url="http://wiki.postgresql.org">wiki</ulink> contains the project's <ulink - url="http://wiki.postgresql.org/wiki/Frequently_Asked_Questions">FAQ</> - (Frequently Asked Questions) list, <ulink - url="http://wiki.postgresql.org/wiki/Todo">TODO</> list, and - detailed information about many more topics. + The <productname>Postgres-XL</productname> <ulink + url="http://sourceforge.net/p/postgres-xl/wiki/Home/">wiki</ulink> contains the project's + information, links to related pages and various development information. </para> </listitem> </varlistentry> @@ -27,11 +24,11 @@ <term>Web Site</term> <listitem> <para> - The <productname>PostgreSQL</productname> - <ulink url="http://www.postgresql.org">web site</ulink> + The <productname>Postgres-XL</productname> + <ulink url="http://postgres-xl.sourceforge.net/">web site</ulink> carries details on the latest release and other information to make your work or play with - <productname>PostgreSQL</productname> more productive. + <productname>Postgres-XL</productname> more productive. </para> </listitem> </varlistentry> @@ -42,7 +39,7 @@ <para> The mailing lists are a good place to have your questions answered, to share experiences with other users, and to contact - the developers. Consult the <productname>PostgreSQL</> web site + the developers. Consult the <productname>Postgres-XL</> web site for details. </para> </listitem> diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml index 89648349c2..6dae7edf7b 100644 --- a/doc/src/sgml/installation.sgml +++ b/doc/src/sgml/installation.sgml @@ -30,17 +30,42 @@ in a standalone-ignore clause. <title>Short Version</title> <para> + The following short installation allows to install a simple cluster on a local machine with + 1 Coordinator, 2 Datanodes and 1 GTM. When installing a more complex cluster, you might + change the number of Coordinators and Datanodes, and might have to start nodes on different + servers. + Also, you can instead use the pgxc_ctl utility, which simplifies the installation and configuration process. <synopsis> ./configure -make +gmake su -make install +gmake install adduser postgres -mkdir /usr/local/pgsql/data -chown postgres /usr/local/pgsql/data +mkdir /usr/local/pgsql/data_coord1 +mkdir /usr/local/pgsql/data_datanode1 +mkdir /usr/local/pgsql/data_datanode2 +mkdir /usr/local/pgsql/data_gtm +chown postgres /usr/local/pgsql/data_coord1 +chown postgres /usr/local/pgsql/data_datanode1 +chown postgres /usr/local/pgsql/data_datanode2 +chown postgres /usr/local/pgsql/data_gtm su - postgres -/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data -/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data >logfile 2>&1 & +/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_coord1 --nodename coord1 +/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_datanode1 --nodename datanode1 +/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data_datanode2 --nodename datanode2 +/usr/local/pgsql/bin/initgtm -D /usr/local/pgsql/data_gtm -Z gtm +/usr/local/pgsql/bin/gtm -D /usr/local/pgsql/data_gtm >logfile 2>&1 & +/usr/local/pgsql/bin/postgres --datanode -p 15432 -D /usr/local/pgsql/data_datanode1 >logfile 2>&1 & +/usr/local/pgsql/bin/postgres --datanode -p 15433 -D /usr/local/pgsql/data_datanode2 >logfile 2>&1 & +/usr/local/pgsql/bin/postgres --coordinator -D /usr/local/pgsql/data_coord1 >logfile 2>&1 & +/usr/local/pgsql/bin/psql -c "ALTER NODE coord1 WITH (TYPE = 'coordinator', PORT = 5432)" postgres +/usr/local/pgsql/bin/psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432)" postgres +/usr/local/pgsql/bin/psql -c "CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15433)" postgres +/usr/local/pgsql/bin/psql -c "EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH (TYPE = ''datanode'', PORT = 15432)'" postgres +/usr/local/pgsql/bin/psql -c "EXECUTE DIRECT ON (datanode1) 'CREATE NODE datanode2 WITH (TYPE = ''datanode'', PORT = 15433)'" postgres +/usr/local/pgsql/bin/psql -c "EXECUTE DIRECT ON (datanode2) 'ALTER NODE datanode2 WITH (TYPE = ''datanode'', PORT = 15433)'" postgres +/usr/local/pgsql/bin/psql -c "EXECUTE DIRECT ON (datanode2) 'CREATE NODE datanode1 WITH (TYPE = ''datanode'', PORT = 15432)'" postgres +/usr/local/pgsql/bin/psql -c "SELECT pgxc_pool_reload()" postgres /usr/local/pgsql/bin/createdb test /usr/local/pgsql/bin/psql test </synopsis> @@ -342,26 +367,6 @@ su - postgres <title>Getting The Source</title> <para> - The <productname>PostgreSQL</> &version; sources can be obtained from the - download section of our - website: <ulink url="http://www.postgresql.org/download/"></ulink>. You - should get a file named <filename>postgresql-&version;.tar.gz</filename> - or <filename>postgresql-&version;.tar.bz2</filename>. After - you have obtained the file, unpack it: -<screen> -<userinput>gunzip postgresql-&version;.tar.gz</userinput> -<userinput>tar xf postgresql-&version;.tar</userinput> -</screen> - (Use <command>bunzip2</command> instead of <command>gunzip</command> if you - have the <filename>.bz2</filename> file.) - This will create a directory - <filename>postgresql-&version;</filename> under the current directory - with the <productname>PostgreSQL</> sources. - Change into that directory for the rest - of the installation procedure. - </para> - - <para> You can also get the source directly from the version control repository, see <xref linkend="sourcerepo">. </para> @@ -1524,6 +1529,82 @@ PostgreSQL, contrib and HTML documentation successfully made. Ready to install. <step id="install"> <title>Installing the Files</title> + <para> + Before learning how to install <productname>Postgres-XL</>, you should determine + what you are going to install on each server. The following lists + <productname>Postgres-XL</> components you've built and you're going to install. + + <variablelist> + <varlistentry> + <term><envar>GTM</envar></term> + <listitem> + <para> + GTM stands for Global Transaction Manager. It provides global transaction ID + and snapshot to each transaction in <productname>Postgres-XL</> database cluster. + It also provide several global value such as sequence and global timestamp. + </para> + <para> + GTM itself can be configured as a backup of other GTM as + GTM-Standby so that GTM can continue to run even if main GTM + fails. You may want to install GTM-Standby to separate + server. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>GTM-Proxy</envar></term> + <listitem> + <para> + Because GTM has to take care of each transaction, it has to + read and write enormous amount of messages which may + restrict <productname>Postgres-XL</> scalability. GTM-Proxy is + a proxy of GTM feature which groups requests and response to + reduce network read/write by GTM. Distributing one snapshot to + multiple transactions also contributes to reduce GTM network + workload. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>Coordinator</envar></term> + <listitem> + <para> + The Coordinator is an entry point to <productname>Postgres-XL</> from applications. + You can run more than one Coordinator simultaneously in the cluster. Each Coordinator behaves + just as a <productname>PostgreSQL</> database server, while all the Coordinators + handles transactions in harmonized way so that any transaction coming into one + Coordinator is protected against any other transactions coming into others. + Updates by a transaction is visible immediately to others running in other + Coordinators. + To simplify the load balance of Coordinators and Datanodes, as mentioned + below, it is highly advised to install same number of Coordinator and Datanode + in a server. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><envar>Datanode</envar></term> + <listitem> + <para> + Datanode + </para> + <para> + The Coordinator and Datanode shares the same binary but their behavior is a little + different. The Coordinator decomposes incoming statements into those handled by + Datanodes. If necessary, the Coordinator materializes response from Datanodes + to calculate final response to applications. + </para> + <para> + The Datanode is very close to PostgreSQL itself because it just handles incoming + statements locally. + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + <note> <para> If you are upgrading an existing system be sure to read @@ -1783,6 +1864,15 @@ export MANPATH is not required, however; the settings can be communicated via command line options to most client programs. </para> + + <para> + When, as typical case, you're configuring both Coordinator and + Datanode in a same server, please be careful not to assign same + resource, such as listening point (IP address and port number) to + different component. If you apply single set of environment + described here to different components, they will conflict + and <productname>Postgres-XL</> will not run correctly. + </para> </sect2> </sect1> @@ -1814,6 +1904,61 @@ export MANPATH <step> <para> + If you follow the previous steps, you will have files ready to + distribute to servers where you want to run one or + more <productname>Postgres-XL</> components. + </para> + <para> + After you've installed your build locally, build target + will include the following directories. + +<screen> +bin/ include/ lib/ share/ +</screen> + + <filename>bin</> directory contains executable binaries and + scripts. <filename>include</> contains header files needed to + build <productname>Postgres-XL</> applications. <filename>lib</> + contains shared libraries needed to run binaries, as well as + static libraries which should be included into your application + binaries. Finally, <productname>share</> contains miscellaneous + files <productname>Postgres-XL</> should read at runtime, as well + as sample files. + + </para> + + <para> + + If your servers has sufficient file space, you can copy all the + files to the target server. Total size is less than 30mega + bytes. If you want to install minimum files to each servers, + please follow the following paragraphs. + + </para> + + <para> + For the server to run GTM or GTM-Standby, you need to copy the + following files to your path: <filename>bin/gtm</> and <filename>bin/gtm_ctl</>. + </para> + + <para> + For the server to run GTM-Proxy (the server you run Coordinator and/or Datanode), + you need to copy the following files to your path: <filename>bin/gtm_proxy</filename> + and <filename>bin/gtm_ctl</>. + </para> + + <para> + For server to run Coordinator or Datanode, or both, you should + copy the following files to your + path: <filename>bin/initdb</>. + You should also copy everything in <filename>path</> directory to + your library search path. + </para> + + </step> + + <step> + <para> Create a database installation with the <command>initdb</> command. To run <command>initdb</> you must be logged in to your <productname>PostgreSQL</> server account. It will not work as @@ -1834,6 +1979,12 @@ postgres$ <userinput>/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data</> doesn't already exist) before starting <command>initdb</>, as illustrated here. </para> + + <para> + If you're configuring both Datanodes and Coordinators on the same + server, you should specify different <option>-D</> values for + each of them. + </para> </step> <step> @@ -1847,24 +1998,427 @@ postgres$ <userinput>/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data</> <step> <para> - The previous <command>initdb</> step should have told you how to - start up the database server. Do so now. The command should look + You should configure GTM and GTM-Proxy, as well as + GTM-Standby if you need high-availability capability for GTM + before you really run <productname>Postgres-XL</> database + cluster. You can do the following before you + run <command>initdb</>. + </para> + <para> + Each of GTM, GTM-Proxy and GTM need their own working directories. + Create them as <productname>Postgres-XL</> owner user. Please + assign a port number to each of them, although you don't have to do + any configuration work now. + </para> + </step> + + <step> + <para> + Now you should configure each Coordinator and Datanode. Because + they have to communicate each other and number of servers, + Datanodes and Coordinators depend upon configurations, we do not + provide default configuration files for them. + </para> + <para> + You can configure Datanodes and Coordinators by + editing the <filename>postgresql.conf</> file located under the + directory you specified with <option>-D</> option + of <command>initdb</>. The following paragraphs describe what + parameter to edit at for Coordinators. You can specify + any other <filename>postgresql.conf</> parameters as + standalone <productname>PostgreSQL</>. + </para> + + <variablelist> + <varlistentry> + <term><envar>max_prepared_transactions</envar></term> + <listitem> + <para> + <option>max_prepared_transactions</> specifies maximum number + of two-phase commit transactions. Even if you don't use + explicit two phase commit operation, the Coordinator may issue + two-phase commit operation implicitly if a transaction is + involved with multiple Datanodes and/or Coordinators. You should + specify <option>max_prepared_transactions</> value at + least the number of <option>max_connection</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>min_pool_size</envar><term> + <listitem> + <par> + Coordinator is associated with a connection pooler which takes + care of connection with other Coordinators and Datanodes. This + parameter specifies minimum number of connection to pool. + If you're not configuring <productname>XL</> cluster in + unbalanced way, you should specify the same value to all the + Coordinators. + </par> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>max_pool_size</envar></term> + <listitem> + <para> + This parameter specifies maximum number of the pooled + connection. This value should be at least more than the number + of all the Coordinators and Datanodes. If you specify less + value, you will see very frequent close and ope connection + which leads to serious performance problem. + If you're not configuring <productname>XL</> cluster in + unbalanced way, you should specify the same value to all the + Coordinators. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>pool_conn_keepalive</envar></term> + <listitem> + <para> + This parameter specifies how long to keep the connection alive. + If older than this amount, the pooler discards the connection. + This parameter is useful in multi-tenant environments where + many connections to many different databases may be used, + so that idle connections may cleaned up. It is also useful + for automatically closing connections occasionally in case + there is some unknown memory leak so that this memory can be freed. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>pool_maintenance_timeout</envar></term> + <listitem> + <para> + This parameter specifies how long to wait until pooler + maintenance is performed. During such maintenance, + old idle connections are discarded. + This parameter is useful in multi-tenant environments where + many connections to many different databases may be used, + so that idle connections may cleaned up. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>remote_query_cost</envar></term> + <listitem> + <para> + This parameter specifies the cost overhead of setting up + a remote query to obtain remote data. It is used by + the planner in costing queries. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>network_byte_cost</envar></term> + <listitem> + <para> + This parameter is used in query cost planning to + estimate the cost involved in row shipping and obtaining + remote data based on the expected data size. + Row shipping is expensive and adds latency, so this + setting helps to favor plans that minimizes row shipping. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>sequence_range</envar></term> + <listitem> + <para> + This parameter is used to get several sequence values + at once from GTM. This greatly speeds up COPY and INSERT SELECT + operations where the target table uses sequences. + <productname>Postgres-XL</productname> will not use this entire + amount at once, but will increase the request size over + time if many requests are done in a short time frame in + the same session. + After a short time without any sequence requests, decreases back down to 1. + Note that any settings here are overriden if the CACHE clause was + used in <xref linkend='sql-createsequence'> or <xref linkend='sql-altersequence'>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>max_coordinators</envar></term> + <listitem> + <para> + This parameter specifies maximum number of the Coordinators that can + be added to the cluster. Cluster would have to be restarted to increase + the value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>max_datanodes</envar></term> + <listitem> + <para> + This parameter specifies maximum number of the Datanodes that can + be added to the cluster. Cluster would have to be restarted to increase + the value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>pgxc_node_name</envar></term> + <listitem> + <para> + Specify the name of this cluster node. This name index refers to the + element of <link linkend="catalog-pgxc-node"><structname>pgxc_node + </structname></link>.node_name value, to identify the node self. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>port</envar></term> + <listitem> + <para> + Specify the port number listened to by this Coordinator. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>pooler_port</envar></term> + <listitem> + <para> + Connection pooler needs separate port. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>gtm_port</envar></term> + <listitem> + <para> + Specify the port number of gtm you're connecting to. This is + local to the server and you should specify the port assigned to + the GTM-Proxy local to the Coordinator. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <step> + <para> + Now you should configure <filename>postgresql.conf</> for each + Datanodes. Please note, as in the case of Coordinator, you can + specify other <filename>postgresql.conf</> parameters as in + standalone <productname>PostgreSQL</>. + </para> + + <variablelist> + + <varlistentry> + <term><envar>max_connections</envar></term> + <listitem> + <para> + <option>max_connections</> is, in short, a maximum number of + background processes of the Datanode. You should be careful + to specify reasonable value to this parameter because each + Coordinator backend may have connections to all the Datanodes. + You should specify this value as <option>max_connections</> of + Coordinator multiplied by the number of Coordinators. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>max_prepared_transactions</envar></term> + <listitem> + <para> + <option>max_prepared_transactions</> specifies maximum number + of two-phase commit transactions. Even if you don't use + explicit two phase commit operation, Coordinator may issue + two-phase commit operation implicitly if a transaction is + involved with multiple Datanodes and/or Coordinators. The value + of this parameter should be at least the value + of <option>max_connections</> of Coordinator multiplied by the number of Coordinators. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>port</envar></term> + <listitem> + <para> + Specify the port number listened to by the Datanode. + </para> + </listitem> + </varlistentry> + + <term><envar>gtm_port</envar></term> + <listitem> + <para> + Specify the port number of gtm you're connecting to. This is + local to the server and you should specify the port assigned to + the GTM-Proxy local to the Datanode. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <para> + Postgres-XL introduces some additional parameters for the + Datanodes as well + </para> + + <variablelist> + + <varlistentry> + <term><envar>shared_queues</envar></term> + <listitem> + <para> + For some joins that occur in queries, data from one Datanode may + need to be joined with data from another Datanode. + <productname>Postgres-XL</productname> uses shared queues for this purpose. + During execution each Datanode knows if it needs to produce or consume + tuples, or both. + </para> + <para> + Note that there may be mulitple shared_queues used even for a single + query. So a value should be set taking into account the number of + connections it can accept and expected number of such joins occurring + simultaneously. + </para> + + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>shared_queue_size</envar></term> + <listitem> + <para> + This parameter sets the size of each each shared queue allocated. + </para> + </listitem> + </varlistentry> + + </variablelist> + </step> + + <step> + <para> + Then you are ready to start the <productname>Postgres-XL</> cluster. + First, you should start GTM bu something like: +<programlisting> +gtm -D /usr/local/pgsql/gtm -h localhost -p 20001 -n 1 -x 1000 +</programlisting> + This will start GTM. <option>-h</> specifies IP address or host + name to listen to the connection + from <command>gtm_standby</>. <option>-p</> specifies the port + number to listen to. <option>-n</> specifies the node number + within GTM. This identifies GTM especially when you run + GTM-Standby. <option>-x</> option specifies the value of initial + Global Transaction ID. We need to specify this + because <command>initdb</> consumes some of the transaction ID + locally and GTM must to begin to provide global transaction ID + greater than the consumed one. A value of at least 2000 is believed + to be safe. + </para> + </step> + + <step> + <para> + Next, you should start GTM-Proxy on each server you're running + Coordinator and/or Datanode like: +<programlisting> +gtm_proxy -h localhost -p 20002 -s localhost -t 20001 -i 1 -n 2 -D /usr/local/pgsql/gtm_proxy +</programlisting> + This will start GTM-Proxy. <option>-h</> option is the host name + or IP address which GTM-Proxy listens to. <option>-p</> option + is the port number to listen to. <option>-s</> + and <option>-t</> are IP address or the host + name and the port number of GTM as specified + above. <option>-i</> is the node number of GTM-Proxy beginning + with 1. <option>-n</> is the number of worker thread of + GTM-Proxy. Usually, 1 or 2 is advised. Then <option>-D</> + option is the working directory of the GTM-Proxy. + </para> + <para> + Please note that you should start GTM-Proxy on all the servers + you run Coordinator/Datanode. + </para> + </step> + + <step> + <para> + Now you can start Datanode on each server like: +<programlisting> +postgres --datanode -D /usr/local/pgsql/Datanode +</programlisting> + This will start the Datanode. <option>--datanode</> + specifies <command>postgres</> to start as a + Datanode. <option>-D</> specifies the data directory of the + Datanode. You can specify other options of standalone <command>postgres</>. + </para> + <para> + Please note that you should issue <command>postgres</> command at + all the servers you're running Datanode. + </para> + </step> + + <para> + Finally, you can start Coordinator like: +<programlisting> +postgres --coordinator -D /usr/local/pgsql/Coordinator +</programlisting> + This will start the Coordinator. <option>--coordinator</> + specifies <command>postgres</> to start as a + Coordinator. <option>-D</> specifies the data directory of the + Coordinator. You can specify other options of standalone <command>postgres</>. + </para> + <para> + Please note that you should issue <command>postgres</> command at + all the servers you're running Coordinators. + </para> + <step> + <para> + The previous step should have told you how to + start up the whole database cluster. Do so now. The command should look something like: <programlisting> -/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data +postgres --datanode -D /usr/local/pgsql/Datanode </programlisting> - This will start the server in the foreground. To put the server + This will start the Datanode in the foreground. To put the Datanode in the background use something like: <programlisting> -nohup /usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data \ +nohup postgres --datanode -D /usr/local/pgsql/data \ </dev/null >>server.log 2>&1 </dev/null & </programlisting> + You can apply this to all the other components, GTM, GTM-Proxies, + and Coordinators. </para> <para> - To stop a server running in the background you can type: + To stop a Datanode running in the background you can type: +<programlisting> +kill `cat /usr/local/pgsql/Datanode/postmaster.pid` +</programlisting> + You can apply this to stop a Coordinator too. + </para> + <para> + To stop the GTM running in the background you can type <programlisting> -kill `cat /usr/local/pgsql/data/postmaster.pid` +kill `cat /usr/local/pgsql/gtm/gtm.pid` +</programlisting> + </para> + <para> + To stop a GTM-Proxy running in the background, you can type +<programlisting> +kill `cat /usr/local/pgsql/gtm-proxy/gtm_proxy.pid </programlisting> </para> </step> @@ -1873,14 +2427,15 @@ kill `cat /usr/local/pgsql/data/postmaster.pid` <para> Create a database: <screen> -<userinput>createdb testdb</> +<userinput>psql -p 20004 testdb</> </screen> Then enter: <screen> -<userinput>psql testdb</> +<userinput>psql -p 20004 testdb</> </screen> - to connect to that database. At the prompt you can enter SQL - commands and start experimenting. + Please do not forget to give the port number of one of the + Coordinators. Then you are connected to a Coordinator listening + to the port you specified. </para> </step> </procedure> @@ -1913,14 +2468,6 @@ kill `cat /usr/local/pgsql/data/postmaster.pid` <listitem> <para> - Usually, you will want to modify your computer so that it will - automatically start the database server whenever it boots. Some - suggestions for this are in the documentation. - </para> - </listitem> - - <listitem> - <para> Run the regression tests against the installed server (using <command>make installcheck</command>). If you didn't run the tests before installation, you should definitely do it now. This @@ -1951,47 +2498,17 @@ kill `cat /usr/local/pgsql/data/postmaster.pid` <title>Supported Platforms</title> <para> - A platform (that is, a CPU architecture and operating system combination) - is considered supported by the <productname>PostgreSQL</> development - community if the code contains provisions to work on that platform and - it has recently been verified to build and pass its regression tests - on that platform. Currently, most testing of platform compatibility - is done automatically by test machines in the - <ulink url="http://buildfarm.postgresql.org/">PostgreSQL Build Farm</ulink>. - If you are interested in using <productname>PostgreSQL</> on a platform - that is not represented in the build farm, but on which the code works - or can be made to work, you are strongly encouraged to set up a build - farm member machine so that continued compatibility can be assured. - </para> - - <para> - In general, <productname>PostgreSQL</> can be expected to work on - these CPU architectures: x86, x86_64, IA64, PowerPC, - PowerPC 64, S/390, S/390x, Sparc, Sparc 64, ARM, MIPS, MIPSEL, M68K, - and PA-RISC. Code support exists for M32R and VAX, but these - architectures are not known to have been tested recently. It is often - possible to build on an unsupported CPU type by configuring with - <option>--disable-spinlocks</option>, but performance will be poor. - </para> - - <para> - <productname>PostgreSQL</> can be expected to work on these operating - systems: Linux (all recent distributions), Windows (Win2000 SP4 and later), - FreeBSD, OpenBSD, NetBSD, OS X, AIX, HP/UX, Solaris, - and UnixWare. Other Unix-like systems may also work but are not currently - being tested. In most cases, all CPU architectures supported by - a given operating system will work. Look in - the <xref linkend="installation-platform-notes"> below to see if - there is information - specific to your operating system, particularly if using an older system. + <productname>Postgres-XL</> can be expected to work on these operating systems: + Linux (all recent distributions), FreeBSD and Mac OS X. Other Unix-like systems may + also work but are not currently being tested. </para> <para> If you have installation problems on a platform that is known to be supported according to recent build farm results, please report - it to <email>[email protected]</email>. If you are interested - in porting <productname>PostgreSQL</> to a new platform, - <email>[email protected]</email> is the appropriate place + it to <email>[email protected]</email>. If you are interested + in porting <productname>Postgres-XL</> to a new platform, + <email>[email protected]</email> is the appropriate place to discuss that. </para> </sect1> @@ -2023,6 +2540,12 @@ kill `cat /usr/local/pgsql/data/postmaster.pid` <secondary>installation on</secondary> </indexterm> + <note> + <para> + <productname>Postgres-XL</productname> has not been tested on AIX. + </para> + </note> + <para> PostgreSQL works on AIX, but getting it installed properly can be challenging. AIX versions from 4.3.3 to 6.1 are considered supported. @@ -2354,6 +2877,13 @@ createlang: language installation failed: ERROR: could not load library "/opt/d <secondary>installation on</secondary> </indexterm> + + <note> + <para> + <productname>Postgres-XL</productname> has not been tested on Cygwin. + </para> + </note> + <para> PostgreSQL can be built using Cygwin, a Linux-like environment for Windows, but that method is inferior to the native Windows build @@ -2457,6 +2987,13 @@ make MAX_CONNECTIONS=5 check <secondary>installation on</secondary> </indexterm> + + <note> + <para> + <productname>Postgres-XL</productname> has not been tested on HP-UX. + </para> + </note> + <para> PostgreSQL 7.3+ should work on Series 700/800 PA-RISC machines running HP-UX 10.X or 11.X, given appropriate system patch levels @@ -2549,6 +3086,13 @@ PHSS_30849 s700_800 u2comp/be/plugin library Patch <secondary>installation on</secondary> </indexterm> + + <note> + <para> + <productname>Postgres-XL</productname> has not been tested on MinGW. + </para> + </note> + <para> PostgreSQL for Windows can be built using MinGW, a Unix-like build environment for Microsoft operating systems, or using @@ -2614,6 +3158,13 @@ PHSS_30849 s700_800 u2comp/be/plugin library Patch <secondary>installation on</secondary> </indexterm> + <note> + <para> + <productname>Postgres-XL</productname> has not been tested on SCO. + </para> + </note> + + <indexterm zone="installation-notes-sco"> <primary>UnixWare</primary> <secondary>installation on</secondary> @@ -2768,6 +3319,13 @@ MANPATH=/usr/lib/scohelp/%L/man:/usr/dt/man:/usr/man:/usr/share/man:scohelp:/usr <secondary>installation on</secondary> </indexterm> + <note> + <para> + <productname>Postgres-XL</productname> has not been tested on Solaris. + </para> + </note> + + <para> PostgreSQL is well-supported on Solaris. The more up to date your operating system, the fewer issues you will experience; details diff --git a/doc/src/sgml/intro.sgml b/doc/src/sgml/intro.sgml index f0dba6f56f..e50547820e 100644 --- a/doc/src/sgml/intro.sgml +++ b/doc/src/sgml/intro.sgml @@ -5,12 +5,23 @@ <para> This book is the official documentation of - <productname>PostgreSQL</productname>. It has been written by the - <productname>PostgreSQL</productname> developers and other + <productname>Postgres-XL</productname>. It has been written by the + <productname>Postgres-XL</productname> developers and other volunteers in parallel to the development of the - <productname>PostgreSQL</productname> software. It describes all + <productname>Postgres-XL</productname> software. It describes all the functionality that the current version of - <productname>PostgreSQL</productname> officially supports. + <productname>Postgres-XL</productname> officially supports. + </para> + <para> + <productname>Postgres-XL</> is essentially a collection of multiple + <productname>PostgreSQL</> database to provide both read and write + performance scalability. It also provides full-featured transaction + consistency as <productname>PostgreSQL</> provides, at the exception + of SSI which is incomplete. + </para> + <para> + <productname>Postgres-XL</> inherits almost all major features from <productname>PostgreSQL</>. + This document is also based upon <productname>PostgreSQL</> reference manual. </para> <para> @@ -151,6 +162,293 @@ distributed by anyone free of charge for any purpose, be it private, commercial, or academic. </para> + + </sect1> + <sect1 id="intro-whatis-postgres-xl"> + <title> What is <productname>Postgres-XL</productname>?</title> + + <sect2 id="whatis-postgres-xl-in-short"> + <title>In short</title> + + <para> + Postgres-XL is an open source project to provide both write-scalability + massively parallel processing transparently to PostgreSQL. + It is a collection of tightly coupled database + components which can be installed in more than one hardware or + virtual machines. + </para> + + <para> + Write-scalable means Postgres-XL can be configured with as many + database servers as you want and handle many more writes (updating + SQL statements) than a single standalone database server could + otherwise do. You can have more than one database + server which all provide a single database view. Any + database update from any database server is immediately visible to + any other transactions running on different servers. Transparent + means you do not necessarily need to worry about how your data is stored in + more than one database servers internally. + <footnote> + <para> + Of course, you should use the information about how tables are stored + internally when you design the database physically to get most + from Postgres-XL. + </para> + </footnote> + </para> + + <para> + You can configure Postgres-XL to run on more than one machines. It + stores your data in a distributed way, that is, partitioned or + replicated depending on what is chosen for each table. + <footnote> + <para> + To distinguish from PostgreSQL's native partitioning, we refer to this + as "distribution". In distributed database textbooks, this is often + referred to as "horizontal fragment"), and more recently, sharding. + </para> + </footnote> + When you issue queries, Postgres-XL determines where the target + data is stored and dispatches corresponding plans to the servers containing the + target data. + </para> + + <para> + In typical web systems, you can have as many web servers or + application servers to handle your transactions. However, you + cannot do this for a database server in general because all the + changing data have to be visible to all the transactions. Unlike + other database cluster solution, Postgres-XL provides this + capability. You can install as many database servers as you + like. Each database server provides uniform data view to your + applications. Any database update from any server is immediately + visible to applications connecting the database from other + servers. This is on of the most important features of + Postgres-XL. + </para> + + <para> + The other significant feature of Postgres-XL is MPP parallelism. + You can use Postgres-XL to handle workloads for Business Intelligence, + Data Warehousing, or Big Data. In Postgres-XL, a plan is generated once + on a coordinator, and sent down to the individual data nodes. This is + then executed, with the data nodes communicating directly with one another, + where each understands from where it is expected to receive any tuples + that it needs to ship, and where it needs to send to others. + </para> + + </sect2> + + <sect2 id="whatis-xc-goal"> + <title>Postgres-XL's Goal</title> + + <para> + The ultimate goal of Postgres-XL is to provide database + scalability with ACID consistency across all types of database workloads. + That is, Postgres-XL should provide the following features: + <itemizedlist spacing="compact"> + <listitem> + <para> + Postgres-XL should provide multiple servers to accept transactions and statements + from applications, which are known as "Coordinator" processes. + </para> + </listitem> + <listitem> + <para> + Any Coordinator should provide consistent database view to + applications. Any updates from any Coordinator must be visible in + real time manner as if such updates are done in single + PostgreSQL server. + </para> + </listitem> + <listitem> + <para> + Postgres-XL should allow Datanodes to communicate directly with one + another execute queries in an efficient and parallel manner. + </para> + </listitem> + <listitem> + <para> + Tables should be able to be stored in the database designated as + replicated or distributed (known as fragments or + partitions). Replication and distribution should be transparent + to applications; that is, such replicated and distributed tables + are seen as single tables and the location or number of copies of + each record/tuple is managed by Postgres-XL and is not visible + to applications. + </para> + </listitem> + <listitem> + <para> + Postgres-XL provides compatible PostgreSQL API to applications. + </para> + </listitem> + <listitem> + <para> + Postgres-XL should provide single and unified view of + underlying PostgreSQL database servers so that SQL statements + do not depend on how the tables are actually stored. + </para> + </listitem> + </itemizedlist> + </para> + </sect2> + + + <sect2 id="whatis-xc-key-components"> + <title>Postgres-XL Key Components</title> + + <para> + In this section, we will describe the main components of Postgres-XL. + </para> + + <para> + Postgres-XL is composed of three major components: the GTM + (Global Transaction Manager), the Coordinator and the Datanode. Their + features are given in the following sections. + </para> + + <sect3> + <title>GTM (Global Transaction Manager)</title> + + <para> + The GTM is a key component of Postgres-XL to provide consistent + transaction management and tuple visibility control. + </para> + + <para> + As described later in this + manual, <productname>PostgreSQL</productname>'s transaction + management is based upon MVCC (Multi-Version Concurrency Control) + technology. <productname>Postgres-XL</productname> extracts this + technology into separate component as GTM so that + any <productname>Postgres-XL</productname> component's + transaction management is based upon single global status. + Details will be described in <xref linkend="overview">. + </para> + </sect3> + + <sect3> + <title>Coordinator</title> + + <para> + The Coordinator is an interface to the database for applications. It acts like + conventional PostgreSQL backend process, however the Coordinator + does not store any actual data. The actual data is stored by the Datanodes + as described below. The Coordinator receives SQL statements, gets Global + Transaction Id and Global Snapshots as needed, determines which + Datanodes are involved and asks them to execute (a part of) + statement. When issuing statement to Datanodes, it is + associated with GXID and Global Snapshot so that Multi-version Concurrency Control + (MVCC) properties extend cluster-wide. + </para> + </sect3> + + <sect3> + <title>Datanode</title> + + <para> + The Datanode actually stores user data. Tables may be distributed + among Datanodes, or replicated to all the Datanodes. + The Datanode does not have global view of the whole database, it + just takes care of locally stored data. Incoming statement is + examined by the Coordinator as described next, and subplans are + made. These are then transferred to + each Datanode involved together with GXID and Global Snapshot + as needed. The datanode may receive request from various + Coordinators in separate sessions. However, because each the + transaction is identified + uniquely and associated with consistent (global) snapshot, each Datanode + can properly execute in its transaction and snapshot context. + </para> + + </sect3> + + </sect2> + + <sect2 id="whatis-xc-inherits-postgresql"> + <title><productname>Postgres-XL</productname> Inherits From <productname>PostgreSQL</productname></title> + + <para> + <productname>Postgres-XL</productname> is an extension + to <productname>PostgreSQL</productname> and inherits most of its + features. + </para> + + <para> + It is an open-source descendant of + <productname>PostgreSQL</productname> and its + original Berkeley code. It supports a large part of the SQL + standard and offers many modern features: + + <itemizedlist spacing="compact"> + <listitem> + <simpara>complex queries</simpara> + </listitem> + <listitem> + <simpara> + foreign keys + <footnote> + <para> + <productname>Postgres-XL</productname>'s foreign key usage has some restrictions. For details, see <xref linkend="SQL-CREATETABLE">. + </para> + </footnote> + </simpara> + </listitem> + <listitem> + <simpara> + triggers + <footnote> + <para> + <productname>Postgres-XL</productname> does not support trigger in the current version. This may be supported in the future releases. + </para> + </footnote> + </simpara> + </listitem> + <listitem> + <simpara>views</simpara> + </listitem> + <listitem> + <simpara>transactional integrity, at the exception of SSI whose support is incomplete</simpara> + </listitem> + <listitem> + <simpara>multiversion concurrency control</simpara> + </listitem> + </itemizedlist> + + Also, similar to <productname>PostgreSQL</productname>, <productname>Postgres-XL</productname> can be extended by the + user in many ways, for example by adding new + + <itemizedlist spacing="compact"> + <listitem> + <simpara>data types</simpara> + </listitem> + <listitem> + <simpara>functions</simpara> + </listitem> + <listitem> + <simpara>operators</simpara> + </listitem> + <listitem> + <simpara>aggregate functions</simpara> + </listitem> + <listitem> + <simpara>index methods</simpara> + </listitem> + <listitem> + <simpara>procedural languages</simpara> + </listitem> + </itemizedlist> + </para> + + <para> + <productname>Postgres-XL</productname> + can be used, modified, and distributed by anyone free of charge + for any purpose, be it private, commercial, or academic, provided it + adheres to the Mozilla Public License version 2. + </para> + + </sect2> </sect1> &history; diff --git a/doc/src/sgml/keywords.sgml b/doc/src/sgml/keywords.sgml index ea582116ab..a21ce75bb0 100644 --- a/doc/src/sgml/keywords.sgml +++ b/doc/src/sgml/keywords.sgml @@ -371,6 +371,14 @@ <entry></entry> </row> <row> + <entry><token>BARRIER</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>BASE64</token></entry> <entry></entry> <entry>non-reserved</entry> @@ -686,6 +694,14 @@ <entry>non-reserved</entry> </row> <row> + <entry><token>CLEAN</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>CLOB</token></entry> <entry></entry> <entry>reserved</entry> @@ -1414,6 +1430,14 @@ <entry>reserved</entry> </row> <row> + <entry><token>DISTRIBUTE</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>DLNEWCOPY</token></entry> <entry></entry> <entry>reserved</entry> @@ -2051,6 +2075,14 @@ <entry></entry> </row> <row> + <entry><token>HASH</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>HAVING</token></entry> <entry>reserved</entry> <entry>reserved</entry> @@ -2786,6 +2818,14 @@ <entry>reserved</entry> </row> <row> + <entry><token>MODULO</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>MONTH</token></entry> <entry>non-reserved</entry> <entry>reserved</entry> @@ -3766,6 +3806,14 @@ <entry></entry> </row> <row> + <entry><token>REPLICATION</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>REQUIRING</token></entry> <entry></entry> <entry>non-reserved</entry> @@ -3878,6 +3926,14 @@ <entry>reserved</entry> </row> <row> + <entry><token>ROBIN</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>ROLE</token></entry> <entry>non-reserved</entry> <entry>non-reserved</entry> @@ -3899,6 +3955,14 @@ <entry></entry> </row> <row> + <entry><token>ROUND</token></entry> + <entry>non-reserved (XL only)</entry> + <entry></entry> + <entry></entry> + <entry></entry> + <entry></entry> + </row> + <row> <entry><token>ROUTINE</token></entry> <entry></entry> <entry>non-reserved</entry> diff --git a/doc/src/sgml/legal.sgml b/doc/src/sgml/legal.sgml index 6b9f4dbc95..648d7069af 100644 --- a/doc/src/sgml/legal.sgml +++ b/doc/src/sgml/legal.sgml @@ -6,6 +6,18 @@ <year>1996-2015</year> <holder>The PostgreSQL Global Development Group</holder> </copyright> +<copyright> + <year>2009-2012</year> + <holder>Postgres-XC Development Group</holder> +</copyright> +<copyright> + <year>2012-2014</year> + <holder>TransLattice, Inc.</holder> +</copyright> +<copyright> + <year>2015</year> + <holder>2ndQuadrant Ltd</holder> +</copyright> <legalnotice id="legalnotice"> <title>Legal Notice</title> @@ -45,4 +57,384 @@ UPDATES, ENHANCEMENTS, OR MODIFICATIONS. </para> + <para> + <productname>Postgres-XL</productname> is Copyright © 2012-2014 + by TransLattice, Inc. and distributed under the terms of the license + below. + </para> + <screen> +Mozilla Public License Version 2.0 +================================== + +1. Definitions +-------------- + +1.1. "Contributor" + means each individual or legal entity that creates, contributes to + the creation of, or owns Covered Software. + +1.2. "Contributor Version" + means the combination of the Contributions of others (if any) used + by a Contributor and that particular Contributor's Contribution. + +1.3. "Contribution" + means Covered Software of a particular Contributor. + +1.4. "Covered Software" + means Source Code Form to which the initial Contributor has attached + the notice in Exhibit A, the Executable Form of such Source Code + Form, and Modifications of such Source Code Form, in each case + including portions thereof. + +1.5. "Incompatible With Secondary Licenses" + means + + (a) that the initial Contributor has attached the notice described + in Exhibit B to the Covered Software; or + + (b) that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the + terms of a Secondary License. + +1.6. "Executable Form" + means any form of the work other than Source Code Form. + +1.7. "Larger Work" + means a work that combines Covered Software with other material, in + a separate file or files, that is not Covered Software. + +1.8. "License" + means this document. + +1.9. "Licensable" + means having the right to grant, to the maximum extent possible, + whether at the time of the initial grant or subsequently, any and + all of the rights conveyed by this License. + +1.10. "Modifications" + means any of the following: + + (a) any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered + Software; or + + (b) any new file in Source Code Form that contains any Covered + Software. + +1.11. "Patent Claims" of a Contributor + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the + License, by the making, using, selling, offering for sale, having + made, import, or transfer of either its Contributions or its + Contributor Version. + +1.12. "Secondary License" + means either the GNU General Public License, Version 2.0, the GNU + Lesser General Public License, Version 2.1, the GNU Affero General + Public License, Version 3.0, or any later versions of those + licenses. + +1.13. "Source Code Form" + means the form of the work preferred for making modifications. + +1.14. "You" (or "Your") + means an individual or a legal entity exercising rights under this + License. For legal entities, "You" includes any entity that + controls, is controlled by, or is under common control with You. For + purposes of this definition, "control" means (a) the power, direct + or indirect, to cause the direction or management of such entity, + whether by contract or otherwise, or (b) ownership of more than + fifty percent (50%) of the outstanding shares or beneficial + ownership of such entity. + +2. License Grants and Conditions +-------------------------------- + +2.1. Grants + +Each Contributor hereby grants You a world-wide, royalty-free, +non-exclusive license: + +(a) under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and + +(b) under Patent Claims of such Contributor to make, use, sell, offer + for sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +2.2. Effective Date + +The licenses granted in Section 2.1 with respect to any Contribution +become effective for each Contribution on the date the Contributor first +distributes such Contribution. + +2.3. Limitations on Grant Scope + +The licenses granted in this Section 2 are the only rights granted under +this License. No additional rights or licenses will be implied from the +distribution or licensing of Covered Software under this License. +Notwithstanding Section 2.1(b) above, no patent license is granted by a +Contributor: + +(a) for any code that a Contributor has removed from Covered Software; + or + +(b) for infringements caused by: (i) Your and any other third party's + modifications of Covered Software, or (ii) the combination of its + Contributions with other software (except as part of its Contributor + Version); or + +(c) under Patent Claims infringed by Covered Software in the absence of + its Contributions. + +This License does not grant any rights in the trademarks, service marks, +or logos of any Contributor (except as may be necessary to comply with +the notice requirements in Section 3.4). + +2.4. Subsequent Licenses + +No Contributor makes additional grants as a result of Your choice to +distribute the Covered Software under a subsequent version of this +License (see Section 10.2) or under the terms of a Secondary License (if +permitted under the terms of Section 3.3). + +2.5. Representation + +Each Contributor represents that the Contributor believes its +Contributions are its original creation(s) or it has sufficient rights +to grant the rights to its Contributions conveyed by this License. + +2.6. Fair Use + +This License is not intended to limit any rights You have under +applicable copyright doctrines of fair use, fair dealing, or other +equivalents. + +2.7. Conditions + +Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted +in Section 2.1. + +3. Responsibilities +------------------- + +3.1. Distribution of Source Form + +All distribution of Covered Software in Source Code Form, including any +Modifications that You create or to which You contribute, must be under +the terms of this License. You must inform recipients that the Source +Code Form of the Covered Software is governed by the terms of this +License, and how they can obtain a copy of this License. You may not +attempt to alter or restrict the recipients' rights in the Source Code +Form. + +3.2. Distribution of Executable Form + +If You distribute Covered Software in Executable Form then: + +(a) such Covered Software must also be made available in Source Code + Form, as described in Section 3.1, and You must inform recipients of + the Executable Form how they can obtain a copy of such Source Code + Form by reasonable means in a timely manner, at a charge no more + than the cost of distribution to the recipient; and + +(b) You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter + the recipients' rights in the Source Code Form under this License. + +3.3. Distribution of a Larger Work + +You may create and distribute a Larger Work under terms of Your choice, +provided that You also comply with the requirements of this License for +the Covered Software. If the Larger Work is a combination of Covered +Software with a work governed by one or more Secondary Licenses, and the +Covered Software is not Incompatible With Secondary Licenses, this +License permits You to additionally distribute such Covered Software +under the terms of such Secondary License(s), so that the recipient of +the Larger Work may, at their option, further distribute the Covered +Software under the terms of either this License or such Secondary +License(s). + +3.4. Notices + +You may not remove or alter the substance of any license notices +(including copyright notices, patent notices, disclaimers of warranty, +or limitations of liability) contained within the Source Code Form of +the Covered Software, except that You may alter any license notices to +the extent required to remedy known factual inaccuracies. + +3.5. Application of Additional Terms + +You may choose to offer, and to charge a fee for, warranty, support, +indemnity or liability obligations to one or more recipients of Covered +Software. However, You may do so only on Your own behalf, and not on +behalf of any Contributor. You must make it absolutely clear that any +such warranty, support, indemnity, or liability obligation is offered by +You alone, and You hereby agree to indemnify every Contributor for any +liability incurred by such Contributor as a result of warranty, support, +indemnity or liability terms You offer. You may include additional +disclaimers of warranty and limitations of liability specific to any +jurisdiction. + +4. Inability to Comply Due to Statute or Regulation +--------------------------------------------------- + +If it is impossible for You to comply with any of the terms of this +License with respect to some or all of the Covered Software due to +statute, judicial order, or regulation then You must: (a) comply with +the terms of this License to the maximum extent possible; and (b) +describe the limitations and the code they affect. Such description must +be placed in a text file included with all distributions of the Covered +Software under this License. Except to the extent prohibited by statute +or regulation, such description must be sufficiently detailed for a +recipient of ordinary skill to be able to understand it. + +5. Termination +-------------- + +5.1. The rights granted under this License will terminate automatically +if You fail to comply with any of its terms. However, if You become +compliant, then the rights granted under this License from a particular +Contributor are reinstated (a) provisionally, unless and until such +Contributor explicitly and finally terminates Your grants, and (b) on an +ongoing basis, if such Contributor fails to notify You of the +non-compliance by some reasonable means prior to 60 days after You have +come back into compliance. Moreover, Your grants from a particular +Contributor are reinstated on an ongoing basis if such Contributor +notifies You of the non-compliance by some reasonable means, this is the +first time You have received notice of non-compliance with this License +from such Contributor, and You become compliant prior to 30 days after +Your receipt of the notice. + +5.2. If You initiate litigation against any entity by asserting a patent +infringement claim (excluding declaratory judgment actions, +counter-claims, and cross-claims) alleging that a Contributor Version +directly or indirectly infringes any patent, then the rights granted to +You by any and all Contributors for the Covered Software under Section +2.1 of this License shall terminate. + +5.3. In the event of termination under Sections 5.1 or 5.2 above, all +end user license agreements (excluding distributors and resellers) which +have been validly granted by You or Your distributors under this License +prior to termination shall survive termination. + +************************************************************************ +* * +* 6. Disclaimer of Warranty * +* ------------------------- * +* * +* Covered Software is provided under this License on an "as is" * +* basis, without warranty of any kind, either expressed, implied, or * +* statutory, including, without limitation, warranties that the * +* Covered Software is free of defects, merchantable, fit for a * +* particular purpose or non-infringing. The entire risk as to the * +* quality and performance of the Covered Software is with You. * +* Should any Covered Software prove defective in any respect, You * +* (not any Contributor) assume the cost of any necessary servicing, * +* repair, or correction. This disclaimer of warranty constitutes an * +* essential part of this License. No use of any Covered Software is * +* authorized under this License except under this disclaimer. * +* * +************************************************************************ + +************************************************************************ +* * +* 7. Limitation of Liability * +* -------------------------- * +* * +* Under no circumstances and under no legal theory, whether tort * +* (including negligence), contract, or otherwise, shall any * +* Contributor, or anyone who distributes Covered Software as * +* permitted above, be liable to You for any direct, indirect, * +* special, incidental, or consequential damages of any character * +* including, without limitation, damages for lost profits, loss of * +* goodwill, work stoppage, computer failure or malfunction, or any * +* and all other commercial damages or losses, even if such party * +* shall have been informed of the possibility of such damages. This * +* limitation of liability shall not apply to liability for death or * +* personal injury resulting from such party's negligence to the * +* extent applicable law prohibits such limitation. Some * +* jurisdictions do not allow the exclusion or limitation of * +* incidental or consequential damages, so this exclusion and * +* limitation may not apply to You. * +* * +************************************************************************ + +8. Litigation +------------- + +Any litigation relating to this License may be brought only in the +courts of a jurisdiction where the defendant maintains its principal +place of business and such litigation shall be governed by laws of that +jurisdiction, without reference to its conflict-of-law provisions. +Nothing in this Section shall prevent a party's ability to bring +cross-claims or counter-claims. + +9. Miscellaneous +---------------- + +This License represents the complete agreement concerning the subject +matter hereof. If any provision of this License is held to be +unenforceable, such provision shall be reformed only to the extent +necessary to make it enforceable. Any law or regulation which provides +that the language of a contract shall be construed against the drafter +shall not be used to construe this License against a Contributor. + +10. Versions of the License +--------------------------- + +10.1. New Versions + +Mozilla Foundation is the license steward. Except as provided in Section +10.3, no one other than the license steward has the right to modify or +publish new versions of this License. Each version will be given a +distinguishing version number. + +10.2. Effect of New Versions + +You may distribute the Covered Software under the terms of the version +of the License under which You originally received the Covered Software, +or under the terms of any subsequent version published by the license +steward. + +10.3. Modified Versions + +If you create software not governed by this License, and you want to +create a new license for such software, you may create and use a +modified version of this License if you rename the license and remove +any references to the name of the license steward (except to note that +such modified license differs from this License). + +10.4. Distributing Source Code Form that is Incompatible With Secondary +Licenses + +If You choose to distribute Source Code Form that is Incompatible With +Secondary Licenses under the terms of this version of the License, the +notice described in Exhibit B of this License must be attached. + +Exhibit A - Source Code Form License Notice +------------------------------------------- + + This Source Code Form is subject to the terms of the Mozilla Public + License, v. 2.0. If a copy of the MPL was not distributed with this + file, You can obtain one at http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular +file, then You may include the notice in a location (such as a LICENSE +file in a relevant directory) where a recipient would be likely to look +for such a notice. + +You may add additional accurate notices of copyright ownership. + +Exhibit B - "Incompatible With Secondary Licenses" Notice +--------------------------------------------------------- + + This Source Code Form is "Incompatible With Secondary Licenses", as + defined by the Mozilla Public License, v. 2.0. +</screen> </legalnotice> diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index de6b3ad86b..f369508265 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -4862,6 +4862,12 @@ typedef struct </indexterm> <para> + <productname>Postgres-XL</> does not + support <command>NOTIFY</>, <command>LISTEN</> + and <command>UNLISTEN</> commands yet. + </para> + + <para> <productname>PostgreSQL</productname> offers asynchronous notification via the <command>LISTEN</command> and <command>NOTIFY</command> commands. A client session registers its interest in a particular diff --git a/doc/src/sgml/lo.sgml b/doc/src/sgml/lo.sgml index cd4ed6030b..ace672c88e 100644 --- a/doc/src/sgml/lo.sgml +++ b/doc/src/sgml/lo.sgml @@ -8,6 +8,12 @@ </indexterm> <para> + <productname>Postgres-XL</> doesn't support the + module <filename>lo</> because it depends upon + consistent <literal>OID</> and triggers. + </para> + + <para> The <filename>lo</> module provides support for managing Large Objects (also called LOs or BLOBs). This includes a data type <type>lo</> and a trigger <function>lo_manage</>. diff --git a/doc/src/sgml/lobj.sgml b/doc/src/sgml/lobj.sgml index d6fb8c5ac4..8042e9c54a 100644 --- a/doc/src/sgml/lobj.sgml +++ b/doc/src/sgml/lobj.sgml @@ -7,6 +7,12 @@ <indexterm><primary>BLOB</><see>large object</></> <para> + Large objects are not supported by <productname>Postgres-XL</>. + <productname>Postgres-XL</> does not provide consistent means + to handle the large object as OID is inconsistent among cluster nodes. + </para> + + <para> <productname>PostgreSQL</productname> has a <firstterm>large object</> facility, which provides stream-style access to user data that is stored in a special large-object structure. Streaming access is useful diff --git a/doc/src/sgml/ltree.sgml b/doc/src/sgml/ltree.sgml index 8a7a36390d..9038f84ddc 100644 --- a/doc/src/sgml/ltree.sgml +++ b/doc/src/sgml/ltree.sgml @@ -13,6 +13,11 @@ Extensive facilities for searching through label trees are provided. </para> + <para> + Please note that a column of the type <type>ltree</> cannot be a + distribution column. + </para> + <sect2> <title>Definitions</title> diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml index e34426ff18..4659896d66 100644 --- a/doc/src/sgml/maintenance.sgml +++ b/doc/src/sgml/maintenance.sgml @@ -12,6 +12,11 @@ </indexterm> <para> + This chapter describes database maintenance tasks + of individual Coordinators and Datanodes. + </para> + + <para> <productname>PostgreSQL</>, like any database software, requires that certain tasks be performed regularly to achieve optimum performance. The tasks discussed here are <emphasis>required</emphasis>, but they @@ -67,6 +72,11 @@ </indexterm> <para> + Please note that this section describes the task of individual + Coordinator and Datanode. It should be done for each of them. + </para> + + <para> <productname>PostgreSQL</productname> databases require periodic maintenance known as <firstterm>vacuuming</>. For many installations, it is sufficient to let vacuuming be performed by the <firstterm>autovacuum @@ -85,6 +95,11 @@ <sect2 id="vacuum-basics"> <title>Vacuuming Basics</title> + <para> + Please note that this section describes the task of individual + Coordinator and Datanode. It should be done for each of them. + </para> + <para> <productname>PostgreSQL</productname>'s <xref linkend="sql-vacuum"> command has to @@ -151,6 +166,11 @@ </indexterm> <para> + Please note that this section describes the task of individual + Coordinator and Datanode. It should be done for each of them. + </para> + + <para> In <productname>PostgreSQL</productname>, an <command>UPDATE</> or <command>DELETE</> of a row does not immediately remove the old version of the row. @@ -265,6 +285,11 @@ </indexterm> <para> + Please note that this section describes the task of individual + Coordinator and Datanode. It should be done for each of them. + </para> + + <para> The <productname>PostgreSQL</productname> query planner relies on statistical information about the contents of tables in order to generate good plans for queries. These statistics are gathered by @@ -386,6 +411,11 @@ </indexterm> <para> + Please note that this section describes the task of individual + Coordinator and Datanode. It should be done for each of them. + </para> + + <para> <productname>PostgreSQL</productname>'s MVCC transaction semantics depend on being able to compare transaction ID (<acronym>XID</>) numbers: a row version with an insertion XID greater than the current @@ -402,6 +432,12 @@ </para> <para> + Please note that <productname>Postgres-XL</>'s transaction is + called global transaction ID (<acronym>GXID</>) and is supplied by + global transaction manager (<acronym>GTM</>). + </para> + + <para> The reason that periodic vacuuming solves the problem is that <command>VACUUM</> will mark rows as <emphasis>frozen</>, indicating that they were inserted by a transaction which committed sufficiently far in @@ -675,6 +711,12 @@ HINT: Stop the postmaster and vacuum that database in single-user mode. <primary>autovacuum</primary> <secondary>general information</secondary> </indexterm> + + <para> + Please note that this section describes the task of individual + Coordinator and Datanode. It should be done for each of them. + </para> + <para> <productname>PostgreSQL</productname> has an optional but highly recommended feature called <firstterm>autovacuum</firstterm>, diff --git a/doc/src/sgml/manage-ag.sgml b/doc/src/sgml/manage-ag.sgml index 78ec509a8b..e229a0171f 100644 --- a/doc/src/sgml/manage-ag.sgml +++ b/doc/src/sgml/manage-ag.sgml @@ -276,6 +276,16 @@ createdb -T template0 <replaceable>dbname</> users and applications to connect to. It is simply a copy of <literal>template1</> and can be dropped and recreated if necessary. </para> + + <para> + <emphasis>For Postgres-XL only.</> + </para> + <para> + In <productname>Postgres-XL</>, template databases are held in + each Coordinator and Datanode. They are locally copied when new + database is created. + </para> + </note> </sect1> diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml index 385691e21e..4f65e98972 100644 --- a/doc/src/sgml/mvcc.sgml +++ b/doc/src/sgml/mvcc.sgml @@ -17,6 +17,15 @@ in this chapter. </para> + <para> + <productname>Postgres-XL</> inherited concurrency control + from <productname>PostgreSQL</> and extended it globally to all of the + Coordinators and Datanodes involved. Regardless of what + Coordinator is connected to, all of the transactions + in the <productname>Postgres-XL</> database cluster behaves in + a consistent way as if they are running in single database. + </para> + <sect1 id="mvcc-intro"> <title>Introduction</title> @@ -1414,6 +1423,15 @@ UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 22222; applications to hold transactions open for long periods of time (e.g., while waiting for user input). </para> + + <para> + Please note that <productname>Postgres-XL</> does not detect + deadlocks which are spread among multiple Coordinators and/or + Datanodes, known as global deadlocks. + Until this is implemented in + <productname>Postgres-XL</>, global deadlocks can be + detected based on statement_timeout. + </para> </sect2> <sect2 id="advisory-locks"> @@ -1489,6 +1507,13 @@ UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 22222; </para> <para> + Please note that <productname>Postgres-XL</>'s advisory lock is + local to each Coordinator or Datanode. If you wish to acquire + advisory locks on different Coordinator, you should do it manually + using <type>EXECUTE DIRECT</> statement. + </para> + + <para> In certain cases using advisory locking methods, especially in queries involving explicit ordering and <literal>LIMIT</> clauses, care must be taken to control the locks acquired because of the order in which SQL @@ -1740,4 +1765,76 @@ SELECT pg_advisory_lock(q.id) FROM indexes should be used instead. </para> </sect1> + <sect1 id="global-transaction-management"> + <title><productname>Postgres-XL</productname>'s Global Transaction Management</title> + + <indexterm> + <primary>Global Transaction Management</primary> + </indexterm> + + + <para> + The above sections + described <productname>Postgres-XL</productname>'s concurrency + control and MVCC common to <productname>PostgreSQL</productname>. + This section describes how <productname>Postgres-XL</productname> + implements global concurrency control and MVCC among multiple + Coordinators and Datanodes. + </para> + + <para> + In conventional replication clusters, you can run read + transactions concurrently in multiple standby, or slave + servers. Replication servers provide read scalability. However, + you cannot issue write transactions to standby servers because + they don't have means to propagate changes safely. They cannot + maintain consistent view of database to applications for write + operations, unless you issue write transactions to single master + server. + </para> + + <para> + Postgres-XL is different. + </para> + + <para> + As described + in <xref linkend="intro-whatis">, <productname>Postgres-XL</productname> + is composed of a GTM (Global Transaction Manager), Coordinators and + Datanodes. + </para> + + <para> + In <productname>Postgres-XL</productname>, any Coordinator can + accept any transaction, regardless whether it is read only or + read/write. Transaction integrity is enforced by GTM (Global + Transaction Manager). Because we have multiple Coordinators, each + of them can handle incoming transactions and statements in + parallel. + </para> + + <para> + Analyzed statements are converted into internal plans, which + include SQL statements targeted to Datanodes. Plans are sent on to + each target Datanode, handled, and the result is sent back to + the originating Coordinator where all the results from target + Datanodes will be combined into the result to be sent back to the + application. + </para> + + <para> + Each table can be distributed or replicated as described + in <xref linkend="intro-whatis">. If you design each table's + distribution carefully, most of the statements may end up targeting + just one Datanode, which is most effecient. In this way, Coordinators and Datanodes + runs transactions in parallel which scales out both read and write + operations. + </para> + + <para> + More detailed internals + about <productname>Postgres-XL</productname>'s transaction + management will be found in <xref linkend="overview">. + </para> + </sect1> </chapter> diff --git a/doc/src/sgml/oid2name.sgml b/doc/src/sgml/oid2name.sgml index 97b170a23f..8843ea5c4f 100644 --- a/doc/src/sgml/oid2name.sgml +++ b/doc/src/sgml/oid2name.sgml @@ -44,6 +44,12 @@ </note> <para> + Please note that you can issue this command to each Datanode or + Coordinator. The result is the information local to Datanode or + Coordinator specified. + </para> + + <para> <application>oid2name</application> connects to a target database and extracts OID, filenode, and/or table name information. You can also have it show database OIDs or tablespace OIDs. diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml index 313cbaea30..45b2762ef4 100644 --- a/doc/src/sgml/pageinspect.sgml +++ b/doc/src/sgml/pageinspect.sgml @@ -13,6 +13,12 @@ debugging purposes. All of these functions may be used only by superusers. </para> + <para> + Functions of this module returns information about connecting + Coordinators locally. To get information of a Datanode, you can + use EXECUTE DIRECT from a Coordinator. + </para> + <sect2> <title>Functions</title> diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml index 7bcbfa7611..e947dca296 100644 --- a/doc/src/sgml/perform.sgml +++ b/doc/src/sgml/perform.sgml @@ -914,6 +914,19 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000 </para> <para> + <productname>Postgres-XL</productname> pulls up statistics + from the Datanodes and stores it in each of the Coordinator's catalog + table to help make query planning decisions. + </para> + + <para> + Traditional PostgreSQL costing applies, but <productname>Postgres-XL</productname> + also tries to take into consideration the cost of shipping rows around the + network for internode joins, which is an expensive operation. + Below general PostgreSQL handling is described. + </para> + + <para> One component of the statistics is the total number of entries in each table and index, as well as the number of disk blocks occupied by each table and index. This information is kept in the table @@ -1084,6 +1097,17 @@ SELECT * FROM a, b, c WHERE a.id = b.id AND b.ref = c.id; </para> <para> + <productname>Postgres-XL</> stores table data in a distributed or + replicated fashion. To leverage this, the planner tries to find + the best way to use as much of Datanode power as possible. If the + equi-join is done by distribution columns and they share the + distribution method (hash/modulo), the Coordinator can tell + the Datanode to perform join. + If not, it may tell Datanodes to ship rows to other Datanodes + and expect rows from other Datanodes to be shipped to it. + </para> + + <para> When a query only involves two or three tables, there aren't many join orders to worry about. But the number of possible join orders grows exponentially as the number of tables expands. Beyond ten or so input @@ -1358,6 +1382,11 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; It won't do much for <command>COPY</> itself, so this advice is only useful when you are using one or both of the above techniques. </para> + + <para> + In <productname>Postgres-XL</>, only the distribution column can have foreign key + constraint. + </para> </sect2> <sect2 id="populate-max-wal-size"> @@ -1376,6 +1405,13 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; data loads, the number of checkpoints that are required can be reduced. </para> + + <para> + Please note that you should tune the configuration variables in + all the nodes involved. Keep in mind that the Datanodes + are more important and will need more resources than Coordinators. + by <acronym>DDL</>s. + </para> </sect2> <sect2 id="populate-pitr"> @@ -1450,6 +1486,13 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; <xref linkend="vacuum-for-statistics"> and <xref linkend="autovacuum"> for more information. </para> + + <para> + In <productname>Postgres-XL</>, a manual <command>VACUUM</> will be + pushed down to all the Datanodes as well. + Depending on your workload, if you have a highly loaded cluster, + you may wish to disable autovacuum and vacuum at specific times. + </para> </sect2> <sect2 id="populate-pg-dump"> diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml index f379be225f..074638e891 100644 --- a/doc/src/sgml/pgbuffercache.sgml +++ b/doc/src/sgml/pgbuffercache.sgml @@ -28,6 +28,12 @@ are security issues lurking. </para> + <para> + <filename>pg_buffercache</filename> returns information local to the + connecting Coordinator. To inquire information local to other node, + use <command>EXECUTE DIRECT</command>. + </para> + <sect2> <title>The <structname>pg_buffercache</structname> View</title> diff --git a/doc/src/sgml/pgfreespacemap.sgml b/doc/src/sgml/pgfreespacemap.sgml index f2f99d571e..52b8f9459d 100644 --- a/doc/src/sgml/pgfreespacemap.sgml +++ b/doc/src/sgml/pgfreespacemap.sgml @@ -20,6 +20,13 @@ there are security issues lurking. </para> + <para> + Functions of this module return information from the + Coordinator that the session is currently connect to. + To get information about a Datanode, you can + use <command>EXECUTE DIRECT</command>. + </para> + <sect2> <title>Functions</title> diff --git a/doc/src/sgml/pgrowlocks.sgml b/doc/src/sgml/pgrowlocks.sgml index d73511579c..4fae2fad98 100644 --- a/doc/src/sgml/pgrowlocks.sgml +++ b/doc/src/sgml/pgrowlocks.sgml @@ -12,6 +12,13 @@ locking information for a specified table. </para> + <para> + Functions of this module return information from the + Coordinator that the session is currently connect to. + To get information about a Datanode, you can + use <command>EXECUTE DIRECT</command>. + </para> + <sect2> <title>Overview</title> diff --git a/doc/src/sgml/pgstatstatements.sgml b/doc/src/sgml/pgstatstatements.sgml index 4d7a6e68ea..78b348e24e 100644 --- a/doc/src/sgml/pgstatstatements.sgml +++ b/doc/src/sgml/pgstatstatements.sgml @@ -8,6 +8,11 @@ </indexterm> <para> + The module <filename>pg_stat_statments</filename> does not work + correctly with <productname>Postgres-XL</productname>. + </para> + + <para> The <filename>pg_stat_statements</filename> module provides a means for tracking execution statistics of all SQL statements executed by a server. </para> diff --git a/doc/src/sgml/pgstattuple.sgml b/doc/src/sgml/pgstattuple.sgml index 40a9669c90..a423baceb9 100644 --- a/doc/src/sgml/pgstattuple.sgml +++ b/doc/src/sgml/pgstattuple.sgml @@ -12,6 +12,13 @@ obtain tuple-level statistics. </para> + <para> + Functions of this module return information from the + Coordinator that the session is currently connect to. + To get information about a Datanode, you can + use <command>EXECUTE DIRECT</command>. + </para> + <sect2> <title>Functions</title> diff --git a/doc/src/sgml/pgxc_ctl-ref.sgml b/doc/src/sgml/pgxc_ctl-ref.sgml new file mode 100644 index 0000000000..b47905ff56 --- /dev/null +++ b/doc/src/sgml/pgxc_ctl-ref.sgml @@ -0,0 +1,1905 @@ + +<sect1 id="pgxc-ctl" xreflabel="pgxc-ctl"> + +<title>pgxc_ctl</title> + +<indexterm zone="pgxc-ctl"> + <primary>pgxc_ctl</primary> +</indexterm> + + <sect2> + <title>Description</title> + + <para> + The manual configuration of the individual components of a Postgres-XL cluster + can be cumbersome and error prone. + The Postgres-XL Cluster Control utility, <application>pgxc_ctl</application>, simplifies this by allowing for the configuration, + initialization, starting, stoping, monitoring and failover of Postgres-XL + components. + This section describes how to use + <application>pgxc_ctl</application>. + </para> + </sect2> + + <sect2 id="R1-APP-PGXC-CTL-1"> + <title>Building and installing pgxc_ctl</title> + + <para> + You should build pgxc_ctl using your Postgres-XL build environment. + <application>pgxc_ctl</application> source code comes with Postgres-XL source code tarball. + The latest version of the source code will be available at its home repository, +<programlisting> +http:// pgxc_ctl +</programlisting> + If you would like to use the latest version from pgxc_ctl home repository, + get the source code tarball and expand it in the source's contrib + directory of your Postgres-XL build environment. + If you are using pgxc_ctl in Postgres-XL tarball, you don't have to do this. + </para> + + <para> + Before building pgxc_ctl, you should build Postgres-XL binary, like +<programlisting> +$ cd <replaceable>your Postgres-XL build directory</replaceable> +$ ./configure <replaceable>your configuration option</replaceable> +$ make +</programlisting> + + Please note that you dont'have to install the + <application>Postgres-XL</application> binary to build + <application>pgxc_ctl</application>. + Also please note that <application>Postgres-XL</application> top + level make and make install command does not take care of pgxc_ctl. + You should build and install it separately. + </para> + + <para> + Then you can build pgxc_ctl as follows: +<programlisting> +$ cd contrib/pgxc_ctl +$ make +$ make install +</programlisting> + The <application>pgxc_ctl</application> binary will be installed in the + same directory as <application>Postgres-XL</application> binaries. + </para> + + <para> + <application>Postgres-XL</application> consists of many components + (or called "nodes") running in various physical or virtual + machines. + Because pgxc_ctl relies on ssh connections between the machines where + <application>pgxc_ctl</application> and other nodes are running, + you should setup ssh-agent authentication to avoid typing password + each time pgcx_ctl issues ssh. + </para> + + </sect2> + + <sect2> + <title>pgxc_ctl home directory</title> + + <para> + <application>pgxc_ctl</application> uses its own work directory, + where it stores configuration files and logs, as well as other + resources. + The default value is <literal>$HOME/pgxc_ctl</literal> and you can + specify this by the <literal>--home</literal> option. + </para> + + <para> + The <application>pgxc_ctl</application> home directory may be referred + to as <literal>$PGXC_CTL_HOME</literal> through this manual. + </para> + + <para> + You do not have to create this directory manually. + <application>pgxc_ctl</application> will create the directory when + it is invoked for the first time. + Please note that you need appropriate privilege to create + <literal>$PGXC_CTL_HOME</literal>. + You can create this manually, of course. + + For details, please refer to later sections. + </para> + + </sect2> + + <sect2> + <title>pgxc_ctl configuration file</title> + + <para> + pgxc_ctl uses configuration file. + The default name and the location is + <literal>$PGXC_CTL_HOME/pgxc_ctl.conf</literal>. + When you change Postgres-XL cluster configuration using pgxc_ctl + commands, this file will be updated. + Depending upon your configuration, + <application>pgxc_ctl</application> will back up this file + according to your configuration. + </para> + + <para> + <application>pgxc_ctl</application> provides the command "prepare" to + setup the prototype of this file. + For details, please refer to command syntax of + <application>pgxc_ctl</application>. + </para> + + </sect2> + + <sect2> + <title>pgxc_ctl initialization file</title> + + <para> + You can specify your preferred parameters of pgxc_ctl behavior. + You can specify parameters in <literal>/etc/pgxc_ctl</literal> + and/or <literal>$HOME/.pgxc_ctl</literal> file. + Setups in <literal>$HOME/.pgxc_ctl</literal> have higher priority + so you can specify system-wide setups at + <literal>/etc/pgxc_ctl</literal> and then your personal preferences + in <literal>$HOME/.pgxc_ctl</literal>. + </para> + + <para> + The format of this file will be described in a later section. + </para> + + </sect2> + + <sect2> + <title>Running pgxc_ctl for the first time</title> + + <para> + Unless you build $PGXC_CTL_HOME and the configuration file from the + scratch, you should run pgxc_ctl to build your $PGXC_CTL_HOME and + get a prototype of configuration file. + From your shell prompt, simply type + <application>pgxc_ctl</application>. + You will have the following prompt: +<programlisting> +$ pgxc_ctl +PGXC$ +</programlisting> + You will get the default prompt, which you can modify at any time + through initialization files. + </para> + + <para> + Try to type <literal>pwd</literal>. + You will see what your <literal>$PGXC_CTL_HOME</literal> is. +<programlisting> +$ pgxc_ctl +PGXC$ pwd +/home/postgres-xl/pgxc_ctl +PGXC$ +</programlisting> + If you specify <option>--home</option> option with another + directory, <application>pgxc_ctl</application> will start at this + directory, after building it if needed. +<programlisting> +$ pgxc_ctl --home /home/postgres-xl/my_pgxc_ctl +PGXC$ pwd +/home/postgres-xl/my_pgxc_ctl +PGXC$ +</programlisting> + You can specify your pgxc_ctl_home as environment variable + <literal>PGXC_CTL_HOME</literal>, or you can specify this as + variable <literal>pgxc_ctl_home</literal> in your initialization + files. + </para> + + <para> + The command line option has the highest priority, then the environment, + <literal>$HOME/.pgxc_ctl</literal> and + <literal>/etc/pgxc_ctl</literal>. + </para> + + <para> + Type prepare or prepare config to get a configuration template file + <filename>pgxc_ctl.conf</filename> at + <literal>$PGXC_CTL_HOME</literal>. + You may add file name as an option to get configuration template in + your favorite file. For example: +<programlisting> +PGXC$ prepare +PGXC$ +</programlisting> +or +<programlisting> +PGXC$ prepare config my_pgxc.conf +PGXC$ +</programlisting> + +You may also generate a template configuration file suitable for testing +<application>Postgres-XL</application> on the localhost. Use option +<literal>minimal</literal> to generate a such a template configuration file. +<programlisting> +PGXC$ prepare config minimal +PGXC$ prepare config minimal my_minimal_pgxc.conf +</programlisting> + + A more detailed syntax of the command will be described in a later section. + </para> + + </sect2> + + <sect2> + <title>Make your configuration</title> + + <para> + Please take a look at the template of the configuration file you + created in the previous section. + This file is actually a bash script file to setup various bash + script variables which are passed to pgxc_ctl next time you run + it. + </para> + + <para> + The <application>Postgres-XL</application> configuration needs to + specify same or similar values to each node configuration, for + example, work directory, port, etc. + To avoid trivial errors, you can specify the same value as defaults + for variables and refer to them later in each variable setups. + </para> + + <para> + For example, a part of your template may look like this: +<programlisting> +#---- Shortcuts ------ +gtmProxyDir=$HOME/pgxc/nodes/gtm_pxy + +#---- Overall ------- +gtmProxy=y # Specify y if you configure at least one GTM + # proxy. + # You may not configure gtm proxies only when + # you dont' configure GTM slaves. + # If you specify this value not to y, the + # following parameters will be set to default + # empty values. + # If we find there're no valid Proxy server + # names (means, every servers are specified + # as none), then gtmProxy value will be set to + # "n" and all the entries will be set to empty + # values. +gtmProxyNames=(gtm_pxy1 gtm_pxy2 gtm_pxy3 gtm_pxy4) # No used if it is not configured +gtmProxyServers=(node06 node07 node08 node09) # Specify none if you dont' configure it. +gtmProxyPorts=(20001 20001 20001 20001) # Not used if it is not configured. +gtmProxyDirs=($gtmProxyDir $gtmProxyDir $gtmProxyDir $gtmProxyDir) # Not used if it is not configured. +</programlisting> + This section specifies the gtm proxy configuration. + We have four <literal>gtm proxies</literal> in each of the server. + They share working directory path and is specified as a shortcut + which is referred to later. + </para> + + <para> + You can do all these in any part of the configuration file. + </para> + + <para> + Please note that the working directory of this script is + <filename>$PGXC_CTL_HOME</filename>, unless you change it explicitly + in this configuration file. + </para> + + </sect2> + + <sect2> + <title>pgxc_ctl invocation options</title> + + <para> + When you invoke pgxc_cmd from your shell, + <application>pgxc_ctl</application> accepts several options to + control its behavior. + <application>pgxc_ctl</application> command format is as follows: +<programlisting> +pgxc [options ... ] [pgxc_command] +</programlisting> + </para> + + <para> + Options are as follows: + </para> + + <variablelist> + <varlistentry> + <term><option>-c <replaceable class="parameter">configuration_file</replaceable></></term> + <term><option>--configuration <replaceable class="parameter">configuration_file</replaceable></></term> + <listitem> + <para> + Specifies configuration file. + The default is <filename>pgxc_ctl_conf</filename>, or the value + of <literal>configFile</literal> option + found in the initialization file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>--home <replaceable class="parameter">home_directory</replaceable></></term> + <listitem> + <para> + Specifies <filename>$PGXC_CTL_HOME</filename> directory. + You can specify this as <literal>pgxc_ctl_home</literal> + variable in the initialization file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-i <replaceable class="parameter">file</replaceable></></term> + <term><option>--infile <replaceable class="parameter">file</replaceable></></term> + <listitem> + <para> + Specifies where to read pgxc_ctl commands. + There's no corresponding variable in the initialization file. + Default is the standard input. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-L <replaceable class="parameter">logfile</replaceable></></term> + <term><option>--logfile <replaceable class="parameter">logfile</replaceable></></term> + <listitem> + <para> + Specifies where to write the log. + The path is relative to <filename>$PGXC_CTL_HOME</filename> or + the value of log directory specified as <option>-l</option> + option or <literal>logDir</literal> variable in the + initialization file, if specified. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-l <replaceable class="parameter">logdir</replaceable></option></term> + <term><option>--logdir <replaceable class="parameter">logdir</replaceable></></term> + <listitem> + <para> + Specifies the directory of the log file. + Default is <filename>$PGXC_HOME/pgxc_log/</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-o <replaceable class="parameter">outfile</replaceable></option></term> + <term><option>--out-file <replaceable class="parameter">outfile</replaceable></></term> + <listitem> + <para> + Specifies where to write <application>pgxc_ctl</application> output. + Default is the standard output. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>--silent</option></term> + <listitem> + <para> + Specifies to run <application>pgxc_ctl</application> without + printing many messages. + This value can also be set as variable + <literal>verbose</literal> in the initialization file. + You can setup level of messages <literal>logMessage</literal> + and <literal>printMessage</literal> variables in the + initialization file as well. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-V</option></term> + <term><option>--version</option></term> + <listitem> + <para> + Prints <application>pgxc_ctl</application> version and exits. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v</option></term> + <term><option>--verbose</option></term> + <listitem> + <para> + Specifies to run pgxc_ctl to print as many messages as + possible. + </para> + </listitem> + </varlistentry> + + </variablelist> + </sect2> + + + <sect2> + <title>pgxc_ctl initialization file</title> + + <para> + As described in previous sections, pgxc_ctl behavior, as specified + in the command line options, can be specified in advance in the + initialization file <filename>/etc/pgxc_ctl</filename> or + <filename>$HOME/.pgxc_ctl</filename>. + </para> + <para> + The syntax is as follows: +<programlisting> +name value [ value ... ] # comment +</programlisting> + </para> + <para> + Blank lines or lines beginning with '#' are simply ignored. If + you'd like to include space or tab in the variable name, enclose + the name with <literal>'...'</literal> or <literal>"..."</literal>. + </para> + <para> + Please note that this file is not a bash script. + </para> + <para> + List of the name and their value is as follows: + </para> + + <variablelist> + + <varlistentry> + <term><option>configFile <replaceable>filename</replaceable></option></term> + <listitem> + <para> + Specify the configuration file name. + Default is <filename>pgxc_ctl.conf</filename>. + This option can be overridden by <option>-c</option> command + line option. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>localTmpDir <replaceable>dirname</replaceable></option></term> + <listitem> + <para> + Specify remote temporary directory, default is <filename>/tmp</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>logDir <replaceable>dirname</replaceable></option></term> + <listitem> + <para> + Specifies the directory to write log. Can be overridden by + <option>-l</option> command line option. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>logFile <replaceable>filename</replaceable></option></term> + <listitem> + <para> + Specifies the log file name, which is relative to + <filename>$PGXC_CTL_HOME/pgxc_log</filename> or value of + <literal>logDir</literal> variable. + Can be overridden by <option>-L,</option> command line option. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>logMessage <replaceable>msglevel</replaceable></option></term> + <listitem> + <para> + Specifies the message level to print to the terminal or output + file. + Valid value is <literal>MANDATORY</literal>, + <literal>PANIC</literal>, <literal>ERROR</literal>, + <literal>WARNING</literal>, <literal>NOTICE</literal>, + <literal>NOTICE2</literal>, <literal>INFO</literal>, + <literal>DEBUG1</literal>, <literal>DEBUG2</literal>, or + <literal>DEBUG3</literal>. Default is + <literal>NOTICE</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>pgxc_ctl_home <replaceable>dirname</replaceable></option></term> + <listitem> + <para> + Specifies <filename>$PGXC_CTL_HOME</filename>. + Default is <filename>$HOME/pgxc_ctl</filename> or environment + variable <literal>$PGXC_CTL_HOME</literal>. + Can be overriden by <option>--home</option> command line option. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>printMessage <replaceable>msglevel</replaceable></option></term> + <listitem> + <para> + Specifies the message level to print to the log file. + Valid value is <literal>MANDATORY</literal>, + <literal>PANIC</literal>, <literal>ERROR</literal>, + <literal>WARNING</literal>, <literal>NOTICE</literal>, + <literal>NOTICE2</literal>, <literal>INFO</literal>, + <literal>DEBUG1</literal>, <literal>DEBUG2</literal>, or + <literal>DEBUG3</literal>. Default is + <literal>NOTICE</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>tmpDir <replaceable>dirname</replaceable></option></term> + <listitem> + <para> + Specify local temporary directory, default is <filename>/tmp</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>verbose <replaceable>value</replaceable></option></term> + <listitem> + <para> + Specifies verbose message output from + <application>pgxc_ctl</application>. + Value should be <literal>y</literal> or <literal>n</literal>. + Can be overridedn by <option>-v</option> or + <option>--silent</option> command line option. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>xc_prompt <replaceable>prompt</replaceable></option></term> + <listitem> + <para> + Specifies pgxc_prompt. Default is <literal>'PGXC$ '</literal>. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <para> + Typical example of this initialization file will be as follows: +<programlisting> +$ cat ~/.pgxc_ctl +xc_prompt 'PGXC$ ' +verbose y +logMessage 'DEBUG3' +printMessage 'DEBUG1' +printLocation y +$ +</programlisting> + + </para> + + </sect2> + + + <sect2> + <title>Postgres-XL basics and its resources</title> + + <sect3 id="R2-APP-PGXC-CTL-configuration"> + <title>Postgres-XL components</title> + + <para> + Postgres-XL consists of the following components. Each component may + be called <option>node</option>, which may not necessarily refer + to physical or virtual server because you can configure more + than one node to one physical/virtual server. + You should consider how many of them to configure. + Hereafter, we call such physical/virtual server as + <option>host</option>. + </para> + + <variablelist> + + <varlistentry> + <term>GTM</term> + <listitem> + <para> + GTM stands for global transaction manager. + You must have one in the cluster. + For production, GTM should be configured in a separate server. + GTM can have a slave which can fail over when GTM fails. + GTM slave can be installed (hopefully) in a separate server but + can be installed in one of the others where you have gtm_proxy, + coordinators and datanodes. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>GTM-Proxy</term> + <listitem> + <para> + GTM proxy reduces the communication load between coordinator and + GTM and helps GTM failover. + You should configure one gtm_proxy in each server where you have + coordinator or datanode as described below. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Coordinator</term> + <listitem> + <para> + Coordinator handles application connection and statement + handling. + For simplicity and load balancing, it is a good idea to + install coordinator to each server other than where GTM (and GTM + slave) are configured. Coordinator can have a slave. Slave can + be configured in one of the servers where other coordinator + master is installed. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Datanode</term> + <listitem> + <para> + Datanode stores the data and run local SQL statement supplied by + a coordinator. Datanode should also be configured in all the + servers except those for GTM (and GTM slave). + </para> + </listitem> + </varlistentry> + + </variablelist> + </sect3> + + <sect3> + <title>Common resource assignment and configuration practice</title> + <para> + Each component requires the following resources: + + <itemizedlist> + <listitem> + <para> + hostname, IP address or host name you can refer through DNS, + <filename>/etc/hosts</filename> or by equivalent means. + </para> + </listitem> + + <listitem> + <para> + port + </para> + </listitem> + + <listitem> + <para> + work directory + </para> + </listitem> + </itemizedlist> + </para> + + <para> + Also, coordinator and datanode needs additional port for connection pooling to + other nodes. + </para> + + <para> + In the same host, you must not assign the same port and the same + work directory to nodes. + <application>pgxc_ctl</application> checks this. + </para> + + <para> + When assign the port, you should be careful not to assign already + assigned one to other service. + </para> + + <para> + Also, please note the following: + <itemizedlist> + <listitem> + <para> + You should not assign the same port to GTM master and GTM + slave. + </para> + </listitem> + + </itemizedlist> + </para> + + <para> + GTM, coordinators and datanodes can configure their slave. + <application>pgxc_ctl</application> does not support cascaded + slave or more than one slave for coordinator and datanode. + It is not a restriction of <application>postgres-XL</application>, + it is a restriction of <application>pgxc_ctl</application>. + </para> + + <para> + At present, coordinator and datanode slaves are connected using + synchronous replication in pgxc_ctl. + This is not a <application>Posgres-XL</application> restriction + either. + In the future, asynchronous, cascaded and multiple slaves may be + supported. + </para> + + </sect3> + + </sect2> + <sect2> + <title>Configuration</title> + + <para> + As described in the previous section, you can configure your + <application>Postgres-XL</application> cluster by editing + <filename>pgxc_ctl.conf</filename> or other configuration files + manually. + But editing the file from the scratch can be a mess. + It is much better to have separate configuration file. + You can create configuration file template by typing + +<programlisting> +PGXC$ prepare config +PGXC$ +</programlisting> + </para> + + <para> + You have your <filename>pgxc_ctl.conf</filename> file at + <filename>$HOME/pgxc_ctl</filename>. + You can edit it to configure your + <application>Postgres-XL</application> cluster. + When it messes up, you can again create the template with + <command>prepare config</command> command. + If you want to use other file name, specify the names + <command>prepare config</command> command option like: + +<programlisting> +PGXC$ prepare config my_config.conf +</programlisting> + </para> + <para> + Then you can edit this file to configure you + <application>postgres-XL</application> cluster. + This file is actually a bash script file defining many variables + to define the cluster configuration. + With template values and comments, it will be easy to understand + what they mean. + </para> + <para> + You can also generate a minimal configuration file, god enough to test +<application>Postgres-XL</application> on the localhost by specifying +<literal>minimal</literal>. For example: +<programlisting> +PGXC$ prepare config minimal +PGXC$ prepare config minimal my_minimal_config.conf +</programlisting> + + The following describes each variable in the order you find in the + configuration template. + </para> + + <sect3> + <title>Overall</title> + + <variablelist> + + <varlistentry> + <term><option>configBackup</option></term> + <listitem> + <para> + Option if you backup the configuration file to a remote + server. Specify <literal>y</literal> if you'd like to backup the + configuration file. <literal>n</literal> otherwise. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>configBackupFile</option></term> + <listitem> + <para> + Name of the configuration backup file. + Effective when configuration file backup is enabled. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>configBackupHost</option></term> + <listitem> + <para> + Host name (or IP address) where you backup the + configuration file. Effective when configuration file + backup is enabled. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>localTmpDir</option></term> + <listitem> + <para> + Local directory used by pgxc_ctl itself. + You need full access to this directory. + </para> + <para> + This parameter was left here to make it compatible with + bash-version. + It is recommended to configure this parameter in + initialization file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>pgxcInstallDir</option></term> + <listitem> + <para> + <application>Postgres-XL</application> should at least be + installed in the server you are running + <application>pgxc_ctl</application>. + This variable specifies this installation directory, as you + specify with <option>--prefix=</option> option of configure + command when you build it. + All the installation will be copied to the same directory at + each servers and you should give appropriate privilege to this + directory in advance. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>pgxcOwner</option></term> + <listitem> + <para> + Name of the database user who owns whole + <application>Postgres-XL</application> database. + This can be different from $pgxcUser. + In the present version, we assume these two should be the same + though. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>pgxcUser</option></term> + <listitem> + <para> + Name of the operating system user you are logging in as + Postgres-XL owner. At present, this should be the same as + <literal>$pgxcOwner</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>tmpDir</option></term> + <listitem> + <para> + Directory used for work at each server except for the one + pgxc_ctl runs. You need full access to this directory at + all the servers. + </para> + <para> + This parameter was left here to make it compatible with + bash-version. + It is recommended to configure this parameter in + initialization file. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </sect3> + + <sect3> + <title>GTM</title> + + <variablelist> + + <varlistentry> + <term><option>gtmName</option></term> + <listitem> + <para> + Node name of GTM master. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmExtraConfig</option></term> + <listitem> + <para> + If you'd like to add specific configuration to both GTM master + and slave, specify the file which contains such lines for + gtm.config file. + Otherwise, specify <literal>none</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmMasterDir</option></term> + <listitem> + <para> + Work directory for GTM master. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmMasterPort</option></term> + <listitem> + <para> + Listening port number of GTM master. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmMasterServer</option></term> + <listitem> + <para> + Host name where GTM master runs. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmMasterSpecificExtraConfig</option></term> + <listitem> + <para> + If you'd like to add specific configuration only to GTM + master, specify the file which contains such lines for + <filename>gtm.config</filename> file. + Otherwise, specify <literal>none</literal> + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmSlave</option></term> + <listitem> + <para> + Option to enable GTM slave. + Specify <literal>y</literal> to enable, <literal>n</literal> + otherwise. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmSlaveName</option></term> + <listitem> + <para> + Node name of GTM slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmSlaveDir</option></term> + <listitem> + <para> + Work directory for GTM slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmSlavePort</option></term> + <listitem> + <para> + Listening port number of GTM slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmSlaveServer</option></term> + <listitem> + <para> + Host name where GTM slave runs. Effective only when GTM slave is effective. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmSlaveSpecificExtraConfig</option></term> + <listitem> + <para> + If you'd like to add specific configuration only to GTM slave, + specify the file which contains such lines for + <filename>gtm.config</filename> file. + Otherwise, specify <literal>none</literal>. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </sect3> + + <sect3> + <title>GTM Proxy</title> + + <variablelist> + + <varlistentry> + <term><option>gtmProxy</option></term> + <listitem> + <para> + This specifies if you configure any GTM proxy in your + <application>Postgres-XL</application> cluster. + Specify the value <literal>y</literal> if you configure gtm + proxy in your <application>Postgres-XL</application> cluster. + Otherwise specify <literal>n</literal>. + If you specify <literal>n</literal>, all the other parameters + for gtm_proxy will be ignored. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmProxyDir</option></term> + <listitem> + <para> + This is a shortcut used to assign same work directory to all + the GTM proxies. + You don't have to worry about it when you specify these values + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmProxyDirs</option></term> + <listitem> + <para> + Specify work directory for each GTM proxy. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmPxyExtraConfig</option></term> + <listitem> + <para> + If you'd like to add configuration value to all the GTM proxy, + specify the file name which contains such lines for + <filename>gtm_proxy.conf</filename>. + Otherwise specify <literal>none</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmProxyNames</option></term> + <listitem> + <para> + Specify unique name for each GTM proxy. + This is an array. + In the template, we have four servers for coordinator and + datanode and we have four gtm proxy as well. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmProxyPorts</option></term> + <listitem> + <para> + Specify listening port number for each GTM proxy. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmProxyServers</option></term> + <listitem> + <para> + Specify host name where each of the GTM Proxy runs. + Specify server name as the same order as <literal>$gtmProxyNames</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>gtmPxySpecificExtraConfig</option></term> + <listitem> + <para> + If you'd like to add specific configuration value to each GTM + proxy, specify file names with such lines for + <filename>gtm_proxy.conf</filename>. + Otherwise specify <literal>none</literal>. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </sect3> + + <sect3> + <title>Coordinators</title> + + <variablelist> + + <varlistentry> + <term><option>coordArchLogDir</option></term> + <listitem> + <para> + Shortcut to assign the same WAL archive directory to all the + coordinator slaves. + Not needed if you specify these manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordArchLogDirs</option></term> + <listitem> + <para> + Array of WAL archive log directory for each datanode slave. + If you don't configure coordinator slaves and specify + coordSlave variable value to <literal>n</literal>, you don't + have to worry about this variable. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordExtraConfig</option></term> + <listitem> + <para> + If you would like to add extra configuration value for all the + coordinators, specify the file name containing such lines for + <filename>postgresql.conf</filename>. + Specify <literal>none</literal> otherwise. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordExtraPgHba</option></term> + <listitem> + <para> + File name which contains entries of + <filename>pg_hba.conf</filename> file for all the + coordinators. + Specify <literal>none</literal> if you do not have such file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordMasterDir</option></term> + <listitem> + <para> + Shortcut to assign the same work directory to all the + coordinator masters. + Not needed if you specify these manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordMasterDirs</option></term> + <listitem> + <para> + Array of coordinator master work directory. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordMasterServers</option></term> + <listitem> + <para> + Array of the host name where each coordinator master runs. + Specify in the order of <option>coordNames</option> above. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordMaxWalSender</option></term> + <listitem> + <para> + Shortcut to assign the same value to each member of + <option>coordMaxWalSenders</option>. Not needed if you assign the value + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordMaxWalSenders</option></term> + <listitem> + <para> + Array of coordinator max_wal_senders value. + Note that a master and the slave shares the same value of this + variable. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordNames</option></term> + <listitem> + <para> + Array to specify coordinator names. + Coordinator slave uses the same name as the master. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordPgHbaEntries</option></term> + <listitem> + <para> + Array of CIDR addresses to be added to + <filename>pg_hba.conf</filename>. + Will create pg_hba.conf file entry with + <option>pgxcOwner</option> user. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordPorts</option></term> + <listitem> + <para> + Array of the listening port number for each coordinator. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>poolerPorts</option></term> + <listitem> + <para> + Array of the port number for each pooler. Pooler takes + care of the connection between coordinator and datanode and + needs separate port. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSlave</option></term> + <listitem> + <para> + Specify <literal>y</literal> if you configure coordinator + slave. + <literal>n</literal> otherwise. If you specify + <literal>n</literal>, then all the other variables for + coordinator slave will be ignored. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSlaveDir</option></term> + <listitem> + <para> + Shortcut to assign the same work directory to all the + coordinator slaves. Not needed if you specify these + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSlaveDirs</option></term> + <listitem> + <para> + Array of work directory for each coordinator slaves. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSlaveServers</option></term> + <listitem> + <para> + Array of the hostname where slave of each + coordinator runs. Specify <literal>none</literal> if you don't configure the + slave for specific coordinator. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSlavePorts</option></term> + <listitem> + <para> + Array of the listening port number for each coordinator slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSlavePoolerPorts</option></term> + <listitem> + <para> + Array of the port number for each pooler. Pooler takes + care of the connection between coordinator and datanode and + needs separate port. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>coordSpecificExtraConfig</option></term> + <listitem> + <para> + Array of the filename which contains extra + configuration values for each coordinator. Specify <literal>none</literal> + if you don't have such file. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </sect3> + + <sect3> + <title>Datanodes</title> + + <variablelist> + + <varlistentry> + <term><option>datanodeArchLogDir</option></term> + <listitem> + <para> + Shortcut to assign the same WAL archive directory + to all the datanode slaves. Not needed if you specify these + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeArchLogDirs</option></term> + <listitem> + <para> + Array of WAL archive log directory for each datanode + slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeExtraConfig</option></term> + <listitem> + <para> + If you would like to add extra configuration value + for all the datanodes, specify the file name containing + such lines for postgresql.conf. Specify <literal>none</literal> otherwise. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeExtraPgHba</option></term> + <listitem> + <para> + File name which contains entries for all the + datanodes' pg_hba.conf file. Specify <literal>none</literal> if you don't + have such file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeMasterDir</option></term> + <listitem> + <para> + Shortcut to assign the same work directory to all + the datanode masters. Not needed if you specify these + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeMasterDirs</option></term> + <listitem> + <para> + Array of datanode master work directory. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeMaterServers</option></term> + <listitem> + <para> + Array of the host name where each datanode master runs. + Specify in the order of $coordNames above. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeMaxWalSender</option></term> + <listitem> + <para> + shortcut to assign the same value to each member of + datanodeMaxWalSenders. Not needed if you assign the value + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeMaxWalSenders</option></term> + <listitem> + <para> + Array of datanode max_wal_senders value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeNames</option></term> + <listitem> + <para> + Array to specify coordinator names. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodePgHbaEntries</option></term> + <listitem> + <para> + Array of CIDR addresses to be added to + pg_hba.conf. Will create pg_hba.conf file entry with + $pgxcOwner user. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodePorts</option></term> + <listitem> + <para> + Array of the listening port number for each datanode. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodePoolerPorts</option></term> + <listitem> + <para> + Array of the port number for each pooler. Pooler takes + care of the connection between datanode and datanode and + needs separate port. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSlave</option></term> + <listitem> + <para> + Specify <literal>y</literal> if you configure datanode slaves. <literal>n</literal> + otherwise. If you specify <literal>n</literal>, all the other variables for + datanode slaves will be ignored. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSlaveDir</option></term> + <listitem> + <para> + Shortcut to assign the same work directory to all + the datanode slaves. Not needed if you specify these + manually. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSlaveDirs</option></term> + <listitem> + <para> + Array of work directory for each datanode slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSlaveServers</option></term> + <listitem> + <para> + Array of the hostname where slave of each + datanode runs. Specify <literal>none</literal> if you don't configure the + slave for specific coordinator. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSlavePorts</option></term> + <listitem> + <para> + Array of the listening port number for each datanode slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSlavePoolerPorts</option></term> + <listitem> + <para> + Array of the port number for each pooler. Pooler takes + care of the connection between datanode and datanode and + needs separate port. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSpecificExtraConfig</option></term> + <listitem> + <para> + Array of the filename which contains extra + configuration values for each datanode. Specify <literal>none</literal> + if you don't have such file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>datanodeSpecificExtraPgHba</option></term> + <listitem> + <para> + Array of file names which contain specific + extra pg_hba.conf entry for each datanode. Specify <literal>none</literal> + if you don't have such file. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>primaryDataode</option></term> + <listitem> + <para> + Specify name of the primary node. This must be one of + the name in $datanodeNames. If you don't want the primary + node, specify <literal>N/A</literal> or <literal>none</literal>. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </sect3> + + + </sect2> + + <sect2> + + <title>pgxc_ctl commands</title> + + <para> + <application>pgxc_ctl</application> command names and literal options are case-insensitive. + Other options are case-sensitive. + </para> + + <para> + If other command is given, it will be passed to your shell. + When the shell stops, then the control returns to <application>pgxc_ctl</application>. + </para> + + <variablelist> + + <varlistentry> + <term><literal>add gtm slave <replaceable class="parameter">name</replaceable> <replaceable class="parameter">host</replaceable> <replaceable class="parameter">port</replaceable> <replaceable class="parameter">dir</replaceable></literal></term> + <term><literal>add gtm_proxy <replaceable class="parameter">name</replaceable> <replaceable class="parameter">host</replaceable> <replaceable class="parameter">port</replaceable> <replaceable class="parameter">dir</replaceable></literal></term> + <term><literal>add coordinator master <replaceable class="parameter">name</replaceable> <replaceable class="parameter">host</replaceable> <replaceable class="parameter">port</replaceable> <replaceable class="parameter">pooler</replaceable> <replaceable class="parameter">dir</replaceable>< <replaceable class="parameter">extraServerConf</replaceable> <replaceable class="parameter">extraPgHbaConf</replaceable></literal></term> + <term><literal>add coordinator slave <replaceable class="parameter">name</replaceable> <replaceable class="parameter">host</replaceable> <replaceable class="parameter">port</replaceable> <replaceable class="parameter">pooler</replaceable> <replaceable class="parameter">dir</replaceable> <replaceable class="parameter">archDir</replaceable></literal></term> + <term><literal>add datanode master <replaceable class="parameter">name</replaceable> <replaceable class="parameter">host</replaceable> <replaceable class="parameter">port</replaceable> <replaceable class="parameter">pooler</replaceable> <replaceable class="parameter">dir</replaceable> <replaceable class="parameter">restoreDatanode</replaceable> <replaceable class="parameter">extraServerConf</replaceable> <replaceable class="parameter">extraPgHbaConf</replaceable></literal></term> + <term><literal>add datanode slave <replaceable class="parameter">name</replaceable> <replaceable class="parameter">host</replaceable> <replaceable class="parameter">port</replaceable> <replaceable class="parameter">pooler</replaceable> <replaceable class="parameter">dir</replaceable> <replaceable class="parameter">archDir</replaceable></literal></term> + <listitem> + <para> + Add the specified node to your Postgres-XL cluster. + Each node need host name and its work directory. gtm slave, + gtm_proxy, coordinator master/slave and datanode master/slave need its own port + to listen to. Coordinator and datanode also need pooler port to pool + connections to datanodes. Coordinator and datanode slave need a + directory to receive WAL segments from their master. While adding coordinator + master and datanode master, extra server configuration and extra pg_hba + configuration can be specified in a file. + </para> + <para> + When you add coordinator and datanode master, node information at + all the coordinators will be updated with the new one and gtm_proxy + will be selected automatically based upon where the new node runs. + </para> + <para> + You cannot add slaves without master. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Createdb [ - <replaceable class="parameter">coordinator</replaceable> ] <replaceable class="parameter"> createdb_option ... </replaceable></literal></term> + <listitem> + <para> + Invokes createdb utility to create a new database using specified + coordinator. If no coordinator is specified, pgxc_ctl chooses one + of the available ones. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Createuser[ - <replaceable class="parameter">coordinator</replaceable> ] <replaceable class="parameter"> createuser_option ... </replaceable></literal></term> + <listitem> + <para> + Invokes createuser utility to create a new user using specified + coordinator. Of coordinator is not specified, <application>pgxc_ctl</application> chooses + one of the available ones. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>deploy [ all | <replaceable class="parameter">host ...</replaceable> ]</literal></term> + <listitem> + <para> + Deploys Postgres-XL binaries and other installation material to + specified hosts. If "all" is specified, they will be deployed to + all the hosts found in the configuration file. If list of the host + is specifies, deployment will be done to all the specified hosts, + regardless if they are found in the configuration file or not. + Target directory is taken from the variable <option>pgxcInstallDir</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>failover [ gtm | coordinator <replaceable class="parameter">nodename</replaceable> | datanode <replaceable class="parameter">nodename</replaceable> | <replaceable class="parameter">nodename</replaceable> ]</literal></term> + <listitem> + <para> + Failover specified node to its master. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>init [force] all</literal></term> + <term><literal>init [force] <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>init [force] gtm [ master | slave | all ]</literal></term> + <term><literal>init [force] gtm_proxy [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>init [force] coordinator <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>init [force] coordinator [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>init [force] datanode <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>init [force] datanode [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <listitem> + <para> + Initializes specified nodes. + </para> + <para> + At initialization, all the working directories of each component + will be created if it does not exist. If it does and +<literal>force</literal> is specified, then all the + contents under the working directory will be removed. Without +<literal>force</literal> option, existing non-empty directories will not be +cleaned and the server will start with the existing data. + </para> + <para> + When "all" option is specified, then node information at each + coordinator will be set up. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>kill all</literal></term> + <term><literal>kill <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>kill gtm [ master | slave | all ]</literal></term> + <term><literal>kill gtm_proxy [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>kill coordinator <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>kill coordinator [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>kill datanode <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>kill datanode [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <listitem> + <para> + Kills specified node. If nodename is specified and it has both + master and slave, then both master and slave will be chosen. + </para> + <para> + When killing components, their ports will be cleaned too. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>log [ variable | var ] <replaceable class="parameter">varname</replaceable></literal></term> + <term><literal>log [ message | msg ] <replaceable class="parameter">message_body</replaceable></literal></term> + <listitem> + <para> + Prints the specified contents to the log file. + variable or var option writes specified variable name and its + value. + message or msg option writes specified message. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>monitor all</literal></term> + <term><literal>monitor <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>monitor gtm [ master | slave | all ]</literal></term> + <term><literal>monitor gtm_proxy [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>monitor coordinator <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>monitor coordinator [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>monitor datanode <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>monitor datanode [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <listitem> + <para> + Monitors if specified nodes are running. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>prepare [ <replaceable class="parameter">path</replaceable> ]</literal></term> + <listitem> + <para> + Write pgxc_ctl configuration file template to the specified file. + If path option is not specified, target file will be default + configuration file, or the file specified by configFile option in + <filename>/etc/pgxc_ctl</filename> or <filename>~/.pgxc_ctl</filename>. If you specify relative path, it + will be against <filename>pgxc_ctl_home</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>psql [ - <replaceable class="parameter">coordinator</replaceable> ] <replaceable class="parameter"> psql_option ... </replaceable></literal></term> + <term><literal></literal></term> + <listitem> + <para> + Invokes psql targetted to specified coordinator. If no + coordinator is specifies, <application>pgxc_ctl</application> will choose one of the available + ones. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>q | quit | exit</literal></term> + <listitem> + <para> + Exits pgxc_ctl. This command has no option. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>reconnect gtm_proxy [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <listitem> + <para> + Reconnects specified gtm_proxy to new gtm. This is needed after + you failover gtm to its slave. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>remove gtm slave</literal></term> + <term><literal>remove gtm_proxy <replaceable class="parameter">nodename</replaceable> [ clean ]</literal></term> + <term><literal>remove coordinator [ master| slave ] <replaceable class="parameter">nodename</replaceable> [ clean ]</literal></term> + <term><literal>remove datanode [ master| slave ] <replaceable class="parameter">nodename</replaceable> [ clean ]</literal></term> + <listitem> + <para> + Removes the specified node from the cluster. + If clean option is specified, then the work directory and listening socket will be cleared. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>set <replaceable class="parameter">varname value ...</replaceable></literal></term> + <listitem> + <para> + Set variable value. You can specify multiple values to a + variable. In this case simply specify these values as separated + value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>show [ variable | var ] [ all | <replaceable class="parameter">varname ...</replaceable> ]</literal></term> + <listitem> + <para> + Displays configuration or variable name and its value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>start all</literal></term> + <term><literal>start <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>start gtm [ master | slave | all ]</literal></term> + <term><literal>start gtm_proxy [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>start coordinator <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>start coordinator [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>start datanode <replaceable class="parameter">nodename ...</replaceable></literal></term> + <term><literal>start datanode [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <listitem> + <para> + Starts specified node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>stop [ -m smart | fast | immediate ] all</literal></term> + <term><literal>stop gtm [ master | slave | all ]</literal></term> + <term><literal>stop gtm_proxy [ all | <replaceable class="parameter">nodename ...</replaceable> ]</literal></term> + <term><literal>stop [ -m smart | fast | immediate ] coordinator <replaceable class="parameter">nodename ...</replaceable> </literal></term> + <term><literal>stop [ -m smart | fast | immediate ] coordinator [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ] </literal></term> + <term><literal>stop [ -m smart | fast | immediate ] datanode <replaceable class="parameter">nodename ...</replaceable> </literal></term> + <term><literal>stop [ -m smart | fast | immediate ] datanode [ master | slave ] [ all | <replaceable class="parameter">nodename ...</replaceable> ] </literal></term> + <listitem> + <para> + Stops specified node. For datanode and coordinator, you can + specify stop mode as in "pg_ctl stop" command. + </para> + <para> + When you stop coordinator or datanode slave, the master will be + reconfigured to remove synchronous replication. + </para> + <para> + When you stop coordinator or datanode slave, the master will be + reconfigurated to remove synchronous replication. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>unregister <replaceable class="parameter">unregister_option ...</replaceable></literal></term> + <listitem> + <para> + Unregisteres specified node from the gtm. This could be needed + when some node crashes and would like to start new one. + + </para> + <para> + unregister_option is one of the following: + + </para> + <para> + <option>-n</option> <replaceable class="parameter">name</replaceable>: Specifies node name to unregister. + </para> + <para> + <option>-Z</option> <literal>{ gtm | gtm_proxy | gtm_proxy_postmaster | coordinator | datanode }</literal>: + Specifies the category of the specified node. + + </para> + </listitem> + </varlistentry> + + </variablelist> + </sect2> +</sect1> diff --git a/doc/src/sgml/pgxcclean.sgml b/doc/src/sgml/pgxcclean.sgml new file mode 100644 index 0000000000..f305afacc4 --- /dev/null +++ b/doc/src/sgml/pgxcclean.sgml @@ -0,0 +1,208 @@ +<sect1 id="pgxcclean" xreflabel="pgxcclean"> + +<title>pgxc_clean</title> + + <indexterm zone="pgxcclean"> + <primary>pgxc_clean</primary> + </indexterm> + + <sect2> + <title>Overview</title> + + <para> + pgxc_clean has the following syntax. +<programlisting> +pgxc_clean <optional> <replaceable>option</> </optional> <optional><replaceable>dbname</><optional><replaceable>username</></optional></optional> +</programlisting> + </para> + + <para> + <application>pgxc_clean</application> is a Postgres-XL utility to maintain transaction status after a crash. + When a Postgres-XL node crashes and recovers or fails over, the commit status of such node may be inconsistent + with other nodes. <application>pgxc_clean</application> checks transaction commit status and corrects them. + </para> + + <para> + You should run this utility against one of the available Coordinators. The tool cleans up transaction status + of all the nodes automatically. + </para> + </sect2> + + <sect2> + <title>Options</title> + + <variablelist> + <varlistentry> + <term><option>-a</></term> + <term><option>--all</></term> + <listitem> + <para> + Cleanup all the database available. + <literal>all</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-d<replaceable class="parameter">databasename</replaceable></option></term> + <term><option>--dbname=<replaceable class="parameter">databasename</replaceable></option></term> + <listitem> + <para> + Database name to clean up. This option can be specified multiple times for more than one database. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-h<replaceable class="parameter">hostname</replaceable></></term> + <term><option>--command=<replaceable class="parameter">hostname</replaceable></></term> + <listitem> + <para> + Hostname of the Coordinator to connect to. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-N</></term> + <term><option>--no-clean</></term> + <listitem> + <para> + If this option is specified, <application>pgxc_clean</> will + not perform the cleanup. It just investigates transaction status. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-o <replaceable class="parameter">filename</replaceable></></term> + <term><option>--output=<replaceable class="parameter">filename</replaceable></></term> + <listitem> + <para> + Name of the file where <application>pgxc_clean</> output will + be written. If not specified, stdout and stderr will be used. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-p <replaceable class="parameter">port_number</replaceable></></term> + <term><option>--port=<replaceable class="parameter">port_number</replaceable></></term> + <listitem> + <para> + Specifies the port number of the Coordinator. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-q</></term> + <term><option>--quiet</></term> + <listitem> + <para> + Surpress messages as much as possible. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-s</></term> + <term><option>--status</></term> + <listitem> + <para> + Prints investigated two phase commit status. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-U <replaceable class="parameter">username</replaceable></></term> + <term><option>--username=<replaceable class="parameter">username</replaceable></></term> + <listitem> + <para> + User name to use. You must be a superuser of the database. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v</></term> + <term><option>--verbose</></term> + <listitem> + <para> + Write as much information as possible. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-V</></term> + <term><option>--version</></term> + <listitem> + <para> + Writes the version of the utility and exits. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-w</></term> + <term><option>--no-password</></term> + <listitem> + <para> + Never issue a password prompt. If the server requires password + authentication and a password is not available by other means + such as a <filename>.pgpass</filename> file, the connection + attempt will fail. This option can be useful in batch jobs and + scripts where no user is present to enter a password. + </para> + + <para> + Note that this option will remain set for the entire session, + and so it affects uses of the meta-command + <command>\connect</command> as well as the initial connection attempt. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-W</></term> + <term><option>--password</></term> + <listitem> + <para> + Force <application>psql</application> to prompt for a + password before connecting to a database. + </para> + + <para> + This option is never essential, since <application>psql</application> + will automatically prompt for a password if the server demands + password authentication. However, <application>psql</application> + will waste a connection attempt finding out that the server wants a + password. In some cases it is worth typing <option>-W</> to avoid + the extra connection attempt. + </para> + + <para> + Note that this option will remain set for the entire session, + and so it affects uses of the meta-command + <command>\connect</command> as well as the initial connection attempt. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-?</></term> + <term><option>--help</></term> + <listitem> + <para> + Show help about <application>psql</application> command line + arguments, and exit. + </para> + </listitem> + </varlistentry> + + </variablelist> + </sect2> + +</sect1> diff --git a/doc/src/sgml/pgxcddl.sgml b/doc/src/sgml/pgxcddl.sgml new file mode 100644 index 0000000000..1f64bb7ed7 --- /dev/null +++ b/doc/src/sgml/pgxcddl.sgml @@ -0,0 +1,186 @@ +<sect1 id="pgxcddl" xreflabel="pgxcddl"> + +<title>pgxc_ddl</title> + + <indexterm zone="pgxcddl"> + <primary>pgxc_ddl</primary> + </indexterm> + + <sect2> + <title> + Overview + </title> + + <para> + pgxc_ddl is a module for Coordinator catalog table synchronization. + It has the following syntax. + +<programlisting> +pgxc_ddl <optional> <replaceable>option</> </optional> +</programlisting> + </para> + + <para> + pgxc_ddl is used to synchronize all Coordinator catalog tables from + one chosen by a user. It is also possible to launch a DDL on one + Coordinator, and then to synchronize all the Coordinator catalog + tables from the catalog table of the Coordinator having received the + DDL. The copy method is "cold". All the Coordinators are stopped, + catalog files are copied, then all the Coordinators are restarted. + </para> + + <para> + This utility is not really needed in <productname>Postgres-XL</productname>; DDL is + synchronized automatically on all the nodes of the + cluster. + </para> + + </sect2> + + <sect2> + <title>Options</title> + + <para> + Options are specified with preceding '<literal>-</literal>', each + option may be associated with a value. + </para> + + <para> + Options are as follows: + </para> + + <para> + <variablelist> + <varlistentry> + <term><option>-D</option></term> + <listitem> + <para> + Specify <filename>pgxc.conf</filename> folder, for + characteristics of all the Coordinators. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-l</option></term> + <listitem> + <para> + Specify application folder, in case <varname>PATH</varname> is not defined. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-f</option></term> + <listitem> + <para> + Specify DDL file location. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-d</option></term> + <listitem> + <para> + Database name. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-n</option></term> + <listitem> + <para> + Coordinator number. Coordinator chosen corresponds to the one + defined in <filename>pgxc.conf</filename> at the nth place. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-t</option></term> + <listitem> + <para> + Specify temporary folder where to copy the configuration files + <filename>postgresql.conf</filename> and <filename>pg_hba.conf</filename> + for each Coordinator. + </para> + </listitem> + </varlistentry> + + </variablelist> + </para> + + <para> + Because <command>pgxc_ddl</command> requires access to Coordinator + configuration file and data folders, the following parameters have + to be set in <filename>pgxc.conf</filename>: + </para> + + <table> + <title><filename>pgxc.conf</filename> entries</title> + + <tgroup cols="3"> + <thead> + <row> + <entry>Name</entry> + <entry>Type</entry> + <entry>Description</entry> + </row> + </thead> + + <tbody> + <row> + <entry><varname>coordinator_ports</varname></entry> + <entry>string</entry> + <entry> + Specify the port number of all the Coordinators. Maintain the + order of the value same as those in coordinator_hosts. It is + necessary to specify a number of ports equal to the number of + hosts. A comma separator is also necessary. + </entry> + </row> + + <row> + <entry><varname>coordinator_folders</varname></entry> + <entry>string</entry> + <entry> + Specify the data folders of all the Coordinators. Maintain the + order of the value same as those in coordinator_hosts. It is + necessary to specify a number of data folders equal to the + number of hosts. A comma separator is also necessary. + </entry> + </row> + + <row> + <entry><varname>coordinator_hosts</varname></entry> + <entry>string</entry> + <entry> + Specify the host name or IP address of Coordinator. Separate + each value with comma. + </entry> + </row> + </tbody> + </tgroup> + </table> + + <note> + <para> + About the temporary folder, this one has the name chosen by the + user. As an extension name, the PID of the shell script is + added. This folder is by default in <filename>/tmp</filename>. The user can choose + the name freely. + </para> + </note> + + <note> + <para> + Configuration files of Coordinators that have their catalog files + changed are defined with an extension name postgresql.conf.number, + "number" being the number of t Coordinator in the order defined + in <filename>pgxc.conf</filename>. + </para> + </note> + </sect2> +</sect1> diff --git a/doc/src/sgml/pgxcmonitor.sgml b/doc/src/sgml/pgxcmonitor.sgml new file mode 100644 index 0000000000..0132dc2749 --- /dev/null +++ b/doc/src/sgml/pgxcmonitor.sgml @@ -0,0 +1,162 @@ +<sect1 id="pgxcmonitor" xreflabel="pgxcmonitor"> + +<title>pgxc_monitor</title> + + <indexterm zone="pgxcmonitor"> + <primary>pgxc_monitor</primary> + </indexterm> + + <sect2> + <title>Overview</title> + + <para> + pgxc_monitor has the following synopsis. +<programlisting> +pgxc_monitor <optional> <replaceable>option</> </optional> +</programlisting> + </para> + + <para> + <application>pgxc_monitor</application> is a <productname>Postgres-XL</> utility to test if + the target node is running. + </para> + + <para> + The target node is specified by option. + </para> + + <para> + If the target node is running, it exits with exit code zero. + If not, it exits with non-zero exit code. + </para> + + <para> + If invalid options are specified, it exits with exit code 3. + </para> + </sect2> + + <sect2> + <title>Options</title> + + <variablelist> + + <varlistentry> + <term><option>-Z <replaceable class="parameter">nodetype</replaceable></option></term> + <listitem> + <para> + Type of node type to test. Specify <literal>gtm</> as <replaceable>nodetype</replaceable> + for gtm and gtm_proxy and <literal>node</> as <replaceable>nodetype</replaceable> for a + Coordinator or a Datanode. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-h <replaceable class="parameter">hostname</replaceable></></term> + <listitem> + <para> + Hostname of the test target. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-n <replaceable class="parameter">nodename</replaceable></></term> + <listitem> + <para> + Node name to use when testing gtm or gtm_proxy. + Default value is <literal>pgxc_monitor</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-p <replaceable class="parameter">port_number</replaceable></></term> + <listitem> + <para> + Specifies the port number to test target. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-U <replaceable class="parameter">username</replaceable></></term> + <listitem> + <para> + Specifies the database username to connect as. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-d <replaceable class="parameter">database</replaceable></></term> + <listitem> + <para> + Specifies the database name to connect to. Default is "postgres". + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-q</></term> + <listitem> + <para> + Quiet mode. Supress messages as much as possible. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v</></term> + <listitem> + <para> + Verbose mode. Prints as many messages as possible. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>--help</></term> + <listitem> + <para> + Show help about <application>pgxc_monitor</application> command line + arguments, and exit. + </para> + </listitem> + </varlistentry> + + </variablelist> + </sect2> + + <sect2> + <title>Options</title> + + <para> + When monitoring Coordinator or Datanode, <option>-p</option> and + <option>-h</option> options can be supplied using + <literal>.pgpass</literal> file. + If you use non-default target database name, and username, as well as + password, + they must also be supplied using <option>.pgpass</option> file. + </para> + <para> + If password is needed, it must also be supplied using + <option>.pgpass</option> file too. + </para> + <para> + Monitoring Coordinator and Datanode uses system(3) function. + Therefore,you should not use set-userid bit or set-groupid bit. + Also, because this uses psql command, psql must be in your PATH. + </para> + <para> + The username and database name can be specified via command line + options too. If password is needed, it must be supplied via + <option>.pgpass</option> file though. + </para> + <para> + If invalid parameters are given, + error message will be printed even if <option>-q</option> is specified. + </para> + </sect2> + +</sect1> diff --git a/doc/src/sgml/plperl.sgml b/doc/src/sgml/plperl.sgml index 9117769125..640cb375f7 100644 --- a/doc/src/sgml/plperl.sgml +++ b/doc/src/sgml/plperl.sgml @@ -569,6 +569,13 @@ SELECT * from lotsa_md5(500); </term> <listitem> + + <para> + <command>PREPARE</> is not supported in the current release + of <productname>Postgres-XL</>. This may be supported in the + future releases. + </para> + <para> <literal>spi_prepare</literal>, <literal>spi_query_prepared</literal>, <literal>spi_exec_prepared</literal>, and <literal>spi_freeplan</literal> implement the same functionality but for prepared queries. diff --git a/doc/src/sgml/plpgsql.sgml b/doc/src/sgml/plpgsql.sgml index 9a7763d18c..fdd597fa84 100644 --- a/doc/src/sgml/plpgsql.sgml +++ b/doc/src/sgml/plpgsql.sgml @@ -65,6 +65,11 @@ administrators could choose to remove it. </para> + <para> + At present, in <productname>Postgres-XL</>, only one SQL statment + can be handled in a function. + </para> + <sect2 id="plpgsql-advantages"> <title>Advantages of Using <application>PL/pgSQL</application></title> @@ -1021,6 +1026,11 @@ DELETE ... RETURNING <replaceable>expressions</replaceable> INTO <optional>STRIC <application>PL/pgSQL</application> function, use the syntax <command>CREATE TABLE ... AS SELECT</command>. </para> + + <para> + <command>SELECT INTO</> command of <productname>PostgerSQL</>, + is not supported by <productname>Postgres-XL</> yet. + </para> </tip> <para> diff --git a/doc/src/sgml/pltcl.sgml b/doc/src/sgml/pltcl.sgml index d2175d552e..005e2a5bb2 100644 --- a/doc/src/sgml/pltcl.sgml +++ b/doc/src/sgml/pltcl.sgml @@ -714,6 +714,11 @@ CREATE TRIGGER trig_mytab_modcount BEFORE INSERT OR UPDATE ON mytab name; that's supplied from the trigger arguments. This lets the trigger procedure be reused with different tables. </para> + + <para> + Trigger is not supported in the current release + of <productname>Postgres-XL</>. + </para> </sect1> <sect1 id="pltcl-event-trigger"> diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml index d1703e9c01..4c2e962fac 100644 --- a/doc/src/sgml/postgres.sgml +++ b/doc/src/sgml/postgres.sgml @@ -12,11 +12,11 @@ ]> <book id="postgres"> - <title>PostgreSQL &version; Documentation</title> + <title>Postgres-XL &version; Documentation</title> <bookinfo> - <corpauthor>The PostgreSQL Global Development Group</corpauthor> - <productname>PostgreSQL</productname> + <corpauthor>The Postgres-XL Global Development Group</corpauthor> + <productname>Postgres-XL</productname> <productnumber>&version;</productnumber> &legal; </bookinfo> @@ -28,19 +28,29 @@ <partintro> <para> - Welcome to the <productname>PostgreSQL</productname> Tutorial. The + Welcome to the <productname>Postgres-XL</productname> Tutorial. The following few chapters are intended to give a simple introduction - to <productname>PostgreSQL</productname>, relational database + to <productname>Postgres-XL</productname>, relational database concepts, and the SQL language to those who are new to any one of these aspects. We only assume some general knowledge about how to use computers. No particular Unix or programming experience is required. This part is mainly intended to give you some hands-on experience with important aspects of the - <productname>PostgreSQL</productname> system. It makes no attempt + <productname>Postgres-XL</productname> system. It makes no attempt to be a complete or thorough treatment of the topics it covers. </para> <para> + Throughout this documentation, the term <productname>PostgreSQL</> also refers to + <productname>Postgres-XL</> unless specifically stated otherwise. We have + deliberately retained the name <productname>PostgreSQL</> to emphasise that + <productname>Postgres-XL</> inherits many things from + <productname>PostgreSQL</> and is very similar to the parent product. The + differeneces are clearly mentioned and also documented separately for easy + reference (link required). + </para> + + <para> After you have worked through this tutorial you might want to move on to reading <xref linkend="sql"> to gain a more formal knowledge of the SQL language, or <xref linkend="client-interfaces"> for diff --git a/doc/src/sgml/problems.sgml b/doc/src/sgml/problems.sgml index 3f79c6ef90..2efb999c9e 100644 --- a/doc/src/sgml/problems.sgml +++ b/doc/src/sgml/problems.sgml @@ -4,11 +4,11 @@ <title>Bug Reporting Guidelines</title> <para> - When you find a bug in <productname>PostgreSQL</productname> we want to + When you find a bug in <productname>Postgres-XL</productname> we want to hear about it. Your bug reports play an important part in making - <productname>PostgreSQL</productname> more reliable because even the utmost + <productname>Postgres-XL</productname> more reliable because even the utmost care cannot guarantee that every part of - <productname>PostgreSQL</productname> + <productname>Postgres-XL</productname> will work on every platform under every circumstance. </para> @@ -236,9 +236,9 @@ If your version is older than &version; we will almost certainly tell you to upgrade. There are many bug fixes and improvements in each new release, so it is quite possible that a bug you have - encountered in an older release of <productname>PostgreSQL</> + encountered in an older release of <productname>Postgres-XL</> has already been fixed. We can only provide limited support for - sites using older releases of <productname>PostgreSQL</>; if you + sites using older releases of <productname>Postgres-XL</>; if you require more than we can provide, consider acquiring a commercial support contract. </para> @@ -278,11 +278,14 @@ <para> When writing a bug report, please avoid confusing terminology. - The software package in total is called <quote>PostgreSQL</quote>, - sometimes <quote>Postgres</quote> for short. If you - are specifically talking about the backend process, mention that, do not - just say <quote>PostgreSQL crashes</quote>. A crash of a single - backend process is quite different from crash of the parent + The software package in total is called <quote>Postgres-XL</quote>. + If you + are specifically talking about the backend server, mention that, do not + just say <quote>Postgres-XL crashes</quote>. + <quote>Postgres-XL</> consists of several separate component. Please + write which component crashes. + A crash of a single + backend server process is quite different from crash of the parent <quote>postgres</> process; please don't say <quote>the server crashed</> when you mean a single backend process went down, nor vice versa. Also, client programs such as the interactive frontend <quote><application>psql</application></quote> @@ -295,58 +298,39 @@ <title>Where to Report Bugs</title> <para> - In general, send bug reports to the bug report mailing list at - <email>[email protected]</email>. + In general, send bug reports to the bug report mailing list. + <footnote> + <para> + <email>[email protected]</email> + </para> + </footnote> You are requested to use a descriptive subject for your email message, perhaps parts of the error message. </para> <para> - Another method is to fill in the bug report web-form available - at the project's - <ulink url="http://www.postgresql.org/">web site</ulink>. - Entering a bug report this way causes it to be mailed to the - <email>[email protected]</email> mailing list. - </para> - - <para> - If your bug report has security implications and you'd prefer that it - not become immediately visible in public archives, don't send it to - <literal>pgsql-bugs</literal>. Security issues can be - reported privately to <email>[email protected]</email>. - </para> - - <para> - Do not send bug reports to any of the user mailing lists, such as - <email>[email protected]</email> or - <email>[email protected]</email>. - These mailing lists are for answering - user questions, and their subscribers normally do not wish to receive - bug reports. More importantly, they are unlikely to fix them. - </para> - - <para> Also, please do <emphasis>not</emphasis> send reports to - the developers' mailing list <email>[email protected]</email>. + the developers' mailing list <email>[email protected]</email>. This list is for discussing the - development of <productname>PostgreSQL</productname>, and it would be nice + development of <productname>Postgres-XL</productname>, and it would be nice if we could keep the bug reports separate. We might choose to take up a - discussion about your bug report on <literal>pgsql-hackers</literal>, + discussion about your bug report on <literal>postgres-xl-developers</literal>, if the problem needs more review. </para> <para> If you have a problem with the documentation, the best place to report it - is the documentation mailing list <email>[email protected]</email>. + is the documentation mailing list +<email>[email protected]</email>. Please be specific about what part of the documentation you are unhappy with. </para> <para> If your bug is a portability problem on a non-supported platform, - send mail to <email>[email protected]</email>, + send mail to <email>[email protected]</email>, so we (and you) can work on - porting <productname>PostgreSQL</productname> to your platform. + porting <productname>Postgres-XL</productname> to your platform. </para> <note> @@ -357,9 +341,6 @@ subscribed to use the bug-report web form, however.) If you would like to send mail but do not want to receive list traffic, you can subscribe and set your subscription option to <literal>nomail</>. - For more information send mail to - <email>[email protected]</email> - with the single word <literal>help</> in the body of the message. </para> </note> </sect2> diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml index 61cf86ca80..321570ea85 100644 --- a/doc/src/sgml/recovery-config.sgml +++ b/doc/src/sgml/recovery-config.sgml @@ -157,8 +157,8 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows By default, recovery will recover to the end of the WAL log. The following parameters can be used to specify an earlier stopping point. At most one of <varname>recovery_target</>, - <varname>recovery_target_name</>, <varname>recovery_target_time</>, or - <varname>recovery_target_xid</> can be used; if more than one of these + <varname>recovery_target_name</>, <varname>recovery_target_time</>, + <varname>recovery_target_xid</> or <varname>recovery_target_barrier</> can be used; if more than one of these is specified in the configuration file, the last entry will be used. </para> @@ -232,6 +232,23 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows </para> </listitem> </varlistentry> + <varlistentry id="recovery-target-barrier" xreflabel="recovery_target_barrier"> + <term><varname>recovery_target_barrier</varname> (<type>string</type>) + <indexterm> + <primary><varname>recovery_target_barrier</> recovery parameter</primary> + </indexterm> + </term> + <listitem> + <para> + This parameter specifies the barrier ID up to which recovery + will proceed. A global consistency is guaranteed when recovery is + stopped at a previously successfully completed barrier. At most + one of <varname>recovery_target_xid</>, + <xref linkend="recovery-target-time"> and + <varname>recovery_target_barrier</> can be specified. + </para> + </listitem> + </varlistentry> </variablelist> <para> diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml index bf95453b6c..0deb459f1f 100644 --- a/doc/src/sgml/ref/allfiles.sgml +++ b/doc/src/sgml/ref/allfiles.sgml @@ -22,6 +22,7 @@ Complete list of usable sgml source files in this directory. <!ENTITY alterLanguage SYSTEM "alter_language.sgml"> <!ENTITY alterLargeObject SYSTEM "alter_large_object.sgml"> <!ENTITY alterMaterializedView SYSTEM "alter_materialized_view.sgml"> +<!entity alterNode SYSTEM "alter_node.sgml"> <!ENTITY alterOperator SYSTEM "alter_operator.sgml"> <!ENTITY alterOperatorClass SYSTEM "alter_opclass.sgml"> <!ENTITY alterOperatorFamily SYSTEM "alter_opfamily.sgml"> @@ -46,6 +47,7 @@ Complete list of usable sgml source files in this directory. <!ENTITY analyze SYSTEM "analyze.sgml"> <!ENTITY begin SYSTEM "begin.sgml"> <!ENTITY checkpoint SYSTEM "checkpoint.sgml"> +<!ENTITY cleanConnection system "clean_connection.sgml"> <!ENTITY close SYSTEM "close.sgml"> <!ENTITY cluster SYSTEM "cluster.sgml"> <!ENTITY commentOn SYSTEM "comment.sgml"> @@ -53,6 +55,7 @@ Complete list of usable sgml source files in this directory. <!ENTITY commitPrepared SYSTEM "commit_prepared.sgml"> <!ENTITY copyTable SYSTEM "copy.sgml"> <!ENTITY createAggregate SYSTEM "create_aggregate.sgml"> +<!ENTITY createBarrier system "create_barrier.sgml"> <!ENTITY createCast SYSTEM "create_cast.sgml"> <!ENTITY createCollation SYSTEM "create_collation.sgml"> <!ENTITY createConversion SYSTEM "create_conversion.sgml"> @@ -67,6 +70,8 @@ Complete list of usable sgml source files in this directory. <!ENTITY createIndex SYSTEM "create_index.sgml"> <!ENTITY createLanguage SYSTEM "create_language.sgml"> <!ENTITY createMaterializedView SYSTEM "create_materialized_view.sgml"> +<!ENTITY createNode SYSTEM "create_node.sgml"> +<!ENTITY createNodeGroup SYSTEM "create_nodegroup.sgml"> <!ENTITY createOperator SYSTEM "create_operator.sgml"> <!ENTITY createOperatorClass SYSTEM "create_opclass.sgml"> <!ENTITY createOperatorFamily SYSTEM "create_opfamily.sgml"> @@ -109,6 +114,8 @@ Complete list of usable sgml source files in this directory. <!ENTITY dropIndex SYSTEM "drop_index.sgml"> <!ENTITY dropLanguage SYSTEM "drop_language.sgml"> <!ENTITY dropMaterializedView SYSTEM "drop_materialized_view.sgml"> +<!ENTITY dropNode SYSTEM "drop_node.sgml"> +<!ENTITY dropNodeGroup SYSTEM "drop_nodegroup.sgml"> <!ENTITY dropOperator SYSTEM "drop_operator.sgml"> <!ENTITY dropOperatorClass SYSTEM "drop_opclass.sgml"> <!ENTITY dropOperatorFamily SYSTEM "drop_opfamily.sgml"> @@ -133,6 +140,7 @@ Complete list of usable sgml source files in this directory. <!ENTITY dropView SYSTEM "drop_view.sgml"> <!ENTITY end SYSTEM "end.sgml"> <!ENTITY execute SYSTEM "execute.sgml"> +<!ENTITY executeDirect system "execute_direct.sgml"> <!ENTITY explain SYSTEM "explain.sgml"> <!ENTITY fetch SYSTEM "fetch.sgml"> <!ENTITY grant SYSTEM "grant.sgml"> @@ -143,6 +151,7 @@ Complete list of usable sgml source files in this directory. <!ENTITY lock SYSTEM "lock.sgml"> <!ENTITY move SYSTEM "move.sgml"> <!ENTITY notify SYSTEM "notify.sgml"> +<!ENTITY pauseCluster SYSTEM "pause_cluster.sgml"> <!ENTITY prepare SYSTEM "prepare.sgml"> <!ENTITY prepareTransaction SYSTEM "prepare_transaction.sgml"> <!ENTITY reassignOwned SYSTEM "reassign_owned.sgml"> @@ -167,6 +176,7 @@ Complete list of usable sgml source files in this directory. <!ENTITY startTransaction SYSTEM "start_transaction.sgml"> <!ENTITY truncate SYSTEM "truncate.sgml"> <!ENTITY unlisten SYSTEM "unlisten.sgml"> +<!ENTITY unpauseCluster SYSTEM "unpause_cluster.sgml"> <!ENTITY update SYSTEM "update.sgml"> <!ENTITY vacuum SYSTEM "vacuum.sgml"> <!ENTITY values SYSTEM "values.sgml"> @@ -180,7 +190,11 @@ Complete list of usable sgml source files in this directory. <!ENTITY droplang SYSTEM "droplang.sgml"> <!ENTITY dropuser SYSTEM "dropuser.sgml"> <!ENTITY ecpgRef SYSTEM "ecpg-ref.sgml"> +<!ENTITY gtm system "gtm.sgml"> +<!ENTITY gtmPxy system "gtm_proxy.sgml"> +<!ENTITY gtmCtl system "gtm_ctl.sgml"> <!ENTITY initdb SYSTEM "initdb.sgml"> +<!ENTITY initgtm SYSTEM "initgtm.sgml"> <!ENTITY pgarchivecleanup SYSTEM "pgarchivecleanup.sgml"> <!ENTITY pgBasebackup SYSTEM "pg_basebackup.sgml"> <!ENTITY pgbench SYSTEM "pgbench.sgml"> diff --git a/doc/src/sgml/ref/alter_database.sgml b/doc/src/sgml/ref/alter_database.sgml index 8b6fa5816f..ff5a7711af 100644 --- a/doc/src/sgml/ref/alter_database.sgml +++ b/doc/src/sgml/ref/alter_database.sgml @@ -95,6 +95,13 @@ ALTER DATABASE <replaceable class="PARAMETER">name</replaceable> RESET ALL database. Certain variables cannot be set this way, or can only be set by a superuser. </para> + + <para> + If there are any live connections to any of the template databases in + one of the Coordinators or Datanodes, you will have an error message. In this + case, you should clean these connections using <command>CLEAN + CONNECITON</> statement. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/alter_foreign_data_wrapper.sgml b/doc/src/sgml/ref/alter_foreign_data_wrapper.sgml index 3f5fb0f77e..7da761dfe6 100644 --- a/doc/src/sgml/ref/alter_foreign_data_wrapper.sgml +++ b/doc/src/sgml/ref/alter_foreign_data_wrapper.sgml @@ -34,6 +34,11 @@ ALTER FOREIGN DATA WRAPPER <replaceable class="parameter">name</replaceable> REN <title>Description</title> <para> + <command>FOREIGN DATA WRAPPER</> is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>ALTER FOREIGN DATA WRAPPER</command> changes the definition of a foreign-data wrapper. The first form of the command changes the support functions or the generic options of the diff --git a/doc/src/sgml/ref/alter_large_object.sgml b/doc/src/sgml/ref/alter_large_object.sgml index a0ed6c22f3..cf1ed6fbfe 100644 --- a/doc/src/sgml/ref/alter_large_object.sgml +++ b/doc/src/sgml/ref/alter_large_object.sgml @@ -28,6 +28,10 @@ ALTER LARGE OBJECT <replaceable class="PARAMETER">large_object_oid</replaceable> <refsect1> <title>Description</title> + <para> + <command>LARGE OBJECT</> is not supported by <productname>Postgres-XL</> yet. + </para> + <para> <command>ALTER LARGE OBJECT</command> changes the definition of a large object. The only functionality is to assign a new owner. diff --git a/doc/src/sgml/ref/alter_node.sgml b/doc/src/sgml/ref/alter_node.sgml new file mode 100644 index 0000000000..733ac9dcad --- /dev/null +++ b/doc/src/sgml/ref/alter_node.sgml @@ -0,0 +1,159 @@ +<refentry id="SQL-ALTERNODE"> + <indexterm zone="sql-alternode"> + <primary>ALTER NODE</primary> + </indexterm> + + <refmeta> + <refentrytitle>ALTER NODE</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ALTER NODE</refname> + <refpurpose>alter a cluster node</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +ALTER NODE <replaceable class="parameter">nodename</replaceable> WITH + ( + [ TYPE = <replaceable class="parameter">nodetype</replaceable>,] + [ HOST = <replaceable class="parameter">hostname</replaceable>,] + [ PORT = <replaceable class="parameter">portnum</replaceable>,] + [ PRIMARY [ = <replaceable class="parameter">boolean</replaceable>],] + [ PREFERRED [ = <replaceable class="parameter">boolean</replaceable> ] ] + ) + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>ALTER NODE</command> is a new SQL command specific + to <productname>Postgres-XL</productname> that modifies + cluster node information in catalog pgxc_node. + </para> + <para> + Node connection that has been modified does not guaranty that connection + information cached in pooler is updated accordingly. + </para> + <para> + <command>ALTER NODE</command> only runs on the local node where it is launched. + </para> + + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><replaceable class="parameter">nodename</replaceable></term> + <listitem> + <para> + The name of the selected cluster node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>TYPE</literal></term> + <listitem> + <para> + The type of the cluster node. It is possible to specify + a Coordinator node or a Datanode node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>PRIMARY</literal></term> + <listitem> + <para> + Defines if the cluster node is used as a primary node for replicated + write operations. A <replaceable class="parameter">boolean</replaceable> + value can be specified. In case no value is specified, <literal>PRIMARY</literal> + acts like <literal>true</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>PREFERRED</literal></term> + <listitem> + <para> + Defines if the cluster node is used as a preferred node for replicated + read operations. A <replaceable class="parameter">boolean</replaceable> + value can be specified. In case no value is specified, <literal>PREFERRED</literal> + acts like <literal>true</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">nodetype</replaceable></term> + <listitem> + <para> + The node type for given cluster node. Possible values are: + 'coordinator' for a Coordinator node and 'datanode' for a + Datanode node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">hostname</replaceable></term> + <listitem> + <para> + The hostname or IP used to connect to the cluster node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">portnum</replaceable></term> + <listitem> + <para> + The port number used to connect to the cluster node. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect1> + + <refsect1> + <title>Notes</title> + <para> + A Datanode can be modified as <literal>PRIMARY</literal> and + as <literal>PREFERRED</literal> as many times as necessary. + </para> + + <para> + A node type cannot be modified. + </para> + </refsect1> + + <refsect1> + <title>Examples</title> + <para> + Modify a Coordinator node located on local machine to use port 6543. +<programlisting> +ALTER NODE coord_node WITH (PORT = 6543, HOST = 'localhost'); +</programlisting> + </para> + + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>ALTER NODE</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> diff --git a/doc/src/sgml/ref/alter_server.sgml b/doc/src/sgml/ref/alter_server.sgml index e6cf511853..43f4337860 100644 --- a/doc/src/sgml/ref/alter_server.sgml +++ b/doc/src/sgml/ref/alter_server.sgml @@ -32,6 +32,10 @@ ALTER SERVER <replaceable class="PARAMETER">name</replaceable> RENAME TO <replac <title>Description</title> <para> + <command>SERVER</> is not yet supported in <productname>Postgres-XL</>. + </para> + + <para> <command>ALTER SERVER</command> changes the definition of a foreign server. The first form changes the server version string or the generic options of the server (at least one clause is required). diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 207fec1758..c030d2a8ff 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -681,6 +681,107 @@ ALTER TABLE ALL IN TABLESPACE <replaceable class="PARAMETER">name</replaceable> </listitem> </varlistentry> + <varlistentry> + <term><literal>DISTRIBUTE BY</literal></term> + <listitem> + <para> + This clause specifies how the table is distributed or replicated among Datanodes. + </para> + + <variablelist> + + <varlistentry> + <term><literal>REPLICATION</literal></term> + <listitem> + <para> + Each row of the table will be replicated into all the + Datanodes of the <productname>Postgres-XL</> database + cluster. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ROUNDROBIN</literal></term> + <listitem> + <para> + Each row of the table will be placed in one of the Datanodes + by round-robin manner. The value of the row will not be + needed to determine what Datanode to go. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HASH ( <replaceable class="PARAMETER">column_name</> )</literal></term> + <listitem> + <para> + Each row of the table will be placed based on the hash value + of the specified column. Following type is allowed as + distribution column: INT8, INT2, OID, INT4, BOOL, INT2VECTOR, + OIDVECTOR, CHAR, NAME, TEXT, BPCHAR, BYTEA, VARCHAR, FLOAT4, + FLOAT8, NUMERIC, CASH, ABSTIME, RELTIME, DATE, TIME, + TIMESTAMP, TIMESTAMPTZ, INTERVAL, and TIMETZ. + </para> + <para> + Please note that floating point is not allowed as a basis of + the distribution column. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>MODULO ( <replaceable class="PARAMETER">column_name</> )</literal></term> + <listitem> + <para> + Each row of the table will be placed based on the modulo + of the specified column. Following type is allowed as + distribution column: INT8, INT2, OID, INT4, BOOL, INT2VECTOR, + OIDVECTOR, CHAR, NAME, TEXT, BPCHAR, BYTEA, VARCHAR, FLOAT4, + FLOAT8, NUMERIC, CASH, ABSTIME, RELTIME, DATE, TIME, + TIMESTAMP, TIMESTAMPTZ, INTERVAL, and TIMETZ. + </para> + <para> + Please note that floating point is not allowed as a basis of + the distribution column. + </para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>TO GROUP</literal></term> + <term><literal>TO NODE</literal></term> + <listitem> + <para> + This defines the list of nodes on which table data exists. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ADD NODE</literal></term> + <listitem> + <para> + This adds a list of nodes where data of table is distributed + to the existing list. If the list of nodes added contains nodes + already used by table, an error is returned. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>DELETE NODE</literal></term> + <listitem> + <para> + This deletes a list of nodes where data of table is distributed + to the existing list. If the list of nodes deleted contains nodes + not used by table, an error is returned. + </para> + </listitem> + </varlistentry> </variablelist> </para> @@ -913,6 +1014,24 @@ ALTER TABLE ALL IN TABLESPACE <replaceable class="PARAMETER">name</replaceable> </listitem> </varlistentry> + <varlistentry> + <term><replaceable class="PARAMETER">nodename</replaceable></term> + <listitem> + <para> + It defines a <productname>Postgres-XL</productname> node of catalog pgxc_node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="PARAMETER">groupname</replaceable></term> + <listitem> + <para> + It defines a <productname>Postgres-XL</productname> node group in catalog pgxc_group. + </para> + </listitem> + </varlistentry> + </variablelist> </refsect1> @@ -1030,6 +1149,74 @@ ALTER TABLE ALL IN TABLESPACE <replaceable class="PARAMETER">name</replaceable> parameters. <xref linkend="ddl"> has further information on inheritance. </para> + + <para> + <command>ALTER TABLE</> with clauses <literal>DISTRIBUTE BY</>, <literal>ADD NODE</>, + <literal>DELETE NODE</>, <literal>TO NODE</> or <literal>TO GROUP</> is used for data + redistribution among nodes specific to <productname>Postgres-XL</>. Those clauses cannot be + used with other commands. + </para> + + <para> + Multiple redistribution scenarios are possible depending on modifications done: + <variablelist> + <varlistentry> + <term>Default redistribution:</term> + <listitem> + <para> + This is the slowest scenario possible. It is done in 3 or 4 steps. Data is firstly + saved on Coordinator by fetching all the data with <command>COPY TO</> command. At + this point all the tuples are saved using tuple store. The amount of cache allowed for + tuple store operation can be controlled with <varname>work_mem</>. Then the table is + truncated on all the nodes. Then catalogs are updated. Finally data inside tuple store + is redistributed using an internal <command>COPY FROM</> mechanism. <command>REINDEX</> + is issued if necessary. The overall performance of this scenario is close to the + time necessary to run consecutively <command>COPY TO</> and <command>COPY FROM</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Redistribution from replicated to replicated table:</term> + <listitem> + <para> + The node list of a table can have new nodes as well as removed nodes. + If nodes are only removed, <command>TRUNCATE</> is launched to remote nodes that are + removed. If new nodes are added, then table data is fetch on Coordinator with <command> + COPY TO</> and stored inside a tuplestore controlled with <varname>work_mem</>, then + data stored is only sent to the new nodes using <command>COPY FROM</> with data stored + inside the tuplestore. <command>REINDEX</> is issued if necessary. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Redistribution from replicated to distributed table:</term> + <listitem> + <para> + If the relation node list contains new nodes, the default redistribution + mechanism is used. However, if the node list of relation after redistribution is + included in node list of relation after redistribution, as all the tuples are already + located on remote nodes, it is not necessary to fetch any data on Coordinator. Hence, + <command>DELETE</> is used to remove on remote nodes only the necessary tuples. This + query uses selects tuples to remove with conditions based on the number of nodes in node + list of relation after redistribution, the <literal>HASH</> or <literal>MODULO</> value + used for new distribution and the remote node itself where <command>DELETE</> is launched.. + <command>REINDEX</> is issued if necessary. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Redistribution from distributed to replicated table:</term> + <listitem> + <para> + In this case the default redistribution mechanism is used. + </para> + </listitem> + </varlistentry> + </variablelist> + </para> </refsect1> <refsect1> @@ -1197,6 +1384,28 @@ ALTER TABLE distributors DROP CONSTRAINT distributors_pkey, ADD CONSTRAINT distributors_pkey PRIMARY KEY USING INDEX dist_id_temp_idx; </programlisting></para> + <para> + To change the distribution type and the list of nodes where table data + is located: +<programlisting> +ALTER TABLE distributors TO NODE (dn1, dn7), DISTRIBUTE BY HASH(dist_id); +</programlisting> + </para> + + <para> + To add a node where data of table is distributed: +<programlisting> +ALTER TABLE distributors ADD NODE (dn9, dn14); +</programlisting> + </para> + + <para> + To remove a node where data of table is distributed: +<programlisting> +ALTER TABLE distributors DELETE NODE (dn4, dn0); +</programlisting> + </para> + </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/alter_trigger.sgml b/doc/src/sgml/ref/alter_trigger.sgml index c63b5dfa02..08db86ebee 100644 --- a/doc/src/sgml/ref/alter_trigger.sgml +++ b/doc/src/sgml/ref/alter_trigger.sgml @@ -29,6 +29,11 @@ ALTER TRIGGER <replaceable class="PARAMETER">name</replaceable> ON <replaceable <title>Description</title> <para> + The <command>ALTER TRIGGER</> statement is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>ALTER TRIGGER</command> changes properties of an existing trigger. The <literal>RENAME</literal> clause changes the name of the given trigger without otherwise changing the trigger diff --git a/doc/src/sgml/ref/alter_user_mapping.sgml b/doc/src/sgml/ref/alter_user_mapping.sgml index 3cd51b1413..8d5c461f30 100644 --- a/doc/src/sgml/ref/alter_user_mapping.sgml +++ b/doc/src/sgml/ref/alter_user_mapping.sgml @@ -31,6 +31,10 @@ ALTER USER MAPPING FOR { <replaceable class="parameter">user_name</replaceable> <title>Description</title> <para> + <command>USER MAPPING</> is not yet supported in <productname>Postgres-XL</>. + </para> + + <para> <command>ALTER USER MAPPING</command> changes the definition of a user mapping. </para> diff --git a/doc/src/sgml/ref/checkpoint.sgml b/doc/src/sgml/ref/checkpoint.sgml index fa4dcc4f27..00d65cfa37 100644 --- a/doc/src/sgml/ref/checkpoint.sgml +++ b/doc/src/sgml/ref/checkpoint.sgml @@ -51,6 +51,11 @@ CHECKPOINT <para> Only superusers can call <command>CHECKPOINT</command>. </para> + + <para> + In <productname>Postrgres-XL</>, <command>CHECKPOINT</> is + performed at local Coordinator and all of the underlying Datanodes. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/clean_connection.sgml b/doc/src/sgml/ref/clean_connection.sgml new file mode 100644 index 0000000000..2b0cdee80d --- /dev/null +++ b/doc/src/sgml/ref/clean_connection.sgml @@ -0,0 +1,143 @@ +<!-- +$PostgreSQL: pgsql/doc/src/sgml/ref/clean_connection.sgml,v 1.50 2010/04/03 07:22:59 petere Exp $ +PostgreSQL documentation +--> + +<refentry id="SQL-CLEANCONNECTION"> + <indexterm zone="sql-cleanconnection"> + <primary>CLEAN CONNECTION</primary> + </indexterm> + + <refmeta> + <refentrytitle>CLEAN CONNECTION</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>CLEAN CONNECTION</refname> + <refpurpose>Clean up pooler connections in a cluster</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +CLEAN CONNECTION TO { COORDINATOR ( <replaceable class="parameter">nodename</replaceable> [, ... ] ) | NODE ( <replaceable class="parameter">nodename</replaceable> [, ... ] ) | ALL {FORCE} } + [ FOR DATABASE <replaceable class="parameter">dbname</replaceable> ] + [ TO USER <replaceable class="parameter">username</replaceable> ] +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + <para> + <command>CLEAN CONNECTION</command> cleans pooler connections in + a Postgres-XL cluster. This can be done for a given database or + role. This command's usage is restricted to superusers. It is + possible to clean connections to a list of Postgres-XL Coordinators + or Datanodes. If TO ALL is specified, connections to all the nodes + are cleaned. A <replaceable class="parameter">username</replaceable> + or a <replaceable class="parameter">dbname</replaceable> has to be + specified to perform <command>CLEAN CONNECTION</command>. + </para> + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><literal>FORCE</literal></term> + <listitem> + <para> + It can only be specified with the clause <literal>TO ALL</literal>. + If specified, all the backends connected to the specified node(s) + are closed by sending to them a signal SIGTERM. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">dbname</replaceable></term> + <listitem> + <para> + If specified in the optional clause <literal>FOR DATABASE</literal>, + pooler connections are cleaned for given database. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">username</replaceable></term> + <listitem> + <para> + If specified in the optional clause <literal>TO USER</literal>, + pooler connections are cleaned for given role. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">nodename</replaceable></term> + <listitem> + <para> + In the case of cleaning connections to a given list of + Coordinator, <replaceable class="parameter">nodename</replaceable> + has to be specified with the clause <literal>TO COORDINATOR + </literal>. + </para> + <para> + In the case of cleaning connections to a given list of + Datanodes, <replaceable class="parameter">nodename</replaceable> + has to be specified with the clause <literal>TO NODE + </literal>. + </para> + <para> + <replaceable class="parameter">num</replaceable> can contain + a list of nodes like in the query: + +<programlisting> +CLEAN CONNECTION TO COORDINATOR coord1,coord2 FOR DATABASE<replaceable>name</replaceable>; +</programlisting> + to clean connections to Coordinators coord1 and coord2. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect1> + + <refsect1> + <title>Examples</title> + + <para> + Cleaning connection to Datanodes dn1 and dn2 for database template1: +<programlisting> +CLEAN CONNECTION TO NODE dn1,dn2 FOR DATABASE template1; +</programlisting> + </para> + + <para> + Cleaning connection to Datanode dn3 for role postgres: +<programlisting> +CLEAN CONNECTION TO NODE dn3 TO USER postgres; +</programlisting> + </para> + + <para> + Cleaning connection to all nodes on database postgres for + role admin and cut connections to backends. +<programlisting> +CLEAN CONNECTION TO ALL FORCE FOR DATABASE postgres TO USER admin; +</programlisting> + </para> + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>CLEAN CONNECTION</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index e6a77095ec..44f63ba66a 100644 --- a/doc/src/sgml/ref/cluster.sgml +++ b/doc/src/sgml/ref/cluster.sgml @@ -75,6 +75,11 @@ CLUSTER [VERBOSE] database operations (both reads and writes) from operating on the table until the <command>CLUSTER</command> is finished. </para> + + <para> + In <productname>Postgres-XL</>, <command>CLUSTER</> will be + executed on all the Datanodes as well. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/commit_prepared.sgml b/doc/src/sgml/ref/commit_prepared.sgml index e1988ad318..a0433964f8 100644 --- a/doc/src/sgml/ref/commit_prepared.sgml +++ b/doc/src/sgml/ref/commit_prepared.sgml @@ -69,6 +69,17 @@ COMMIT PREPARED <replaceable class="PARAMETER">transaction_id</replaceable> <link linkend="view-pg-prepared-xacts"><structname>pg_prepared_xacts</structname></link> system view. </para> + <para> + If more than one Datanode and/or Coordinator are involved in the + transaction, <command>COMMIT PREPARED</> command will propagate to + all these nodes. + </para> + <para> + If <literal>xc_maintenance_mode</literal> GUC parameter is set to + <literal>ON</literal>, <command>COMMIT PREPARED</command> will not propagate to + other nodes. It just runs locally and report the result to + <literal>GTM</literal>. + </para> </refsect1> <refsect1 id="sql-commit-prepared-examples"> @@ -101,6 +112,7 @@ COMMIT PREPARED 'foobar'; <simplelist type="inline"> <member><xref linkend="sql-prepare-transaction"></member> <member><xref linkend="sql-rollback-prepared"></member> + <member><xref linkend="guc-xc-maintenance-mode"></member> </simplelist> </refsect1> diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgml index 2850b4763f..3b4617618c 100644 --- a/doc/src/sgml/ref/copy.sgml +++ b/doc/src/sgml/ref/copy.sgml @@ -205,6 +205,12 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable have OIDs, or in the case of copying a <replaceable class="parameter">query</replaceable>.) </para> + + <para> + In <productname>Postgres-XL</>, OIDs are only maintained locally + in Coordinators and Datanodes. The value of <literal>OIDs</> + may conflict if you copy this value to another table. + </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/create_aggregate.sgml b/doc/src/sgml/ref/create_aggregate.sgml index eaa410bc94..c8ebcac1c6 100644 --- a/doc/src/sgml/ref/create_aggregate.sgml +++ b/doc/src/sgml/ref/create_aggregate.sgml @@ -24,10 +24,12 @@ PostgreSQL documentation CREATE AGGREGATE <replaceable class="parameter">name</replaceable> ( [ <replaceable class="parameter">argmode</replaceable> ] [ <replaceable class="parameter">argname</replaceable> ] <replaceable class="parameter">arg_data_type</replaceable> [ , ... ] ) ( SFUNC = <replaceable class="PARAMETER">sfunc</replaceable>, STYPE = <replaceable class="PARAMETER">state_data_type</replaceable> + [ , CFUNC = <replaceable class="PARAMETER">cfunc</replaceable> ] [ , SSPACE = <replaceable class="PARAMETER">state_data_size</replaceable> ] [ , FINALFUNC = <replaceable class="PARAMETER">ffunc</replaceable> ] [ , FINALFUNC_EXTRA ] [ , INITCOND = <replaceable class="PARAMETER">initial_condition</replaceable> ] + [ , INITCOLLECT = <replaceable class="PARAMETER">initial_collection_condition</replaceable> ] [ , MSFUNC = <replaceable class="PARAMETER">msfunc</replaceable> ] [ , MINVFUNC = <replaceable class="PARAMETER">minvfunc</replaceable> ] [ , MSTYPE = <replaceable class="PARAMETER">mstate_data_type</replaceable> ] @@ -42,10 +44,12 @@ CREATE AGGREGATE <replaceable class="parameter">name</replaceable> ( [ [ <replac ORDER BY [ <replaceable class="parameter">argmode</replaceable> ] [ <replaceable class="parameter">argname</replaceable> ] <replaceable class="parameter">arg_data_type</replaceable> [ , ... ] ) ( SFUNC = <replaceable class="PARAMETER">sfunc</replaceable>, STYPE = <replaceable class="PARAMETER">state_data_type</replaceable> + [ , CFUNC = <replaceable class="PARAMETER">cfunc</replaceable> ] [ , SSPACE = <replaceable class="PARAMETER">state_data_size</replaceable> ] [ , FINALFUNC = <replaceable class="PARAMETER">ffunc</replaceable> ] [ , FINALFUNC_EXTRA ] [ , INITCOND = <replaceable class="PARAMETER">initial_condition</replaceable> ] + [ , INITCOLLECT = <replaceable class="PARAMETER">initial_collection_condition</replaceable> ] [ , HYPOTHETICAL ] ) @@ -55,10 +59,12 @@ CREATE AGGREGATE <replaceable class="PARAMETER">name</replaceable> ( BASETYPE = <replaceable class="PARAMETER">base_type</replaceable>, SFUNC = <replaceable class="PARAMETER">sfunc</replaceable>, STYPE = <replaceable class="PARAMETER">state_data_type</replaceable> + [ , CFUNC = <replaceable class="PARAMETER">cfunc</replaceable> ] [ , SSPACE = <replaceable class="PARAMETER">state_data_size</replaceable> ] [ , FINALFUNC = <replaceable class="PARAMETER">ffunc</replaceable> ] [ , FINALFUNC_EXTRA ] [ , INITCOND = <replaceable class="PARAMETER">initial_condition</replaceable> ] + [ , INITCOLLECT = <replaceable class="PARAMETER">initial_collection_condition</replaceable> ] [ , MSFUNC = <replaceable class="PARAMETER">msfunc</replaceable> ] [ , MINVFUNC = <replaceable class="PARAMETER">minvfunc</replaceable> ] [ , MSTYPE = <replaceable class="PARAMETER">mstate_data_type</replaceable> ] @@ -101,36 +107,76 @@ CREATE AGGREGATE <replaceable class="PARAMETER">name</replaceable> ( </para> <para> - A simple aggregate function is made from one or two ordinary + A simple aggregate function is made from one to maximum three ordinary functions: a state transition function <replaceable class="PARAMETER">sfunc</replaceable>, + an optional collection function + <replaceable class="PARAMETER">cfunc</replaceable>, and an optional final calculation function <replaceable class="PARAMETER">ffunc</replaceable>. These are used as follows: <programlisting> <replaceable class="PARAMETER">sfunc</replaceable>( internal-state, next-data-values ) ---> next-internal-state +<replaceable class="PARAMETER">cfunc</replaceable>( internal-state, internal-state ) ---> next-internal-state <replaceable class="PARAMETER">ffunc</replaceable>( internal-state ) ---> aggregate-value </programlisting> </para> <para> - <productname>PostgreSQL</productname> creates a temporary variable + In <productname>Postgres-XL</productname> the aggregation works in two + different modes. + <itemizedlist> + <listitem> + <para> + Two phased aggregation - is used when the entire aggregation takes place on + the Coordinator node. In first phase called transition phase, + <productname>Postgres-XL</productname> creates a temporary variable of data type <replaceable class="PARAMETER">stype</replaceable> to hold the current internal state of the aggregate. At each input row, the aggregate argument value(s) are calculated and the state transition function is invoked with the current state value and the new argument value(s) to calculate a new - internal state value. After all the rows have been processed, - the final function is invoked once to calculate the aggregate's return - value. If there is no final function then the ending state value + internal state value. After all the rows have been processed, in the second + phase or finalization phase the final function is invoked once to calculate + the aggregate's return value. If there is no final function then the ending + state value is returned as-is. + </para> + </listitem> + <listitem> + <para> + Three phased aggregation - is used when the process of aggregation is divided + between Coordinator and Datanodes. In this mode, each + <productname>Postgres-XL</productname> Datanode involved in the query carries + out the first phase named transition phase. This phase is similar to the first + phase in the two phased aggregation mode discussed above, except that, every + Datanode applies this phase on the rows available at the Datanode. The + result of transition phase is then transferred to the Coordinator node. + Second phase called collection phase takes place on the Coordinator. + <productname>Postgres-XL</productname> Coordinator node creates a temporary variable + of data type <replaceable class="PARAMETER">stype</replaceable> + to hold the current internal state of the collection phase. For every input + from the Datanode (result of transition phase on that node), the collection + function is invoked with the current collection state value and the new + transition value (obtained from the Datanode) to calculate a new + internal collection state value. After all the transition values from data + nodes have been processed, in the third or finalization phase the final + function is invoked once to calculate the aggregate's return + value. If there is no final function then the ending collection state value is returned as-is. + </para> + </listitem> + </itemizedlist> + Irrespective of the mode used for aggregation, the result of aggregation + should be same if the same set of data rows is participating in the + aggregate. <productname>Postgres-XL</productname> planner chooses the cheapest + feasible mode of the above two, during planning. </para> <para> An aggregate function can provide an initial condition, - that is, an initial value for the internal state value. - This is specified and stored in the database as a value of type + that is, an initial value for the internal transition or collection state + value. This is specified and stored in the database as a value of type <type>text</type>, but it must be a valid external representation of a constant of the state value data type. If it is not supplied then the state value starts out null. @@ -155,10 +201,15 @@ CREATE AGGREGATE <replaceable class="PARAMETER">name</replaceable> ( </para> <para> - If the state transition function is not strict, then it will be called - unconditionally at each input row, and must deal with null inputs - and null state values for itself. This allows the aggregate - author to have full control over the aggregate's handling of null values. + If the collection function is declared <quote>strict</quote>, + then it cannot be called with null inputs. With such a collection + function, aggregate execution behaves as follows. Null state transition + results are ignored (the function is not called and the previous collection + state value is retained). If the initial state value is null, then at the + first non-null state transition result replaces the collection state + value, and the collection function is invoked at subsequent rows with + all-nonnull transition values. + This is handy for implementing aggregates like <function>max</function>. </para> <para> @@ -336,6 +387,26 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; </varlistentry> <varlistentry> + <term><replaceable class="PARAMETER">cfunc</replaceable></term> + <listitem> + <para> + The name of the state collection function to be called for each + input row. The <replaceable class="PARAMETER">sfunc</> must + take <replaceable class="PARAMETER">2 arguments</replaceable>, both of them + being of + type <replaceable class="PARAMETER">state_data_type</replaceable>. + The function must return a value of + type <replaceable class="PARAMETER">state_data_type</replaceable>. + This function takes the current collection state value and the + current transition value, and returns the next collection state + value. If cfunc is omitted for an aggregate, the two phase + aggregation mode is used for that aggregate. All the aggregates + involed in a query use the same aggregation mode. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><replaceable class="PARAMETER">state_data_type</replaceable></term> <listitem> <para> @@ -406,6 +477,21 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; </varlistentry> <varlistentry> + <term><replaceable class="PARAMETER">initial_collection_condition</replaceable></term> + <listitem> + <para> + The initial setting for the state collection value. This must + be a string constant in the form accepted for the data + type <replaceable class="PARAMETER">state_data_type</replaceable>. + If not specified, the collection state value starts out null. If + the collection + function <replaceable class="PARAMETER">cfunc</replaceable> is + not supplied, this parameter is stored but not used. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><replaceable class="PARAMETER">msfunc</replaceable></term> <listitem> <para> diff --git a/doc/src/sgml/ref/create_barrier.sgml b/doc/src/sgml/ref/create_barrier.sgml new file mode 100644 index 0000000000..505cfec9e9 --- /dev/null +++ b/doc/src/sgml/ref/create_barrier.sgml @@ -0,0 +1,55 @@ +<refentry id="SQL-CREATEBARRIER"> + <indexterm zone="sql-createbarrier"> + <primary>CREATE BARRIER</primary> + </indexterm> + + <refmeta> + <refentrytitle>CREATE BARRIER</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>CREATE BARRIER</refname> + <refpurpose>create a new barrier</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +CREATE BARRIER <replaceable class="PARAMETER">barrier_name</replaceable> +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>CREATE BARRIER</command> is new SQL command specific + to <productname>Postgres-XL</productname> that creates + a new XLOG record on each node of the cluster consistently. + Essentially a barrier is a consistent point in the cluster that + you can recover to. Note that these are currently created manually, + not autoatically. Without barriers, if you recover an individual + component, it may be possible that it is not consistent with the + other nodes depending on when it was committed. + </para> + <para>A barrier + is created via a 2PC-like mechanism from a remote Coordinator in 3 + phases with a prepare, execute and ending phases. A new recovery + parameter called recovery_target_barrier has been added in + recovery.conf. In order to perform a complete PITR recovery, it is + necessary to set recovery_target_barrier to the value of a barrier + already created. Then distribute recovery.conf to each data folder + of each node, and then to restart the nodes one by one. + </para> + + <para> + The default barrier name is <literal>dummy_barrier_id</literal>. It is + used when no barrier name is specified when using <command>CREATE + BARRIER</command>. + </para> + + </refsect1> + +</refentry> + diff --git a/doc/src/sgml/ref/create_database.sgml b/doc/src/sgml/ref/create_database.sgml index 9711b1f98e..9db0102083 100644 --- a/doc/src/sgml/ref/create_database.sgml +++ b/doc/src/sgml/ref/create_database.sgml @@ -60,6 +60,13 @@ CREATE DATABASE <replaceable class="PARAMETER">name</replaceable> any installation-local objects that might have been added to <literal>template1</>. </para> + + <para> + If there are any live connections to any one of the template databases in + a Coordinator or Datanode, you will receive an error message. In this + case, you should clean these connections using <command>CLEAN + CONNECTION</> statement. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/create_foreign_data_wrapper.sgml b/doc/src/sgml/ref/create_foreign_data_wrapper.sgml index a3811a3b63..e792dc736b 100644 --- a/doc/src/sgml/ref/create_foreign_data_wrapper.sgml +++ b/doc/src/sgml/ref/create_foreign_data_wrapper.sgml @@ -32,6 +32,10 @@ CREATE FOREIGN DATA WRAPPER <replaceable class="parameter">name</replaceable> <title>Description</title> <para> + <command>FOREIGN DATA WRAPPER</> has not been supported + by <productname>Postgres-XL</> yet. + </para> + <para> <command>CREATE FOREIGN DATA WRAPPER</command> creates a new foreign-data wrapper. The user who defines a foreign-data wrapper becomes its owner. diff --git a/doc/src/sgml/ref/create_function.sgml b/doc/src/sgml/ref/create_function.sgml index c5beb166cf..b65d81542c 100644 --- a/doc/src/sgml/ref/create_function.sgml +++ b/doc/src/sgml/ref/create_function.sgml @@ -43,6 +43,33 @@ CREATE [ OR REPLACE ] FUNCTION <refsect1 id="sql-createfunction-description"> <title>Description</title> + <note> + <para> + <productname>Postgres-XL</> function usage currently requires a lot of care, + otherwise unexepected results may occur, and you may bring your data in an inconsistent state. + This behavior may change in a future version to make it safer. + </para> + + <para> + A call such as <literal>SELECT my_function(1,2);</>, without any FROM clause, + will execute on a local coordinator, and + may involve other datanodes and behave as expected, being driven from a coordinator. + </para> + + <para> + A call such as <literal>SELECT col1, my_table_function(col2) FROM mytable</> will be + pushed down to the datanodes involved. If <literal>my_table_function</> happens to + do a SELECT, it will only be from data local to that node. Similarly, if it + executes an UPDATE, it will only update data local to that node. + If the UPDATE writes to a replicated table for example, it would mean that + the tables would be out of sync. + </para> + + <para> + In summary, one should be very careful in creating and using functions in Postgres-XL. + </para> + </note> + <para> <command>CREATE FUNCTION</command> defines a new function. <command>CREATE OR REPLACE FUNCTION</command> will either create a @@ -508,6 +535,11 @@ CREATE [ OR REPLACE ] FUNCTION reload the file (perhaps during development), start a new session. </para> + <para> + It is user's responsibility to deploy the file to all the + servers where Coordinator or Datanode may run. + </para> + </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml index ce36a1ba48..07b46d5f10 100644 --- a/doc/src/sgml/ref/create_index.sgml +++ b/doc/src/sgml/ref/create_index.sgml @@ -114,6 +114,10 @@ CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] <replaceable class= <varlistentry> <term><literal>CONCURRENTLY</literal></term> <listitem> + <para> + <productname>Postgres-XL</productname> does not + support <literal>CONCURRENTLY</literal> in the current release. + </para> <para> When this option is used, <productname>PostgreSQL</> will build the index without taking any locks that prevent concurrent inserts, diff --git a/doc/src/sgml/ref/create_node.sgml b/doc/src/sgml/ref/create_node.sgml new file mode 100644 index 0000000000..b7c628ac0a --- /dev/null +++ b/doc/src/sgml/ref/create_node.sgml @@ -0,0 +1,192 @@ +<refentry id="SQL-CREATENODE"> + <indexterm zone="sql-createnode"> + <primary>CREATE NODE</primary> + </indexterm> + + <refmeta> + <refentrytitle>CREATE NODE</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>CREATE NODE</refname> + <refpurpose>create a new cluster node</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +CREATE NODE <replaceable class="parameter">nodename</replaceable> WITH + ( + [ TYPE = <replaceable class="parameter">nodetype</replaceable>,] + [ HOST = <replaceable class="parameter">hostname</replaceable>,] + [ PORT = <replaceable class="parameter">portnum</replaceable>,] + [ PRIMARY [ = <replaceable class="parameter">boolean</replaceable> ],] + [ PREFERRED [ = <replaceable class="parameter">boolean</replaceable> ] ] + ) + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>CREATE NODE</command> is a SQL command specific + to <productname>Postgres-XL</productname> that creates + a new entry in catalog table pgxc_node with node data. + </para> + <para> + This node data is directly used by a Coordinator session and its + corresponding Datanode sessions when connecting + to build connection data to cluster nodes through <productname>Postgres-XL + </productname> pooler. + </para> + <para> + Node connection information is created on pooler only if it has not been + the case yet on Coordinator connected at the moment of connection. + </para> + <para> + <command>CREATE NODE</command> only runs on the local node where it is launched. + </para> + + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><replaceable class="parameter">nodename</replaceable></term> + <listitem> + <para> + The name of the selected cluster node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>TYPE</literal></term> + <listitem> + <para> + The type of the cluster node. It is possible to specify + a Coordinator node or a Datanode node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>PRIMARY</literal></term> + <listitem> + <para> + Defines if the cluster node is used as a primary node for replicated + write operations. A <replaceable class="parameter">boolean</replaceable> + value can be specified. In case no value is specified, <literal>PRIMARY</literal> + acts like <literal>false</>. + </para> + <para> + To avoid deadlocks and make update consistent, you should specify the same <literal>PRIMARY</literal> + node at all the nodes. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>PREFERRED</literal></term> + <listitem> + <para> + Defines if the cluster node is used as a preferred node for replicated + read operations if no node is determined. A <replaceable class="parameter">boolean</replaceable> + value can be specified. In case no value is specified, <literal>PREFERRED</literal> + acts like <literal>false</>. + </para> + <para> + You can specify different <literal>PREFERRED</literal> node at different coordinator. + This parameter affects performance of your <literal>Postgres-XL</literal> cluster. + If you configure a datanode where you configure a coordinator, + you should specify <literal>PREFERRED</literal> for the coordinator to such a local datanode. + This will save network communication and improve cluster-wide performance. + </para> + + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">nodetype</replaceable></term> + <listitem> + <para> + The node type for given cluster node. Possible values are: + 'coordinator' for a Coordinator node and 'datanode' for a + Datanode node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">hostname</replaceable></term> + <listitem> + <para> + The hostname or IP used to connect to the cluster node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">portnum</replaceable></term> + <listitem> + <para> + The port number used to connect to the cluster node. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect1> + + <refsect1> + <title>Notes</title> + <para> + <replaceable class="parameter">nodename</replaceable> remains constant + as long as it is in use. + </para> + + <para> + When using a cluster with 1 Coordinator and 1 Datanode on each server, + defining the local Datanode as <literal>PREFERRED</literal> can greatly + improve the performance of a system by avoiding any network overhead for + replicated reads, as in this case a local socket is used for communication + between nodes. This has even more effects when the application frequently uses + replicated tables for remote join operations and that those operations can + be operated on the local <literal>PREFERRED</literal> node. + </para> + </refsect1> + + <refsect1> + <title>Examples</title> + <para> + Create a Coordinator node located on local machine using port 6543 +<programlisting> +CREATE NODE node2 WITH (TYPE = 'coordinator', HOST = 'localhost', PORT = 6543); +</programlisting> + </para> + + <para> + Create a Datanode which is a preferred and primary node + located on remote machine with IP '192.168.0.3' on port 8888. +<programlisting> +CREATE NODE node2 WITH (TYPE = 'datanode', HOST = '192.168.0.3', PORT = 8888, PRIMARY, PREFERRED); +</programlisting> + </para> + + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>CREATE NODE</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> + diff --git a/doc/src/sgml/ref/create_nodegroup.sgml b/doc/src/sgml/ref/create_nodegroup.sgml new file mode 100644 index 0000000000..3b72c11d29 --- /dev/null +++ b/doc/src/sgml/ref/create_nodegroup.sgml @@ -0,0 +1,94 @@ + +<refentry id="SQL-CREATENODEGROUP"> + <indexterm zone="sql-createnodegroup"> + <primary>CREATE NODE GROUP</primary> + </indexterm> + + <refmeta> + <refentrytitle>CREATE NODE GROUP</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>CREATE NODE GROUP</refname> + <refpurpose>create a group of cluster nodes</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +CREATE NODE GROUP <replaceable class="parameter">groupname</replaceable> +WITH ( <replaceable class="parameter">nodename</replaceable> [, ... ] ) + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>CREATE NODE GROUP</command> is a SQL command specific + to <productname>Postgres-XL</productname> that creates + node group information in catalog pgxc_group. + </para> + + <para> + <command>CREATE NODE</command> only runs on the local node where it is launched. + </para> + + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><replaceable class="parameter">groupname</replaceable></term> + <listitem> + <para> + The name of the selected cluster node group. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">nodename</replaceable></term> + <listitem> + <para> + The name of a cluster node. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect1> + + <refsect1> + <title>Notes</title> + <para> + A group of nodes works as an alias for node lists when defining tables + on sub-clusters. Only Datanode can be included in node groups. + </para> + </refsect1> + + <refsect1> + <title>Examples</title> + + <para> + Create a cluster node group made of nodes called Datanode1, Datanode2. +<programlisting> +CREATE NODE GROUP cluster_group WITH Datanode1, Datanode2; +</programlisting> + </para> + + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>CREATE NODE GROUP</command> does not conform to the <acronym> + SQL</acronym> standards, it is a <productname>Postgres-XL</productname> specific command. + </para> + </refsect1> + +</refentry> + diff --git a/doc/src/sgml/ref/create_server.sgml b/doc/src/sgml/ref/create_server.sgml index 734c6c9fe8..2a2f2b6adb 100644 --- a/doc/src/sgml/ref/create_server.sgml +++ b/doc/src/sgml/ref/create_server.sgml @@ -31,6 +31,10 @@ CREATE SERVER <replaceable class="parameter">server_name</replaceable> [ TYPE '< <title>Description</title> <para> + <command>SERVER</> is not yet supported in <productname>Postgres-XL</>. + </para> + + <para> <command>CREATE SERVER</command> defines a new foreign server. The user who defines the server becomes its owner. </para> diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index fac7e1ec5e..34ff649040 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -31,6 +31,8 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI [ WITH ( <replaceable class="PARAMETER">storage_parameter</replaceable> [= <replaceable class="PARAMETER">value</replaceable>] [, ... ] ) | WITH OIDS | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE <replaceable class="PARAMETER">tablespace_name</replaceable> ] +[ DISTRIBUTE BY { REPLICATION | ROUNDROBIN | { [HASH | MODULO ] ( <replaceable class="PARAMETER">column_name</replaceable> ) } } ] +[ TO { GROUP <replaceable class="PARAMETER">groupname</replaceable> | NODE ( <replaceable class="PARAMETER">nodename</replaceable> [, ... ] ) } ] CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="PARAMETER">table_name</replaceable> OF <replaceable class="PARAMETER">type_name</replaceable> [ ( @@ -41,6 +43,8 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI [ WITH ( <replaceable class="PARAMETER">storage_parameter</replaceable> [= <replaceable class="PARAMETER">value</replaceable>] [, ... ] ) | WITH OIDS | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE <replaceable class="PARAMETER">tablespace_name</replaceable> ] +[ DISTRIBUTE BY { REPLICATION | ROUNDROBIN | { [HASH | MODULO ] ( <replaceable class="PARAMETER">column_name</replaceable> ) } } ] +[ TO { GROUP <replaceable class="PARAMETER">groupname</replaceable> | NODE ( <replaceable class="PARAMETER">nodename</replaceable> [, ... ] ) } ] <phrase>where <replaceable class="PARAMETER">column_constraint</replaceable> is:</phrase> @@ -307,6 +311,10 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI </para> <para> + In <productname>Postgres-XL</>, it is currently not possible to distribute a table with more than one parent. + </para> + + <para> Column <literal>STORAGE</> settings are also copied from parent tables. </para> @@ -493,6 +501,12 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI </para> <para> + In <productname>Postgres-XL</>, if <command>DISTRIBUTE BY + REPLICATION</> is not specified, only the distribution key is + allowed to have this constraint. + </para> + + <para> Each unique table constraint must name a set of columns that is different from the set of columns named by any other unique or primary key constraint defined for the table. (Otherwise it @@ -526,6 +540,12 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI different from other sets of columns named by any unique constraint defined for the same table. </para> + + <para> + In <productname>Postgres-XL</>, if <command>DISTRIBUTE BY REPLICATION</> is not specified, the + distribution key must be included in the set of primary key + columns. + </para> </listitem> </varlistentry> @@ -576,6 +596,11 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI exclusion constraint on a subset of the table; internally this creates a partial index. Note that parentheses are required around the predicate. </para> + + <para> + In <productname>Postgres-XL</>, exclusion constraints are currently not + supported. + </para> </listitem> </varlistentry> @@ -788,6 +813,12 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI both an <literal>OIDS</> setting and storage parameters, you must use the <literal>WITH ( ... )</> syntax; see above. </para> + + <para> + In <productname>Postgres-XL</>, OID is managed locally in each + Datanode and Coordinator. The OID value may be inconsistent for + rows stored in different Datanodes. + </para> </listitem> </varlistentry> @@ -849,6 +880,116 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI </varlistentry> <varlistentry> + <term><literal>DISTRIBUTE BY</literal></term> + <listitem> + <para> + This clause specifies how the table is distributed or replicated among Datanodes. + </para> + + <variablelist> + + <varlistentry> + <term><literal>REPLICATION</literal></term> + <listitem> + <para> + Each row of the table will be replicated to all the + Datanode of the <productname>Postgres-XL</> database + cluster. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ROUNDROBIN</literal></term> + <listitem> + <para> + Each row of the table will be placed in one of the Datanodes + in a round-robin manner. The value of the row will not be + needed to determine what Datanode to go. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HASH ( <replaceable class="PARAMETER">column_name</> )</literal></term> + <listitem> + <para> + Each row of the table will be placed based on the hash value + of the specified column. Following type is allowed as + distribution column: INT8, INT2, OID, INT4, BOOL, INT2VECTOR, + OIDVECTOR, CHAR, NAME, TEXT, BPCHAR, BYTEA, VARCHAR, FLOAT4, + FLOAT8, NUMERIC, CASH, ABSTIME, RELTIME, DATE, TIME, + TIMESTAMP, TIMESTAMPTZ, INTERVAL, and TIMETZ. + </para> + <para> + Please note that floating point is not allowed as a basis of + the distribution column. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>MODULO ( <replaceable class="PARAMETER">column_name</> )</literal></term> + <listitem> + <para> + Each row of the table will be placed based on the modulo + of the specified column. Following type is allowed as + distribution column: INT8, INT2, OID, INT4, BOOL, INT2VECTOR, + OIDVECTOR, CHAR, NAME, TEXT, BPCHAR, BYTEA, VARCHAR, FLOAT4, + FLOAT8, NUMERIC, CASH, ABSTIME, RELTIME, DATE, TIME, + TIMESTAMP, TIMESTAMPTZ, INTERVAL, and TIMETZ. + </para> + <para> + Please note that floating point is not allowed as a basis of + the distribution column. + </para> + </listitem> + </varlistentry> + + </variablelist> + <para> + If <literal>DISTRIBUTE BY</> is not specified, columns with + UNIQUE constraint will be chosen as the distribution key. If no + such column is specified, distribution column is the first + eligible column in the definition. If no such column is found, + then the table will be distributed by <literal>ROUNDROBIN</>. + </para> + + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>TO GROUP</literal></term> + <term><literal>TO NODE</literal></term> + <listitem> + <para> + This defines on the list of nodes on which table data exists. + If this is not specified table data is present on all Datanodes. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="PARAMETER">nodename</replaceable></term> + <listitem> + <para> + Associated with <literal>TO NODE</literal>, it defines a <productname> + Postgres-XL</productname> node of catalog pgxc_node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="PARAMETER">groupname</replaceable></term> + <listitem> + <para> + Associated with <literal>TO GROUP</literal>, it defines a <productname> + Postgres-XL</productname> node group in catalog pgxc_group. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><literal>USING INDEX TABLESPACE <replaceable class="PARAMETER">tablespace_name</replaceable></literal></term> <listitem> <para> @@ -1329,7 +1470,7 @@ CREATE TABLE circles ( <programlisting> CREATE TABLE cinemas ( - id serial, + id integer, name text, location text ) TABLESPACE diskvol1; @@ -1516,6 +1657,30 @@ CREATE TABLE employees OF employee_type ( effect can be had using the OID feature. </para> </refsect2> + <refsect2> + <title><productname>Postgres-XL</> Specifics</title> + + <para> + Currently, non-immutable functions are not allowed + as <literal>DEFAULT</> values. + </para> + <para> + <literal>PRIMARY KEY</> and foreign key must include the + distribution column. + </para> + <para> + <literal>TEMP</> tables and exclusion constraint are not supported + yet. + </para> + <para> + </para> + <para> + In <productname>Postgres-XL</>, OID is maintained locally in each + Datanode and Coordinator. The OID value may be inconsistent for rows + stored in different Datanodes. + </para> + + </refsect2> </refsect1> diff --git a/doc/src/sgml/ref/create_table_as.sgml b/doc/src/sgml/ref/create_table_as.sgml index 8e4ada794d..b59156bd74 100644 --- a/doc/src/sgml/ref/create_table_as.sgml +++ b/doc/src/sgml/ref/create_table_as.sgml @@ -26,6 +26,8 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI [ WITH ( <replaceable class="PARAMETER">storage_parameter</replaceable> [= <replaceable class="PARAMETER">value</replaceable>] [, ... ] ) | WITH OIDS | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE <replaceable class="PARAMETER">tablespace_name</replaceable> ] + [ DISTRIBUTE BY { REPLICATION | ROUNDROBIN | { [HASH | MODULO ] ( <replaceable class="PARAMETER">column_name</replaceable> ) } } ] + [ TO { GROUP <replaceable class="PARAMETER">groupname</replaceable> | NODE ( <replaceable class="PARAMETER">nodename</replaceable> [, ... ] ) } ] AS <replaceable>query</replaceable> [ WITH [ NO ] DATA ] </synopsis> @@ -208,6 +210,116 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI </varlistentry> <varlistentry> + <term><literal>DISTRIBUTE BY</literal></term> + <listitem> + <para> + This clause specifies how the table is distributed or replicated among Datanodes. + </para> + + <variablelist> + + <varlistentry> + <term><literal>REPLICATION</literal></term> + <listitem> + <para> + Each row of the table will be replicated into all the + Datanode of the <productname>Postgres-XL</> database + cluster. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ROUNDROBIN</literal></term> + <listitem> + <para> + Each row of the table will be placed in one of the Datanodes + by round-robin manner. The value of the row will not be + needed to determine what Datanode to go. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HASH ( <replaceable class="PARAMETER">column_name</> )</literal></term> + <listitem> + <para> + Each row of the table will be placed based on the hash value + of the specified column. Following type is allowed as + distribution column: INT8, INT2, OID, INT4, BOOL, INT2VECTOR, + OIDVECTOR, CHAR, NAME, TEXT, BPCHAR, BYTEA, VARCHAR, FLOAT4, + FLOAT8, NUMERIC, CASH, ABSTIME, RELTIME, DATE, TIME, + TIMESTAMP, TIMESTAMPTZ, INTERVAL, and TIMETZ. + </para> + <para> + Please note that floating point is not allowed as a basis of + the distribution column. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>MODULO ( <replaceable class="PARAMETER">column_name</> )</literal></term> + <listitem> + <para> + Each row of the table will be placed based on the modulo + of the specified column. Following type is allowed as + distribution column: INT8, INT2, OID, INT4, BOOL, INT2VECTOR, + OIDVECTOR, CHAR, NAME, TEXT, BPCHAR, BYTEA, VARCHAR, FLOAT4, + FLOAT8, NUMERIC, CASH, ABSTIME, RELTIME, DATE, TIME, + TIMESTAMP, TIMESTAMPTZ, INTERVAL, and TIMETZ. + </para> + <para> + Please note that floating point is not allowed as a basis of + the distribution column. + </para> + </listitem> + </varlistentry> + + </variablelist> + <para> + If <literal>DISTRIBUTE BY</> is not specified, columns with + UNIQUE constraint will be chosen as the distribution key. If no + such column is specified, distribution column is the first + eligible column in the definition. If no such column is found, + then the table will be distributed by <literal>ROUNDROBIN</>. + </para> + + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>TO GROUP</literal></term> + <term><literal>TO NODE</literal></term> + <listitem> + <para> + This defines on the list of nodes on which table data exists. + If this is not specified table data is present on all Datanodes. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="PARAMETER">nodename</replaceable></term> + <listitem> + <para> + Associated with <literal>TO NODE</literal>, it defines a <productname> + Postgres-XL</productname> node of catalog pgxc_node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="PARAMETER">groupname</replaceable></term> + <listitem> + <para> + Associated with <literal>TO GROUP</literal>, it defines a <productname> + Postgres-XL</productname> node group in catalog pgxc_group. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><replaceable>query</replaceable></term> <listitem> <para> diff --git a/doc/src/sgml/ref/create_tablespace.sgml b/doc/src/sgml/ref/create_tablespace.sgml index 5756c3e080..ecd4fd3e66 100644 --- a/doc/src/sgml/ref/create_tablespace.sgml +++ b/doc/src/sgml/ref/create_tablespace.sgml @@ -130,6 +130,20 @@ CREATE TABLESPACE <replaceable class="parameter">tablespace_name</replaceable> <command>CREATE TABLESPACE</> cannot be executed inside a transaction block. </para> + + <para> + <productname>Postgres-XL</> assigns the same path to a tablespace + for all the Coordinators and Datanodes. So when creating a tablespace, + user needs to have permission to the same location path on all the servers + involved in the cluster. + </para> + + <para> + <command>CREATE TABLESPACE</> can be run with EXECUTE DIRECT to control + tablespace existence on remote nodes. This works for both remote Coordinators + and Datanodes but not on this operation is not authorized on node where + application client is connected. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/create_trigger.sgml b/doc/src/sgml/ref/create_trigger.sgml index 4bde815012..726d9ae8e8 100644 --- a/doc/src/sgml/ref/create_trigger.sgml +++ b/doc/src/sgml/ref/create_trigger.sgml @@ -42,6 +42,11 @@ CREATE [ CONSTRAINT ] TRIGGER <replaceable class="PARAMETER">name</replaceable> <title>Description</title> <para> + The <command>CREATE TRIGGER</> statement is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>CREATE TRIGGER</command> creates a new trigger. The trigger will be associated with the specified table, view, or foreign table and will execute the specified diff --git a/doc/src/sgml/ref/create_user_mapping.sgml b/doc/src/sgml/ref/create_user_mapping.sgml index bb0c9c0b03..91b6f7954f 100644 --- a/doc/src/sgml/ref/create_user_mapping.sgml +++ b/doc/src/sgml/ref/create_user_mapping.sgml @@ -31,6 +31,11 @@ CREATE USER MAPPING FOR { <replaceable class="parameter">user_name</replaceable> <title>Description</title> <para> + <command>USER MAPPING</> is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>CREATE USER MAPPING</command> defines a mapping of a user to a foreign server. A user mapping typically encapsulates connection information that a foreign-data wrapper uses together diff --git a/doc/src/sgml/ref/create_view.sgml b/doc/src/sgml/ref/create_view.sgml index 8fa3564021..c422cfd295 100644 --- a/doc/src/sgml/ref/create_view.sgml +++ b/doc/src/sgml/ref/create_view.sgml @@ -65,6 +65,10 @@ CREATE [ OR REPLACE ] [ TEMP | TEMPORARY ] [ RECURSIVE ] VIEW <replaceable class <term><literal>TEMPORARY</> or <literal>TEMP</></term> <listitem> <para> + <literal>TEMPORARY</> views are not yet supported + in <productname>Postgres-XL</>. + </para> + <para> If specified, the view is created as a temporary view. Temporary views are automatically dropped at the end of the current session. Existing diff --git a/doc/src/sgml/ref/declare.sgml b/doc/src/sgml/ref/declare.sgml index 5cb85cc568..0b89cc7799 100644 --- a/doc/src/sgml/ref/declare.sgml +++ b/doc/src/sgml/ref/declare.sgml @@ -78,6 +78,9 @@ DECLARE <replaceable class="parameter">name</replaceable> [ BINARY ] [ INSENSITI <term><literal>INSENSITIVE</literal></term> <listitem> <para> + In <productname>Postgres-XL</>, <literal>INSENSITIVE</literal> cursor is currently not supported. + </para> + <para> Indicates that data retrieved from the cursor should be unaffected by updates to the table(s) underlying the cursor that occur after the cursor is created. In <productname>PostgreSQL</productname>, @@ -91,6 +94,9 @@ DECLARE <replaceable class="parameter">name</replaceable> [ BINARY ] [ INSENSITI <term><literal>SCROLL</literal></term> <term><literal>NO SCROLL</literal></term> <listitem> + <para> + In <productname>Postgres-XL</>, <literal>SCROLL</literal> cursor is currently not supported. + </para> <para><literal>SCROLL</literal> specifies that the cursor can be used to retrieve rows in a nonsequential fashion (e.g., backward). Depending upon the complexity of the query's @@ -109,6 +115,9 @@ DECLARE <replaceable class="parameter">name</replaceable> [ BINARY ] [ INSENSITI <term><literal>WITH HOLD</literal></term> <term><literal>WITHOUT HOLD</literal></term> <listitem> + <para> + In <productname>Postgres-XL</>, <literal>WITH HOLD</literal> cursor is currently not supported. + </para> <para><literal>WITH HOLD</literal> specifies that the cursor can continue to be used after the transaction that created it successfully commits. <literal>WITHOUT HOLD</literal> specifies diff --git a/doc/src/sgml/ref/delete.sgml b/doc/src/sgml/ref/delete.sgml index 74ea90787b..ff0431d6e1 100644 --- a/doc/src/sgml/ref/delete.sgml +++ b/doc/src/sgml/ref/delete.sgml @@ -81,6 +81,10 @@ DELETE FROM [ ONLY ] <replaceable class="PARAMETER">table_name</replaceable> [ * <term><replaceable class="parameter">with_query</replaceable></term> <listitem> <para> + <replaceable class="parameter">with_query</replaceable> is not + supported by current release of <productname>Postgres-XL</>. + </para> + <para> The <literal>WITH</literal> clause allows you to specify one or more subqueries that can be referenced by name in the <command>DELETE</> query. See <xref linkend="queries-with"> and <xref linkend="sql-select"> @@ -157,6 +161,12 @@ DELETE FROM [ ONLY ] <replaceable class="PARAMETER">table_name</replaceable> [ * <xref linkend="sql-declare"> for more information about using cursors with <literal>WHERE CURRENT OF</>. + <footnote> + <para> + <literal>WHERE CURRENT OF</literal> is not supported in the + current version of <productname>Postgres-XL</productname>. + </para> + </footnote> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/drop_database.sgml b/doc/src/sgml/ref/drop_database.sgml index 740aa31995..93cd955c27 100644 --- a/doc/src/sgml/ref/drop_database.sgml +++ b/doc/src/sgml/ref/drop_database.sgml @@ -40,6 +40,12 @@ DROP DATABASE [ IF EXISTS ] <replaceable class="PARAMETER">name</replaceable> <para> <command>DROP DATABASE</command> cannot be undone. Use it with care! </para> + <para> + If there are any live connections to any of the template databases in + a Coordinator or Datanode, you will receive an error message. In this + case, you should clean these connections using <command>CLEAN + CONNECITON</> statement. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/drop_foreign_data_wrapper.sgml b/doc/src/sgml/ref/drop_foreign_data_wrapper.sgml index e43e0bda8b..0ba8773d9a 100644 --- a/doc/src/sgml/ref/drop_foreign_data_wrapper.sgml +++ b/doc/src/sgml/ref/drop_foreign_data_wrapper.sgml @@ -29,6 +29,12 @@ DROP FOREIGN DATA WRAPPER [ IF EXISTS ] <replaceable class="parameter">name</rep <title>Description</title> <para> + <command>FOREIGN DATA WRAPPER</> is not yet supported + in <productname>Postgres-XL</>. This command may be supported + in the future releases. + </para> + + <para> <command>DROP FOREIGN DATA WRAPPER</command> removes an existing foreign-data wrapper. To execute this command, the current user must be the owner of the foreign-data wrapper. diff --git a/doc/src/sgml/ref/drop_node.sgml b/doc/src/sgml/ref/drop_node.sgml new file mode 100644 index 0000000000..a689280e80 --- /dev/null +++ b/doc/src/sgml/ref/drop_node.sgml @@ -0,0 +1,80 @@ + +<refentry id="SQL-DROPNODE"> + <indexterm zone="sql-dropnode"> + <primary>DROP NODE</primary> + </indexterm> + + <refmeta> + <refentrytitle>DROP NODE</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>DROP NODE</refname> + <refpurpose>drop a cluster node</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +DROP NODE <replaceable class="parameter">nodename</replaceable> + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>DROP NODE</command> is new SQL command specific + to <productname>Postgres-XL</productname> that deletes + cluster node information in catalog pgxc_node. + </para> + <para> + Node connection that has been deleted does not guarantee that connection + information cached in pooler is updated accordingly. + </para> + <para> + <command>DROP NODE</command> only runs on the local node where it is launched. + </para> + + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><replaceable class="parameter">nodename</replaceable></term> + <listitem> + <para> + The name of the selected cluster node. + </para> + </listitem> + </varlistentry> + </variablelist> + + </refsect1> + + <refsect1> + <title>Examples</title> + + <para> + Drop a cluster node. +<programlisting> +DROP NODE cluster_node; +</programlisting> + </para> + + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>DROP NODE</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> + diff --git a/doc/src/sgml/ref/drop_nodegroup.sgml b/doc/src/sgml/ref/drop_nodegroup.sgml new file mode 100644 index 0000000000..155b185206 --- /dev/null +++ b/doc/src/sgml/ref/drop_nodegroup.sgml @@ -0,0 +1,76 @@ +<refentry id="SQL-DROPNODEGROUP"> + <indexterm zone="sql-dropnodegroup"> + <primary>DROP NODE GROUP</primary> + </indexterm> + + <refmeta> + <refentrytitle>DROP NODE GROUP</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>DROP NODE GROUP</refname> + <refpurpose>drop a group of cluster nodes</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +DROP NODE GROUP <replaceable class="parameter">groupname</replaceable> + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>DROP NODE GROUP</command> is SQL command specific + to <productname>Postgres-XL</productname> that deletes + node group information in catalog pgxc_group. + </para> + <para> + A group of nodes works as an alias for node lists when defining tables + on sub-clusters. + </para> + + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><replaceable class="parameter">groupname</replaceable></term> + <listitem> + <para> + The name of the selected cluster node group. + </para> + </listitem> + </varlistentry> + </variablelist> + + </refsect1> + + <refsect1> + <title>Examples</title> + + <para> + Drop a cluster node group. +<programlisting> +DROP NODE GROUP cluster_group; +</programlisting> + </para> + + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>DROP NODE GROUP</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> + diff --git a/doc/src/sgml/ref/drop_server.sgml b/doc/src/sgml/ref/drop_server.sgml index 497d83fb4a..0e9bd4c4cd 100644 --- a/doc/src/sgml/ref/drop_server.sgml +++ b/doc/src/sgml/ref/drop_server.sgml @@ -29,6 +29,12 @@ DROP SERVER [ IF EXISTS ] <replaceable class="parameter">name</replaceable> [ CA <title>Description</title> <para> + <command>SERVER</> is not yet supported + in <productname>Postgres-XL</>. This command may be supported + in the future releases. + </para> + + <para> <command>DROP SERVER</command> removes an existing foreign server descriptor. To execute this command, the current user must be the owner of the server. diff --git a/doc/src/sgml/ref/drop_trigger.sgml b/doc/src/sgml/ref/drop_trigger.sgml index 2067aefca2..8c5ba20832 100644 --- a/doc/src/sgml/ref/drop_trigger.sgml +++ b/doc/src/sgml/ref/drop_trigger.sgml @@ -29,6 +29,11 @@ DROP TRIGGER [ IF EXISTS ] <replaceable class="PARAMETER">name</replaceable> ON <title>Description</title> <para> + The <command>DROP TRIGGER</> statement is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>DROP TRIGGER</command> removes an existing trigger definition. To execute this command, the current user must be the owner of the table for which the trigger is defined. diff --git a/doc/src/sgml/ref/drop_user_mapping.sgml b/doc/src/sgml/ref/drop_user_mapping.sgml index ddfad0bcad..756f791350 100644 --- a/doc/src/sgml/ref/drop_user_mapping.sgml +++ b/doc/src/sgml/ref/drop_user_mapping.sgml @@ -29,6 +29,12 @@ DROP USER MAPPING [ IF EXISTS ] FOR { <replaceable class="parameter">user_name</ <title>Description</title> <para> + <command>USER MAPPING</> has not been supported + by <productname>Postgres-XL</> yet. This command may be supported + in the future releases. + </para> + + <para> <command>DROP USER MAPPING</command> removes an existing user mapping from foreign server. </para> diff --git a/doc/src/sgml/ref/execute_direct.sgml b/doc/src/sgml/ref/execute_direct.sgml new file mode 100644 index 0000000000..cc97b4052e --- /dev/null +++ b/doc/src/sgml/ref/execute_direct.sgml @@ -0,0 +1,116 @@ + +<refentry id="SQL-EXECUTEDIRECT"> + <indexterm zone="sql-executedirect"> + <primary>EXECUTE DIRECT</primary> + </indexterm> + + <refmeta> + <refentrytitle>EXECUTE DIRECT</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>EXECUTE DIRECT</refname> + <refpurpose>Launch queries directly to dedicated nodes</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +EXECUTE DIRECT ON ( <replaceable class="parameter">nodename</replaceable> [, ... ] ) + <replaceable class="parameter">query</replaceable> +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>EXECUTE DIRECT</command> is a SQL command specially created + for <productname>Postgres-XL</productname> to allow to launch queries directly to dedicated + nodes determined by a list of nodes <replaceable class="parameter"> + nodelist</replaceable>. + </para> + + <para> + EXECUTE DIRECT is currenlty limited to be used on 1 node at a time + only. + The usage of transaction queries + (<literal>BEGIN</literal>, <literal>COMMIT</literal>...), DDL, and DML + (<literal>INSERT</literal>, <literal>UPDATE</literal>, <literal>DELETE + </literal>) is forbidden to avoid data inconsistency among nodes + in the cluster. EXECUTE DIRECT usage is also limited to superusers. + </para> + + <para> + Either a Coordinator or Datanode can be selected by its node name. + </para> + + <para> + <command>EXECUTE DIRECT</> allows the usage of the following utility commands + on remote nodes: <command>CREATE TABLESPACE</>, <command>DROP TABLESPACE</>. + </para> + + </refsect1> + + <refsect1> + <title>Parameters</title> + + <variablelist> + <varlistentry> + <term><replaceable class="parameter">nodename</replaceable></term> + <listitem> + <para> + This mandatory clause specifies the node name on where to launch + <replaceable class="parameter">query</replaceable>. When specifying + multiple nodes, node names have to be separated by a comma. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><replaceable class="parameter">query</replaceable></term> + <listitem> + <para> + This mandatory clause specifies the raw query to launch + on specified node list <replaceable class="parameter">nodelist</replaceable>. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect1> + + <refsect1> + <title>Examples</title> + + <para> + Select some data in a given table tenk1 on remote Datanode named dn1: +<programlisting> +EXECUTE DIRECT ON NODE dn1 'SELECT * FROM tenk1 WHERE col_char = ''foo'''; +</programlisting> + </para> + + <para> + Select local timestamp of a remote node named coord2: +<programlisting> +EXECUTE DIRECT ON coord2 'select clock_timestamp()'; +</programlisting> + </para> + + <para> + Select list of tables of a remote node named dn50: +<programlisting> +EXECUTE DIRECT ON dn50 'select tablename from pg_tables'; +</programlisting> + </para> + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>EXECUTE DIRECT</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml index f14a58dfc6..eb48b9614f 100644 --- a/doc/src/sgml/ref/explain.sgml +++ b/doc/src/sgml/ref/explain.sgml @@ -82,6 +82,12 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac are close to reality. </para> + <para> + In <productname>Postgres-XL</productname>, EXPLAIN provides the cluster-wide + execution plan that will be executed, and provides insight into the inner + workings of how queries are processed in parallel. + </para> + <important> <para> Keep in mind that the statement is actually executed when @@ -180,6 +186,28 @@ ROLLBACK; </varlistentry> <varlistentry> + <term><literal>NODES</literal></term> + <listitem> + <para> + Include information on the Datanodes involved in the execution of Data + Scan Node. This parameter defaults to <literal>TRUE</literal>. This option + is available in <productname>Postgres-XL</productname>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>NUM_NODES</literal></term> + <listitem> + <para> + Include information on the number of nodes involved in the execution of + Data Node Scan node. This parameter defaults to <literal>FALSE</literal>. + This option is available in <productname>Postgres-XL</productname>. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><literal>TIMING</literal></term> <listitem> <para> diff --git a/doc/src/sgml/ref/fetch.sgml b/doc/src/sgml/ref/fetch.sgml index 24c8c49156..47b89f11ce 100644 --- a/doc/src/sgml/ref/fetch.sgml +++ b/doc/src/sgml/ref/fetch.sgml @@ -78,6 +78,12 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < </para> <para> + The forms <literal>PRIOR</>, <literal>FIRST</>, <literal>LAST</> + and <literal>ABSOLUTE</> are not supported by the current release + of <productname>Postgres-XL</> + </para> + + <para> The forms using <literal>FORWARD</> and <literal>BACKWARD</> retrieve the indicated number of rows moving in the forward or backward direction, leaving the cursor positioned on the @@ -87,6 +93,11 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < </para> <para> + The form using <literal>BACKWARD</> is not supported in the current + release of <productname>Postgres-XL</>. + </para> + + <para> <literal>RELATIVE 0</>, <literal>FORWARD 0</>, and <literal>BACKWARD 0</> all request fetching the current row without moving the cursor, that is, re-fetching the most recently fetched @@ -94,6 +105,11 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < first row or after the last row; in which case, no row is returned. </para> + <para> + The form using <literal>BACKWARD</> is not supported in the current + release of <productname>Postgres-XL</>. + </para> + <note> <para> This page describes usage of cursors at the SQL command level. @@ -132,6 +148,23 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < <para> Fetch the prior row. </para> + <para> + <literal>PRIOR</literal> is not supported by the current + release of <productname>Postgres-XL</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>FIRST</literal></term> + <listitem> + <para> + Fetch the first row of the query (same as <literal>ABSOLUTE 1</literal>). + </para> + <para> + <literal>FIRST</literal> is not supported by the current + release of <productname>Postgres-XL</>. + </para> </listitem> </varlistentry> @@ -150,6 +183,10 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < <para> Fetch the last row of the query (same as <literal>ABSOLUTE -1</literal>). </para> + <para> + <literal>LAST</literal> is not supported by the current + release of <productname>Postgres-XC</>. + </para> </listitem> </varlistentry> @@ -168,6 +205,10 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < particular, <literal>ABSOLUTE 0</literal> positions before the first row. </para> + <para> + <literal>ABSOLUTE</literal> is not supported by the current + release of <productname>Postgres-XL</>. + </para> </listitem> </varlistentry> @@ -242,6 +283,10 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < <para> Fetch the prior row (same as <literal>PRIOR</literal>). </para> + <para> + <literal>BACKWARD</literal> is not supported by the current + release of <productname>Postgres-XL</>. + </para> </listitem> </varlistentry> @@ -254,6 +299,10 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < backwards). <literal>BACKWARD 0</literal> re-fetches the current row. </para> + <para> + <literal>BACKWARD <replaceable class="PARAMETER">count</replaceable></literal> is not supported by the current + release of <productname>Postgres-XL</>. + </para> </listitem> </varlistentry> @@ -263,6 +312,10 @@ FETCH [ <replaceable class="PARAMETER">direction</replaceable> [ FROM | IN ] ] < <para> Fetch all prior rows (scanning backwards). </para> + <para> + <literal>BACKWARD ALL</literal> is not supported by the current + release of <productname>Postgres-XL</>. + </para> </listitem> </varlistentry> </variablelist></para> diff --git a/doc/src/sgml/ref/gtm.sgml b/doc/src/sgml/ref/gtm.sgml new file mode 100644 index 0000000000..2e33f00c55 --- /dev/null +++ b/doc/src/sgml/ref/gtm.sgml @@ -0,0 +1,416 @@ + +<refentry id="APP-GTM"> + <indexterm zone="app-gtm"> + <primary>gtm</primary> + </indexterm> + + <refmeta> + <refentrytitle>gtm</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo>Application</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>gtm</refname> + <refpurpose> + provides global transaction management for the <productname>Postgres-XL</productname> + across the entire cluster. + </refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>gtm</command> + <arg rep="repeat"><replaceable>option</></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1 id="R1-APP-GTM-1"> + <title> + Description + </title> + <para> + gtm provides consistent transaction management fully compatible with + vanilla PostgreSQL. It is highly advised to start and stop gtm + using gtm_ctl(8). + </para> + + <para> + You must provide a gtm configuration + file <filename>gtm.conf</filename> placed in the gtm working directory + as specified by <literal>-D</literal> command line option. The + configuration file specifies gtm running environment and resources. + </para> + + <para> + Some of the parameters specified in the control file can be overridden by + command line options. + </para> + + <para> + As with other cluster components, the recommended method of + configuring and managing them is with the <xref linkend="pgxc-ctl"> utility. + </para> + + </refsect1> + + <refsect1> + <title>Configuration File</title> + + <para> + <literal>GTM</literal> configuration parameters are specified in the configuration file + <filename>gtm.conf</filename><indexterm><primary>gtm.conf</></> placed in the working directory + specified as <literal>-D</literal> option + of <application>gtm</application> command line option as described + in the next section. + </para> + <para> + The format of the configuration file is patterned after the <filename>postgresql.conf</filename>. + The options are as follows. + </para> + +<!-- Add description of each gtm.conf entry --> +<!-- Notice + The following options are found in the source code (gtm_opt.c) but is only for + internal use and will not be presented in the following list. + + 1. data_di + 2. config_file + + Also the following options are for high-availability hook. This will be + documented later. + + 1. error_reporter + 2. status_reader +--> + <variablelist> + + <varlistentry id="gtm-opt-active-host" xreflabel="gtm_opt_active_host"> + <term><varname>active_host</varname> (<type>string</type>) + <indexterm> + <primary><varname>active_host</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the listen addresses (host name or IP address) of active <application>gtm</application>. This + parameter is effective only when this <application>gtm</application> + is specified as a standby. There is no default value for this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-active-port" xreflabel="gtm_opt_active_port"> + <term><varname>active_port</varname> (<type>integer</type>) + <indexterm> + <primary><varname>active_port</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the port number of active <application>gtm</application>. This + parameter is effective only when this <application>gtm</application> + is specified as a GTM standby. The standby connects to the primary gtm + to obtain a stream of updates of transaction status values. + There is no default value for this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-keepalives-count" xreflabel="gtm_opt_keepalives_count"> + <term><varname>keepalives_count</varname> (<type>integer</type>) + <indexterm> + <primary><varname>keepalives_count</varname> (<type>integer</type>)</primary> + </indexterm></term> + <listitem> + <para> + Specifies <literal>keepalives_count</literal> option for the + connection to <application>gtm</application>. This option is + effective only when it runs as GTM Standby. + The default value is zero and keepalives feature is disabled. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-keepalives-idle" xreflabel="gtm_opt_keepalives_idle"> + <term><varname>keepalives_idle</varname> (<type>integer</type>) + <indexterm> + <primary><varname>keepalives_idle</varname> (<type>integer</type>)</primary> + </indexterm></term> + <listitem> + <para> + Specifies <literal>keepalives_idle</literal> option for the + connection to <application>gtm</application>. This option is + effective only when it runs as GTM Standby. + The default value is zero and keepalives feature is disabled. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-keepalives-interval" xreflabel="gtm_opt_keepalives_interval"> + <term><varname>keepalives_interval</varname> (<type>integer</type>) + <indexterm> + <primary><varname>keepalives_interval</varname> (<type>integer</type>)</primary> + </indexterm></term> + <listitem> + <para> + Specifies <literal>keepalives_interval</literal> option for the + connection to <application>gtm</application>. This option is + effective only when it runs as GTM Standby. + The default value is zero and keepalives feature is disabled. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-listen-addresses" xreflabel="gtm_opt_listen_addresses"> + <term><varname>listen_addresses</varname> (<type>string</type>) + <indexterm> + <primary><varname>listen_addresses</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Specifies listen addresses (host name or IP address) of + this <application>gtm</application> + or <application>gtm</application> standby. + Default value is '*'. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-log-file" xreflabel="gtm_opt_log_file"> + <term><varname>log_file</varname> (<type>string</type>) + <indexterm> + <primary><varname>log_file</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Specifies <filename>log</filename> file name. This file will be + created in the working directory of + this <application>gtm</application> as specified + by <literal>-D</literal> command line option. + The default is <filename>gtm.log</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-log-min-messages" xreflabel="gtm_opt_log_min_messages"> + <term><varname>log_min_messages</varname> (<type>enum</type>) + <indexterm> + <primary><varname>log_min_messages</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Controls which message levels are written to the log. + Valid values are <literal>DEBUG</literal>, <literal>DEBUG5</literal>, + <literal>DEBUG4</literal>, <literal>DEBUG3</literal>, <literal>DEBUG2</literal>, + <literal>DEBUG1</literal>, <literal>INFO</literal>, <literal>NOTICE</literal>, + <literal>WARNING</literal>, <literal>ERROR</literal>, <literal>LOG</literal>, + <literal>FATAL</literal> and <literal>PANIC</literal>. + Each level includes all the levels that follow it. + The later the level, the fewer messages are sent. + The default is <literal>WARNING</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-nodename" xreflabel="gtm_opt_nodename"> + <term><varname>nodename</varname> (<type>string</type>) + <indexterm> + <primary><varname>nodename</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Specifies the name of this <application>gtm</application> or <application>gtm</application> standby. + There is no default value for this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-port" xreflabel="gtm_opt_port"> + <term><varname>port</varname> (<type>integer</type>) + <indexterm> + <primary><varname>port</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Specifies the port number of <application>gtm</application> or <application>gtm</application> standby. + Default value is 6666. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-startup" xreflabel="gtm_opt_startup"> + <term><varname>startup</varname> (<type>enum</type>) + <indexterm> + <primary><varname>startup</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Specifies the startup mode of this <application>gtm</application>. + Valied values are <literal>act</literal> or <literal>standby</literal>. + <literal>act</literal> means to start up + this <application>gtm</application> as usual so + that <application>gtm</application> clients (Coordinators, data + nodes or gtm-proxies) can connect for transaction + management. <literal>standby</literal> means + this <application>gtm</application> starts up as a backup + of <literal>act</literal> + gtm. <literal>standby</literal> <literal>gtm</literal> can be + promoted to <literal>act</literal> when <literal>act</literal> + fails. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-opt-synchronous-backup" xreflabel="gtm_opt_synchronous_backup"> + <term><varname>synchronous-backup</varname> (<type>boolean</type>) + <indexterm> + <primary><varname>synchronous-backup</varname> configuration parameter</primary> + </indexterm></term> + <listitem> + <para> + Specifies if the backup to GTM-Standby is taken synchronously. If this is turned on, + GTM will send and receive synchronize message to make sure that all the backups + reached to the standby. + </para> + <para> + If it is turned off, all the backup information will be sent without checking they + reached to GTM-Standby. + </para> + <para> + Default value is off. + </para> + </listitem> + </varlistentry> + + + </variablelist> + + </refsect1> + + + <refsect1> + <title>Options</title> + + <para> + Options are specified with preceding '<literal>-</literal>', each + option may be associated with a value. They can be specified + in <literal>-o</literal> option of gtm_ctl(8). + </para> + + <para> + Parameters specified as command line options override these specified in the configuration file described + in the previous section. + </para> + + <para> + Options are as follows: + </para> + + <para> + <variablelist> + <varlistentry> + <term><option>D</option></term> + <listitem> + <para> + Specify a directory which holds data for gtm or gtm_proxy + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>l</option></term> + <listitem> + <para> + Specify a log file for gtm_ctl. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>x</option></term> + <listitem> + <para> + Specify the global transaction ID to start with. This is needed + when you start gtm for the first time + because <command>initdb</command> consumes XID locally and gtm + should start to give GXID greater than the last one + each <command>initdb</command> consumed locally. If gtm has + been shut down gracefully, then this value will be taken from + the last run. + </para> + <para> + If <literal>-x</literal> option is not specified at the first + run, initial global transaction ID value will be set to a + default initial value which is considered to be safe enough (in + version &version;, it is 10000). If many statements are + involved in initdb, you should consider to increasing this + value. + </para> + <para> + To find the precise value to start with, you should + run <application>pg_controldata</application> to + find <literal>Latest checkpoint's NextXID</literal> of all the + Coordinators and Datanodes and choose the value larger than or + equals to the maximum value found. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>h</option></term> + <listitem> + <para> + Specify host name or IP address used. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>p</option></term> + <listitem> + <para> + Specify port number to listen. + </para> + </listitem> + </varlistentry> + + </variablelist> + </para> + + <para> + When starting GTM as a standby instance, the following options can also be provided. + </para> + + <para> + <variablelist> + <varlistentry> + <term><option>s</option></term> + <listitem> + <para> + Specify if GTM starts as a standby + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>i</option></term> + <listitem> + <para> + Specify host name or IP address of active GTM instance to connect to + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>q</option></term> + <listitem> + <para> + Specify port number of active GTM instance to connect to + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + </refsect1> +</refentry> diff --git a/doc/src/sgml/ref/gtm_ctl.sgml b/doc/src/sgml/ref/gtm_ctl.sgml new file mode 100644 index 0000000000..e78d62e21d --- /dev/null +++ b/doc/src/sgml/ref/gtm_ctl.sgml @@ -0,0 +1,235 @@ +<refentry id="APP-GTM-CTL"> + <indexterm zone="app-gtm-ctl"> + <primary>gtm_proxy</primary> + </indexterm> + + <refmeta> + <refentrytitle>gtm_ctl</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo>Application</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>gtm_ctl</refname> + <refpurpose> + <productname>Postgres-XL</productname> GTM operation module + </refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>gtm_proxy</command> + <arg rep="repeat"><replaceable>option</></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1 id="R1-APP-GTM-CTL-1"> + <title> + Description + </title> + <para> + <command>gtm_ctl</command> starts/stops <command>gtm</command> + or <command>gtm_proxy</command>. The options specify the GTM + configuration. + </para> + + </refsect1> + + <refsect1> + <title>Options</title> + + <para> + Options are specified with preceding '<literal>-</literal>', each + option may be associated with a value. + </para> + + <para> + Options are as follows: + </para> + + <para> + <variablelist> + <varlistentry> + <term><option>D</option></term> + <listitem> + <para> + Specify a directory which holds data + for <command>gtm</command> or <command>gtm_proxy</command> + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>l</option></term> + <listitem> + <para> + Specify a log file for <command>gtm_ctl</command>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>m</option></term> + <listitem> + <para> + Set mode. Value can + be <literal>smart</literal>, <literal>fast</literal> + or <literal>immediate</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>o</option></term> + <listitem> + <para> + Option method. This parameter will be passed down + to <command>gtm</command> or <command>gtm_proxy</command>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>p</option></term> + <listitem> + <para> + Set up postgres bin repository. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>S</option></term> + <listitem> + <para> + Specify which to start, gtm or gtm_proxy. The value can be + specified as <literal>gtm</literal> + or <literal>gtm_proxy</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>t</option></term> + <listitem> + <para> + Specify the wait time in seconds. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>W</option> or <option>w</option></term> + <listitem> + <para> + Wait option. + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + + <para> + <command>gtm_ctl</command> issues the following keywords to select + operations on <command>gtm</command> + and <command>gtm_proxy</command>. + </para> + + <para> + <variablelist> + + <varlistentry> + <term><option>start</option></term> + <listitem> + <para> + Start a GTM/GTM proxy instance. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>restart</option></term> + <listitem> + <para> + Restart a GTM/GTM proxy instance. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>stop</option></term> + <listitem> + <para> + Stop a GTM/GTM proxy instance. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>status</option></term> + <listitem> + <para> + Look at the status of GTM instance. If active, 1 is printed. If + standby, 0 is printed. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>promote</option></term> + <listitem> + <para> + Promote a GTM instance as active. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>reconnect</option></term> + <listitem> + <para> + Reconnect a GTM Proxy to another GTM instance. + </para> + </listitem> + </varlistentry> + + </variablelist> + </para> + + <para> + Typically, you can issue the following command to start <command>gtm</command>/ +<programlisting> +gtm_ctl start -Z gtm -D datafolder +</programlisting> + </para> + + <para> + Or <command>gtm_proxy</command>: +<programlisting> +gtm_ctl start -Z gtm_proxy -D datafolder_proxy +</programlisting> + </para> + + <para> + Promote a GTM as active: +<programlisting> +gtm_ctl promote -Z gtm -D datafolder +</programlisting> + </para> + + <para> + Reconnect a GTM proxy to another GTM instance: +<programlisting> +gtm_ctl reconnect -Z gtm_proxy -D datafolder_proxy -o '-s hostname -t port_number' +</programlisting> + </para> + + <para> + Look at the status of a GTM server: +<programlisting> +gtm_ctl status -Z gtm -D datafolder +</programlisting> + </para> + </refsect1> + +</refentry> diff --git a/doc/src/sgml/ref/gtm_proxy.sgml b/doc/src/sgml/ref/gtm_proxy.sgml new file mode 100644 index 0000000000..6c706d8471 --- /dev/null +++ b/doc/src/sgml/ref/gtm_proxy.sgml @@ -0,0 +1,366 @@ + +<refentry id="APP-GTM-PROXY"> + <indexterm zone="app-gtm-proxy"> + <primary>gtm_proxy</primary> + </indexterm> + + <refmeta> + <refentrytitle>gtm_proxy</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo>Application</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>gtm_proxy</refname> + <refpurpose> + Proxy to <command>gtm</command>, <productname>Postgres-XL</productname> Global Transaction Manager + </refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>gtm_proxy</command> + <arg rep="repeat"><replaceable>option</></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1 id="R1-APP-GTM-PROXY-1"> + <title> + Description + </title> + <para> + The Gtm proxy provides proxy feature from Postgres-XL Coordinator and + Datanode to gtm. Gtm proxy groups connections and interactions + between gtm and other Postgres-XL components to reduce both the + number of interactions and the size of messages. Performance tests + have shown greater performance with high concurrency workloads as a result. + </para> + + <para> + It is highly advised to start and stop gtm_proxy with gtm_ctl(8). + </para> + + <para> + You must provide a gtm configuration + file <filename>gtm_proxy.conf</filename> placed at the gtm working directory + as specified by <literal>-D</literal> command line option. The + configuration file specifies gtm running environment and resources. + </para> + + <para> + Some of the parameters specified in the control file can be overridden by + command line options. + </para> + + </refsect1> + +<refsect1> + <title>Configuration File</title> + + <para> + <literal>GTM-Proxy</literal> configuration parameters are specified in the configuration file + <filename>gtm_proxy.conf</filename><indexterm><primary>gtm_proxy.conf</></> placed in the working directory + specified as <literal>-D</literal> option + of <application>gtm_proxy</application> command line option as described + in the next section. + </para> + + <para> + The format of the configuration file is the same as <filename>postgresql.conf</filename>. + Options are as follows. + </para> + +<!-- Add description of each gtm.conf entry --> +<!-- Notice + The following options are found in the source code (gtm_opt.c) but is only for + internal use and will not be presented in the following list. + + 1. data_dir + 2. config_file + + Also the following options are for high-availability hook. This will be + documented later. + 1. error_reporter + 2. status_reader +--> + <variablelist> + + <varlistentry id="gtm-proxy-opt-gtm-connect-retry-interval" xreflabel="gtm_proxy_opt_gtm_connect_retry_interval"> + <term><varname>gtm_connect_retry_interval</varname> (<type>integer</type>) + <indexterm> + <primary><varname>gtm_connect_retry_interval</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies how long in seconds GTM-Proxy waits between each retry to connect to GTM when + communication error with GTM is detected. + Default value is 60. + Refer to <varname>gtm_connect_retry_count</varname> and <varname>gtm_connect_retry_idle</varname>. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-gtm-host" xreflabel="gtm_proxy_opt_gtm_host"> + <term><varname>gtm_host</varname> (<type>string</type>) + <indexterm> + <primary><varname>gtm_host</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies listen addresses (host name or IP address) of <application>gtm</application>. + There is no default value for this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-gtm-port" xreflabel="gtm_proxy_opt_gtm_port"> + <term><varname>gtm_port</varname> (<type>integer</type>) + <indexterm> + <primary><varname>gtm_port</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the port number of <application>gtm</application>. + There is no default value for this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-keepalives-count" xreflabel="gtm_proxy_opt_keepalives_count"> + <term><varname>keepalives_count</varname> (<type>integer</type>) + <indexterm> + <primary><varname>keepalives_count</varname> (<type>integer</type>)</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies <literal>keepalives_count</literal> option for the + connection to <application>gtm</application>. + Default value is zero and keepalives feature is disabled. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-keepalives-idle" xreflabel="gtm_proxy_opt_keepalives_idle"> + <term><varname>keepalives_idle</varname> (<type>integer</type>) + <indexterm> + <primary><varname>keepalives_idle</varname> (<type>integer</type>)</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies <literal>keepalives_idle</literal> option for the + connection to <application>gtm</application>. + Default value is zero and keepalives feature is disabled. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-keepalives-interval" xreflabel="gtm_proxy_opt_keepalives_interval"> + <term><varname>keepalives_interval</varname> (<type>integer</type>) + <indexterm> + <primary><varname>keepalives_interval</varname> (<type>integer</type>)</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies <literal>keepalives_interval</literal> option for the + connection to <application>gtm</application>. + Default value is zero and keepalives feature is disabled. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-log-file" xreflabel="gtm_proxy_opt_log_file"> + <term><varname>log_file</varname> (<type>string</type>) + <indexterm> + <primary><varname>log_file</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies <filename>log</filename> file name. This file will be + created in the working directory of + this <application>gtm_proxy</application> as specified + by <literal>-D</literal> command line option. + The default is <filename>gtm_proxy.log</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-log-min-messages" xreflabel="gtm_proxy_opt_log_min_messages"> + <term><varname>log_min_messages</varname> (<type>enum</type>) + <indexterm> + <primary><varname>log_min_messages</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Controls which message levels are written to the log. + Valid values are <literal>DEBUG</literal>, <literal>DEBUG5</literal>, + <literal>DEBUG4</literal>, <literal>DEBUG3</literal>, <literal>DEBUG2</literal>, + <literal>DEBUG1</literal>, <literal>INFO</literal>, <literal>NOTICE</literal>, + <literal>WARNING</literal>, <literal>ERROR</literal>, <literal>LOG</literal>, + <literal>FATAL</literal> and <literal>PANIC</literal>. + Each level includes all the levels that follow it. + The later the level, the fewer messages are sent. + The default is <literal>WARNING</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-listen-addresses" xreflabel="gtm_proxy_opt_listen_addresses"> + <term><varname>listen_addresses</varname> (<type>string</type>) + <indexterm> + <primary><varname>listen_addresses</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies listen addresses (host name or IP address) of + this <application>gtm_proxy</application>. + Default value is '*'. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-nodename" xreflabel="gtm_proxy_opt_nodename"> + <term><varname>nodename</varname> (<type>string</type>) + <indexterm> + <primary><varname>nodename</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the name of this <application>gtm_proxy</application>. + There is no default value for this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-port" xreflabel="gtm_proxy_opt_port"> + <term><varname>port</varname> (<type>integer</type>) + <indexterm> + <primary><varname>port</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the port number of this <application>gtm_proxy</application>. + The default port value is 6666. + </para> + </listitem> + </varlistentry> + + <varlistentry id="gtm-proxy-opt-worker-threads" xreflabel="gtm_proxy_opt_worker_threads"> + <term><varname>worker_threads</varname> (<type>integer</type>) + <indexterm> + <primary><varname>worker_threads</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Specifies the number of worker threads for this <literal>gtm_proxy</literal>. + The default value is 1. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </refsect1> + + <refsect1> + <title>Options</title> + + <para> + Options are specified with preceding '<literal>-</literal>', each + option may be associated with a value. They can be specified + in <literal>-o</literal> option of gtm_ctl(8). + </para> + + <para> + Options are as follows: + </para> + + <para> + <variablelist> + <varlistentry> + <term><option>D</option></term> + <listitem> + <para> + Specify a directory which holds data for <command>gtm_proxy</command> + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>l</option></term> + <listitem> + <para> + Specify a log file for <command>gtm_ctl</command>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>h</option></term> + <listitem> + <para> + Specify host name or IP address used by the <command>gtm_proxy</command>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>p</option></term> + <listitem> + <para> + Specify port number to listen on. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>i</option></term> + <listitem> + <para> + Specify gtm_proxy id, which is registered to gtm. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>n</option></term> + <listitem> + <para> + Specify number of worker threads of gtm_proxy. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>s</option></term> + <listitem> + <para> + Specify host name or IP address of target gtm. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>t</option></term> + <listitem> + <para> + Specify port number of target gtm. + </para> + </listitem> + </varlistentry> + + </variablelist> + </para> + </refsect1> +</refentry> diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml index 4e339ecce8..5c20edf493 100644 --- a/doc/src/sgml/ref/initdb.sgml +++ b/doc/src/sgml/ref/initdb.sgml @@ -16,7 +16,8 @@ PostgreSQL documentation <refnamediv> <refname>initdb</refname> - <refpurpose>create a new <productname>PostgreSQL</productname> database cluster</refpurpose> + <refpurpose>create a new <productname>Postgres-XL</productname> Coordinator + or Datanode database cluster</refpurpose> </refnamediv> <refsynopsisdiv> @@ -39,7 +40,8 @@ PostgreSQL documentation </title> <para> <command>initdb</command> creates a new - <productname>PostgreSQL</productname> database cluster. A database + <productname>Postgres-XL</productname> database cluster for Coordinator or +Datanode. A database cluster is a collection of databases that are managed by a single server instance. </para> @@ -108,6 +110,11 @@ PostgreSQL documentation More details can be found in <xref linkend="multibyte">. </para> + <para> + <command>initdb</> will be performed locally. This has to be + performed for each Coordinators and Datanodes manually. + </para> + </refsect1> <refsect1> @@ -168,6 +175,18 @@ PostgreSQL documentation </varlistentry> <varlistentry> + <term><option>--nodename=<replaceable class="parameter">nodename</replaceable></option></term> + <listitem> + <para> + Set the name of Postgres-XL node initialized. This option is mandatory when setting a node. + It permits one to define the node itself in cluster node catalog + <link linkend="catalog-pgxc-node"><structname>pgxc_node</structname></link>. The node name + specified is also added in <filename>postgresql.conf</> as value of <varname>pgxc_node_name</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>-E <replaceable class="parameter">encoding</replaceable></option></term> <term><option>--encoding=<replaceable class="parameter">encoding</replaceable></option></term> <listitem> @@ -439,6 +458,11 @@ PostgreSQL documentation <title>Notes</title> <para> + <command>initdb</> runs only locally. You should + run <command>initdb</> for each Coordinator and Datanode. + </para> + + <para> <command>initdb</command> can also be invoked via <command>pg_ctl initdb</command>. </para> diff --git a/doc/src/sgml/ref/initgtm.sgml b/doc/src/sgml/ref/initgtm.sgml new file mode 100644 index 0000000000..0319a54797 --- /dev/null +++ b/doc/src/sgml/ref/initgtm.sgml @@ -0,0 +1,197 @@ + +<refentry id="APP-INITGTM"> + <indexterm zone="app-initgtm"> + <primary>initgtm</primary> + </indexterm> + + <refmeta> + <refentrytitle>initgtm</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo>Application</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>initgtm</refname> + <refpurpose>create a new <productname>Postgres-XL</productname> GTM or GTM-Proxy for database cluster</refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>initgtm</command> + <arg rep="repeat"><replaceable>option</replaceable></arg> + <group choice="plain"> + <arg>--pgdata</arg> + <arg>-D </arg> + <replaceable>directory</replaceable> + </group> + <arg choice="plain">-Z <replaceable>nodetype</replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1 id="R1-APP-INITGTM-1"> + <title> + Description + </title> + <para> + <command>initgtm</command> creates a new GTM or GTM-Proxy node for a + <productname>Postgres-XL</productname> database cluster. A database + cluster has a unique GTM. A GTM-Proxy acts as an intermediate component + between GTM and Postgres-XL nodes to group request messages. Each Coordinator + and Datanode of the cluster need to register to GTM when starting up. + </para> + + <para> + Creating a GTM for cluster consists of creating the directories and files in + which the GTM data will live. + </para> + + <para> + Although <command>initgtm</command> will attempt to create the + specified data directory, it might not have permission if the parent + directory of the desired data directory is root-owned. To initialize + in such a setup, create an empty data directory as root, then use + <command>chown</command> to assign ownership of that directory to the + database user account, then <command>su</command> to become the + database user to run <command>initgtm</command>. + </para> + + <para> + <command>initgtm</command> must be run as the user that will own the + server process, because the server needs to have access to the + files and directories that <command>initgtm</command> creates. + Since the server cannot be run as root, you must not run + <command>initgtm</command> as root either. (It will in fact refuse + to do so.) + </para> + + <para> + <command>initgtm</> will be performed locally. + </para> + + </refsect1> + + <refsect1> + <title>Options</title> + + <para> + <variablelist> + <varlistentry> + <term><option>-D <replaceable class="parameter">directory</replaceable></option></term> + <term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term> + <listitem> + <para> + This option specifies the directory where the GTM data + should be stored. Data folder and node type are the only information + required by <command>initgtm</command>. You can avoid writing it by + setting the <envar>PGDATA</envar> environment variable. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-Z <replaceable class="parameter">nodetype</replaceable></option></term> + <listitem> + <para> + This option specifies the node type which is initialized. It is possible to + specify gtm to set up a GTM node, or gtm_proxy to set up a GTM-Proxy. + </para> + </listitem> + </varlistentry> + + </variablelist> + </para> + + <para> + Other, less commonly used, parameters are also available: + + <variablelist> + <varlistentry> + <term><option>-d</option></term> + <term><option>--debug</option></term> + <listitem> + <para> + Print debugging output from the bootstrap backend and a few other + messages of lesser interest for the general public. + The bootstrap backend is the program <command>initgtm</command> + uses to create the catalog tables. This option generates a tremendous + amount of extremely boring output. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-n</option></term> + <term><option>--noclean</option></term> + <listitem> + <para> + By default, when <command>initgtm</command> + determines that an error prevented it from completely creating GTM data + it removes any files it might have created before discovering + that it cannot finish the job. This option inhibits tidying-up and is + thus useful for debugging. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-V</></term> + <term><option>--version</></term> + <listitem> + <para> + Print the <application>initgtm</application> version and exit. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-?</></term> + <term><option>--help</></term> + <listitem> + <para> + Show help about <application>initgtm</application> command line + arguments, and exit. + </para> + </listitem> + </varlistentry> + + </variablelist> + </para> + + </refsect1> + + <refsect1> + <title>Environment</title> + + <variablelist> + <varlistentry> + <term><envar>PGDATA</envar></term> + + <listitem> + <para> + Specifies the directory where the GTM data is to be + stored; can be overridden using the <option>-D</option> option. + </para> + </listitem> + </varlistentry> + </variablelist> + + </refsect1> + + <refsect1> + <title>Notes</title> + + <para> + <command>initgtm</> runs only locally. + </para> + </refsect1> + + <refsect1> + <title>See Also</title> + + <simplelist type="inline"> + <member><xref linkend="app-gtm-ctl"></member> + </simplelist> + </refsect1> + +</refentry> + diff --git a/doc/src/sgml/ref/listen.sgml b/doc/src/sgml/ref/listen.sgml index 9cd53b02bb..3de5d9011d 100644 --- a/doc/src/sgml/ref/listen.sgml +++ b/doc/src/sgml/ref/listen.sgml @@ -29,6 +29,11 @@ LISTEN <replaceable class="PARAMETER">channel</replaceable> <title>Description</title> <para> + The <command>LISTEN</> statement is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>LISTEN</command> registers the current session as a listener on the notification channel named <replaceable class="PARAMETER">channel</replaceable>. diff --git a/doc/src/sgml/ref/load.sgml b/doc/src/sgml/ref/load.sgml index a5a6ef8e67..74aff0c833 100644 --- a/doc/src/sgml/ref/load.sgml +++ b/doc/src/sgml/ref/load.sgml @@ -57,6 +57,14 @@ LOAD '<replaceable class="PARAMETER">filename</replaceable>' responsibility to ensure that only <quote>safe</> libraries are installed there.) </para> + + <para> + Please note that in <productname>Postgres-XL</>, <command>LOAD</command> command loads the library only + locally. You should load library manually on each Datanode and + Coordinator (you can use psql directly to Datanodes for this + puupose), or <filename>edit postgresql.conf</filename> for all the + Datanodes and Coordinators. + </para> </refsect1> <refsect1 id="sql-load-compat"> diff --git a/doc/src/sgml/ref/notify.sgml b/doc/src/sgml/ref/notify.sgml index ad574e9ea0..44c987bcfd 100644 --- a/doc/src/sgml/ref/notify.sgml +++ b/doc/src/sgml/ref/notify.sgml @@ -29,6 +29,11 @@ NOTIFY <replaceable class="PARAMETER">channel</replaceable> [ , <replaceable cla <title>Description</title> <para> + <command>The NOTIFY</> statement is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> The <command>NOTIFY</command> command sends a notification event together with an optional <quote>payload</> string to each client application that has previously executed diff --git a/doc/src/sgml/ref/pause_cluster.sgml b/doc/src/sgml/ref/pause_cluster.sgml new file mode 100644 index 0000000000..6522872b83 --- /dev/null +++ b/doc/src/sgml/ref/pause_cluster.sgml @@ -0,0 +1,64 @@ + +<refentry id="SQL-PAUSECLUSTER"> + <indexterm zone="sql-pausecluster"> + <primary>PAUSE CLUSTER</primary> + </indexterm> + + <refmeta> + <refentrytitle>PAUSE CLUSTER</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>PAUSE CLUSTER</refname> + <refpurpose>pause the <productname>Postgres-XL</productname> cluster</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +PAUSE CLUSTER + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>PAUSE CLUSTER </command> is a SQL command specific + to <productname>Postgres-XL</productname> that pauses + cluster operation. + </para> + <para> + Pause blocks any new transactions from starting and waits until + existing transactions complete, then returns. Existing sessions + are still connected to Coordinators, it is just that any new statements + will be held up and not be executed. + </para> + + <para> + The session that paused the cluster can perform tasks exclusively on the + cluster. This is useful for maintenance tasks to resolve a problem, + restart a Datanode, manually failover a Datanode, etc. Applications will + not receive error messages unless they themselves timeout, statement execution + will just be briefly suspended. + </para> + + <para> + Once the DBA has completed whatever tasks were needed, the command <xref linkend="sql-unpausecluster"> can be used. + </para> + + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>PAUSE CLUSTER</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> + + diff --git a/doc/src/sgml/ref/pg_controldata.sgml b/doc/src/sgml/ref/pg_controldata.sgml index 4a360d61fd..4020f1566d 100644 --- a/doc/src/sgml/ref/pg_controldata.sgml +++ b/doc/src/sgml/ref/pg_controldata.sgml @@ -16,7 +16,8 @@ PostgreSQL documentation <refnamediv> <refname>pg_controldata</refname> - <refpurpose>display control information of a <productname>PostgreSQL</productname> database cluster</refpurpose> + <refpurpose>display control information of a +<productname>Postgres-XL</productname> Coordinator or Datanode database cluster</refpurpose> </refnamediv> <refsynopsisdiv> @@ -47,6 +48,11 @@ PostgreSQL documentation supports options <option>-?</> and <option>--help</>, which output the supported arguments. </para> + + <para> + To print information of each Datanode and Coordinator, you should + issue <command>pg_controldata</> against each of them. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/pg_ctl-ref.sgml b/doc/src/sgml/ref/pg_ctl-ref.sgml index 2df65db0bd..86e38741cd 100644 --- a/doc/src/sgml/ref/pg_ctl-ref.sgml +++ b/doc/src/sgml/ref/pg_ctl-ref.sgml @@ -39,6 +39,7 @@ PostgreSQL documentation <arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg> <arg choice="opt"><option>-p</option> <replaceable>path</replaceable></arg> <arg choice="opt"><option>-c</option></arg> + <arg>-Z <replaceable>nodeopt</replaceable></arg> </cmdsynopsis> <cmdsynopsis> @@ -73,6 +74,7 @@ PostgreSQL documentation </group> </arg> <arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg> + <arg>-Z <replaceable>nodeopt</replaceable></arg> </cmdsynopsis> <cmdsynopsis> @@ -347,6 +349,22 @@ PostgreSQL documentation </varlistentry> <varlistentry> + <term><option>-Z <replaceable class="parameter">nodeopt</replaceable></option></term> + <listitem> + <para> + Specifies to run node as a Coordinator, as a Datanode or in restore mode. + <replaceable>nodeopt</replaceable> can be respectively + <literal>coordinator</literal>, <literal>datanode</literal> or + <literal>restoremode</literal>. + </para> + <para> + With restore mode, you can import Postgres-XL's catalog data from other coordinator + or datanode + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>-s</option></term> <term><option>--silent</option></term> <listitem> @@ -505,6 +523,10 @@ PostgreSQL documentation (see <xref linkend="libpq-envars">). For additional server variables, see <xref linkend="app-postgres">. </para> + + <para> + In <productname>Postgres-XL</>, this command controls individual Coordinator or Datanode. + </para> </refsect1> diff --git a/doc/src/sgml/ref/pg_resetxlog.sgml b/doc/src/sgml/ref/pg_resetxlog.sgml index 59280f01cb..de64adf789 100644 --- a/doc/src/sgml/ref/pg_resetxlog.sgml +++ b/doc/src/sgml/ref/pg_resetxlog.sgml @@ -222,6 +222,12 @@ PostgreSQL documentation <command>pg_resetxlog</command> to run. But before you do so, make doubly certain that there is no server process still alive. </para> + + <para> + In <productname>Postgres-XL</>, <command>pg_resetxlog</command> + will only run locally for Coordinators and Datanodes. You should run it + for each Coordinator or Datanode manually. + </para> </refsect1> </refentry> diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index a8085463a5..9f73bf6dd9 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -211,6 +211,19 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</> </varlistentry> <varlistentry> + <term><option>-k</option></term> + <listitem> + <para> + Use column <structname>bid</> as hash distribution key for tables + <structname>pgbench_accounts</>, <structname>pgbench_tellers</>, + <structname>pgbench_branches</>. This limits the selectivity of + remote nodes to a single node for each transaction as all the tables + are distributed respecting the same key. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>--foreign-keys</option></term> <listitem> <para> @@ -553,6 +566,18 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</> </varlistentry> <varlistentry> + <term><option>-k</option></term> + <listitem> + <para> + Add an additional condition based on column <structname>bid</> based + on its values as a hash key to enforce selectivity of all write queries + to a single remote node. This functionality permits to check the + write-scalability performance of the system by distributing data among cluster nodes. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>--aggregate-interval=<replaceable>seconds</></option></term> <listitem> <para> diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml index e97fb480f3..2d53eda644 100644 --- a/doc/src/sgml/ref/pgupgrade.sgml +++ b/doc/src/sgml/ref/pgupgrade.sgml @@ -34,6 +34,12 @@ <refsect1> <title>Description</title> + <para> + <application>pg_updrade</> is not compatible + with <productname>Postgres-XL</> due to the difference of internal + catalog. + </para> + <para> <application>pg_upgrade</> (formerly called <application>pg_migrator</>) allows data stored in <productname>PostgreSQL</> data files to be upgraded to a later <productname>PostgreSQL</> diff --git a/doc/src/sgml/ref/postgres-ref.sgml b/doc/src/sgml/ref/postgres-ref.sgml index e2e9909242..cdf5f89f3e 100644 --- a/doc/src/sgml/ref/postgres-ref.sgml +++ b/doc/src/sgml/ref/postgres-ref.sgml @@ -115,6 +115,15 @@ PostgreSQL documentation </varlistentry> <varlistentry> + <term><option>--coordinator</option></term> + <listitem> + <para> + Specifies postgres server should run as a Coordinator. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>-c <replaceable>name</replaceable>=<replaceable>value</replaceable></option></term> <listitem> <para> @@ -347,6 +356,15 @@ PostgreSQL documentation </varlistentry> <varlistentry> + <term><option>--datanode</option></term> + <listitem> + <para> + Specifies postgres server should run as a Datanode. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>-V</></term> <term><option>--version</></term> <listitem> @@ -518,6 +536,18 @@ PostgreSQL documentation </para> </listitem> </varlistentry> + + <varlistentry> + <term><option>--localxid</option></term> + <listitem> + <para> + Use local transaction IDs rather than GXID. This option applies + only to <productname>Postgres-XL</productname>. It is only + used by initdb. Explicit use of this option may cause + database inconsistency. + </para> + </listitem> + </varlistentry> </variablelist> </refsect2> @@ -809,19 +839,28 @@ PostgreSQL documentation <title>Examples</title> <para> - To start <command>postgres</command> in the background + To start <command>postgres</command> as a Datanode in the background + using default values, type: + +<screen> +<prompt>$</prompt> <userinput>nohup postgres --datanode >logfile 2>&1 </dev/null &</userinput> +</screen> + </para> + + <para> + To start <command>postgres</command> as a Coordinator in the background using default values, type: <screen> -<prompt>$</prompt> <userinput>nohup postgres >logfile 2>&1 </dev/null &</userinput> +<prompt>$</prompt> <userinput>nohup postgres --coordinator >logfile 2>&1 </dev/null &</userinput> </screen> </para> <para> - To start <command>postgres</command> with a specific + To start <command>postgres</command> as a Coordinator with a specific port, e.g. 1234: <screen> -<prompt>$</prompt> <userinput>postgres -p 1234</userinput> +<prompt>$</prompt> <userinput>postgres --coordinator -p 1234</userinput> </screen> To connect to this server using <application>psql</>, specify this port with the -p option: <screen> @@ -837,8 +876,8 @@ PostgreSQL documentation <para> Named run-time parameters can be set in either of these styles: <screen> -<prompt>$</prompt> <userinput>postgres -c work_mem=1234</userinput> -<prompt>$</prompt> <userinput>postgres --work-mem=1234</userinput> +<prompt>$</prompt> <userinput>postgres --coordinator -c work_mem=1234</userinput> +<prompt>$</prompt> <userinput>postgres --coordinator --work-mem=1234</userinput> </screen> Either form overrides whatever setting might exist for <varname>work_mem</> in <filename>postgresql.conf</>. Notice that diff --git a/doc/src/sgml/ref/postmaster.sgml b/doc/src/sgml/ref/postmaster.sgml index 0a58a63331..23e4ce83d9 100644 --- a/doc/src/sgml/ref/postmaster.sgml +++ b/doc/src/sgml/ref/postmaster.sgml @@ -16,7 +16,7 @@ PostgreSQL documentation <refnamediv> <refname id="postmaster-ref">postmaster</refname> - <refpurpose><productname>PostgreSQL</productname> database server</refpurpose> + <refpurpose><productname>Postgres-XL</productname> database server for Coordinator or Datanode</refpurpose> </refnamediv> <refsynopsisdiv> diff --git a/doc/src/sgml/ref/prepare_transaction.sgml b/doc/src/sgml/ref/prepare_transaction.sgml index 626753f576..539d9080c9 100644 --- a/doc/src/sgml/ref/prepare_transaction.sgml +++ b/doc/src/sgml/ref/prepare_transaction.sgml @@ -121,6 +121,12 @@ PREPARE TRANSACTION <replaceable class="PARAMETER">transaction_id</replaceable> system view. </para> + <para> + If the transaction is involved by more than one Datanode and/or + Coordinator, <command>COMMIT PREPARED</> will be propagated to + these nodes. + </para> + <caution> <para> It is unwise to leave transactions in the prepared state for a long time. diff --git a/doc/src/sgml/ref/release_savepoint.sgml b/doc/src/sgml/ref/release_savepoint.sgml index b331b7226b..a5fd1f6c2e 100644 --- a/doc/src/sgml/ref/release_savepoint.sgml +++ b/doc/src/sgml/ref/release_savepoint.sgml @@ -34,6 +34,11 @@ RELEASE [ SAVEPOINT ] <replaceable>savepoint_name</replaceable> <title>Description</title> <para> + <command>RELEASE SAVEPOINT</command> is not supported by the + current release of <productname>Postgres-XL</productname>. + </para> + + <para> <command>RELEASE SAVEPOINT</command> destroys a savepoint previously defined in the current transaction. </para> diff --git a/doc/src/sgml/ref/rollback_prepared.sgml b/doc/src/sgml/ref/rollback_prepared.sgml index a5328e96a2..141c77b128 100644 --- a/doc/src/sgml/ref/rollback_prepared.sgml +++ b/doc/src/sgml/ref/rollback_prepared.sgml @@ -69,6 +69,15 @@ ROLLBACK PREPARED <replaceable class="PARAMETER">transaction_id</replaceable> <link linkend="view-pg-prepared-xacts"><structname>pg_prepared_xacts</structname></link> system view. </para> + + <para> + If more than one Datanode and/or Coordinator are involved in the + transaction, the <command>ROLLBACK PREPARED</> command will propagate to + all the nodes involved. + </para> + <para> + If <literal>xc_maintenance_mode</literal> GUC parameter is set to <literal>ON</literal>, <command>ROLLBACK PREPARED</command> will not propagate to other nodes. It just runs locally and report the result to <literal>GTM</literal>. + </para> </refsect1> <refsect1 id="sql-rollback-prepared-examples"> @@ -101,6 +110,7 @@ ROLLBACK PREPARED 'foobar'; <simplelist type="inline"> <member><xref linkend="sql-prepare-transaction"></member> <member><xref linkend="sql-commit-prepared"></member> + <member><xref linkend="guc-xc-maintenance-mode"></member> </simplelist> </refsect1> diff --git a/doc/src/sgml/ref/rollback_to.sgml b/doc/src/sgml/ref/rollback_to.sgml index 060f408a63..360bbb089d 100644 --- a/doc/src/sgml/ref/rollback_to.sgml +++ b/doc/src/sgml/ref/rollback_to.sgml @@ -34,6 +34,12 @@ ROLLBACK [ WORK | TRANSACTION ] TO [ SAVEPOINT ] <replaceable>savepoint_name</re <title>Description</title> <para> + <command>ROLLBACK TO</> is not yet supported + in <productname>Postgres-XL</>. This may be supported in the + future releases. + </para> + + <para> Roll back all commands that were executed after the savepoint was established. The savepoint remains valid and can be rolled back to again later, if needed. diff --git a/doc/src/sgml/ref/savepoint.sgml b/doc/src/sgml/ref/savepoint.sgml index 5b944a2561..8bbc855277 100644 --- a/doc/src/sgml/ref/savepoint.sgml +++ b/doc/src/sgml/ref/savepoint.sgml @@ -34,6 +34,12 @@ SAVEPOINT <replaceable>savepoint_name</replaceable> <title>Description</title> <para> + <command>SAVEPOINT</> has not been supported + by <productname>Postgres-XL</> yet. This command may be supported + in the future releases. + </para> + + <para> <command>SAVEPOINT</command> establishes a new savepoint within the current transaction. </para> diff --git a/doc/src/sgml/ref/select_into.sgml b/doc/src/sgml/ref/select_into.sgml index 84b0dd831f..a9131ad39f 100644 --- a/doc/src/sgml/ref/select_into.sgml +++ b/doc/src/sgml/ref/select_into.sgml @@ -111,6 +111,14 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac variable. Alternatively, <command>CREATE TABLE AS</command> can be used with the <literal>WITH OIDS</literal> clause. </para> + + <para> + In <productname>Postgres-XL</>, the <command>SELECT INTO</command> distributes data + of the newly-created table on all the nodes respecting the default + distribution which is <literal>HASH</literal> on the first column + having a type that can be distributed. If no columns are found, + distribution is done by <literal>ROUNDROBIN</literal>. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/show.sgml b/doc/src/sgml/ref/show.sgml index 46bb239baf..05fe9dca0a 100644 --- a/doc/src/sgml/ref/show.sgml +++ b/doc/src/sgml/ref/show.sgml @@ -135,6 +135,11 @@ SHOW ALL system view produces the same information. </para> + + <para> + <productname>Postgres-XL</> only shows local settings of the node to which + client has connected. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/unlisten.sgml b/doc/src/sgml/ref/unlisten.sgml index f7c3c47e2f..37f906ee2b 100644 --- a/doc/src/sgml/ref/unlisten.sgml +++ b/doc/src/sgml/ref/unlisten.sgml @@ -29,6 +29,11 @@ UNLISTEN { <replaceable class="PARAMETER">channel</replaceable> | * } <title>Description</title> <para> + <command>UNLISTEN</> statement is not yet supported + in <productname>Postgres-XL</>. + </para> + + <para> <command>UNLISTEN</command> is used to remove an existing registration for <command>NOTIFY</command> events. <command>UNLISTEN</command> cancels any existing registration of diff --git a/doc/src/sgml/ref/unpause_cluster.sgml b/doc/src/sgml/ref/unpause_cluster.sgml new file mode 100644 index 0000000000..f30d2cee99 --- /dev/null +++ b/doc/src/sgml/ref/unpause_cluster.sgml @@ -0,0 +1,49 @@ + +<refentry id="SQL-UNPAUSECLUSTER"> + <indexterm zone="sql-pausecluster"> + <primary>UNPAUSE CLUSTER</primary> + </indexterm> + + <refmeta> + <refentrytitle>UNPAUSE CLUSTER</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo>SQL - Language Statements</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>UNPAUSE CLUSTER</refname> + <refpurpose>unpause the <productname>Postgres-XL</productname> cluster</refpurpose> + </refnamediv> + + <refsynopsisdiv> +<synopsis> +UNPAUSE CLUSTER + +</synopsis> + </refsynopsisdiv> + + <refsect1> + <title>Description</title> + + <para> + <command>UNPAUSE CLUSTER </command> is a SQL command specific + to <productname>Postgres-XL</productname> that unpauses + cluster operation. + </para> + + <para> + If the DBA previously paused the cluster via the command <xref linkend="sql-pausecluster">, the DBA can resume operation vi <command>UNPAUSE CLUSTER</command>. + </para> + </refsect1> + + <refsect1> + <title>Compatibility</title> + <para> + <command>UNPAUSE CLUSTER</command> does not conform to the <acronym> + SQL</acronym> standards, it is a Postgres-XL specific command. + </para> + </refsect1> + +</refentry> + + diff --git a/doc/src/sgml/ref/update.sgml b/doc/src/sgml/ref/update.sgml index 35b0699f08..86e8817c72 100644 --- a/doc/src/sgml/ref/update.sgml +++ b/doc/src/sgml/ref/update.sgml @@ -203,6 +203,12 @@ UPDATE [ ONLY ] <replaceable class="PARAMETER">table_name</replaceable> [ * ] [ <xref linkend="sql-declare"> for more information about using cursors with <literal>WHERE CURRENT OF</>. + <footnote> + <para> + <literal>WHERE CURRENT OF</literal> clause is not supported by + the current release of <productname>Postgres-XL</productname>. + </para> + </footnote> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml index 19fd748dde..b947332673 100644 --- a/doc/src/sgml/ref/vacuum.sgml +++ b/doc/src/sgml/ref/vacuum.sgml @@ -74,6 +74,11 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] ANALYZE [ <replaceable class="PARAMETER"> <productname>PostgreSQL</productname> 9.0; the unparenthesized syntax is deprecated. </para> + + <para> + In <productname>Postgres-XL</>, <command>VACUUM</> will be performed + on all the Datanodes as well. + </para> </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml index e38c34aea3..4235cdd9e7 100644 --- a/doc/src/sgml/ref/vacuumdb.sgml +++ b/doc/src/sgml/ref/vacuumdb.sgml @@ -68,6 +68,11 @@ PostgreSQL documentation server. </para> + <para> + In <productname>Postgres-XL</>, <command>VACUUM</> will be + performed in all the Datanodes as well. + </para> + </refsect1> diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml index 03020dfec4..1d71d95d47 100644 --- a/doc/src/sgml/reference.sgml +++ b/doc/src/sgml/reference.sgml @@ -50,6 +50,7 @@ &alterLanguage; &alterLargeObject; &alterMaterializedView; + &alterNode; &alterOperator; &alterOperatorClass; &alterOperatorFamily; @@ -74,6 +75,7 @@ &analyze; &begin; &checkpoint; + &cleanConnection; &close; &cluster; &commentOn; @@ -81,6 +83,7 @@ &commitPrepared; ©Table; &createAggregate; + &createBarrier; &createCast; &createCollation; &createConversion; @@ -95,6 +98,8 @@ &createIndex; &createLanguage; &createMaterializedView; + &createNode; + &createNodeGroup; &createOperator; &createOperatorClass; &createOperatorFamily; @@ -137,6 +142,8 @@ &dropIndex; &dropLanguage; &dropMaterializedView; + &dropNode; + &dropNodeGroup; &dropOperator; &dropOperatorClass; &dropOperatorFamily; @@ -161,6 +168,7 @@ &dropView; &end; &execute; + &executeDirect; &explain; &fetch; &grant; @@ -171,6 +179,7 @@ &lock; &move; ¬ify; + &pauseCluster; &prepare; &prepareTransaction; &reassignOwned; @@ -195,6 +204,7 @@ &startTransaction; &truncate; &unlisten; + &unpauseCluster; &update; &vacuum; &values; @@ -259,7 +269,11 @@ </para> </partintro> + >m; + >mPxy; + >mCtl; &initdb; + &initgtm; &pgarchivecleanup; &pgControldata; &pgCtl; diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml index 504d8daa71..c91386a5b1 100644 --- a/doc/src/sgml/regress.sgml +++ b/doc/src/sgml/regress.sgml @@ -12,6 +12,13 @@ </indexterm> <para> + Current version of <productname>Postgres-XL</productname> doesn't + support the regression test to run after building but before + installation. You can use each test manually but it is left to + users. + </para> + + <para> The regression tests are a comprehensive set of tests for the SQL implementation in <productname>PostgreSQL</productname>. They test standard SQL operations as well as the extended capabilities of diff --git a/doc/src/sgml/release-xl-9.5.sgml b/doc/src/sgml/release-xl-9.5.sgml new file mode 100644 index 0000000000..67115bd75a --- /dev/null +++ b/doc/src/sgml/release-xl-9.5.sgml @@ -0,0 +1,38 @@ +<!-- doc/src/sgml/release-9.5.sgml --> +<!-- See header comment in release.sgml about typical markup --> + + <sect1 id="release-xl-9-5"> + <title>Postgres-XL Release 9.5</title> + + <note> + <title>Release Date</title> + <simpara>2015-??-??</simpara> + <simpara>Current as of 2015-06-01</simpara> + </note> + + <sect2> + <title>Overview</title> + + <para> + Major enhancements in <productname>Postgres-XL</> 9.5 include: + </para> + + <!-- This list duplicates items below, but without authors or details--> + + <itemizedlist> + + <listitem> + <para> + ... to be filled in ... + </para> + </listitem> + + </itemizedlist> + + <para> + The above items are explained in more detail in the sections below. + </para> + + </sect2> + + </sect1> diff --git a/doc/src/sgml/release.sgml b/doc/src/sgml/release.sgml index 7004318c44..dda63ef070 100644 --- a/doc/src/sgml/release.sgml +++ b/doc/src/sgml/release.sgml @@ -73,6 +73,7 @@ For new features, add links to the documentation sections. The reason for splitting the release notes this way is so that appropriate subsets can easily be copied into back branches. --> +&release-xl-9.5; &release-9.5; &release-9.4; &release-9.3; diff --git a/doc/src/sgml/remove-node.sgml b/doc/src/sgml/remove-node.sgml new file mode 100644 index 0000000000..61c2a91b76 --- /dev/null +++ b/doc/src/sgml/remove-node.sgml @@ -0,0 +1,160 @@ +<!-- doc/src/sgml/remove-node.sgml --> + +<chapter id="remove-node"> + <title>Removing an Existing Node</title> + + <indexterm zone="remove-node"> + <primary>Remove an existing node</primary> + </indexterm> + + + + + +&xlonly; + + + <para> + This chapter outlines steps to remove an existing coordinator or a datanode from a running cluster. + </para> + + <para> + </para> + + <sect1 id="remove-node-coordinator"> + <title>Removing an Existing Coordinator</title> + + <indexterm zone="remove-node-coordinator"> + <primary>Remove an existing coordinator</primary> + </indexterm> + + <para> + Assume a two coordinator cluster, COORD_1 and COORD_2. Suppose we want to remove COORD_2 for any reason. The following steps should be performed to remove an existing coordinator from a running cluster: + </para> + + <para> + <orderedlist> + <listitem> + <para> + Stop the coordinator to be removed. In our example we need to stop COORD_2. + </para> + </listitem> + + <listitem> + <para> + Connect to any of the coordinators except the one to be removed. In our example assuming COORD_1 is running on port 5432, the following command would connect to COORD_1 + </para> + <programlisting> + psql postgres -p 5432 + </programlisting> + </listitem> + + <listitem> + <para> + Drop the coordinator to be removed. For example to drop coordinator COORD_2 + </para> + <programlisting> + DROP NODE COORD_2; + </programlisting> + </listitem> + + <listitem> + <para> + Update the connection information cached in pooler. + </para> + <programlisting> + SELECT pgxc_pool_reload(); + </programlisting> + </listitem> + + </orderedlist> + </para> + <para> + COORD_2 is now removed from the cluster and COORD_1 would work as if COORD_2 never existed. + </para> + <para> + CAUTION : If COORD_2 is still running and clients are connected to it, any queries issued would create inconsistencies in the cluster. + </para> + + </sect1> + + <sect1 id="remove-node-datanode"> + <title>Removing an Existing Datanode</title> + + <indexterm zone="remove-node-datanode"> + <primary>Remove an existing Datanode</primary> + </indexterm> + + <para> + Assume a two coordinator cluster, COORD_1 and COORD_2 with three datanodes DATA_NODE_1, DATA_NODE_2 and DATA_NODE_3. Suppose we want to remove DATA_NODE_3 for any reason. Further assume there is a table named rr_tab_foo distributed in round robin fashion and has rows on all the three datanodes. Following steps should be performed to remove an existing datanode from a running cluster: + </para> + + <para> + <orderedlist> + + <listitem> + <para> + Transfer the data from the datanode to be removed to the rest of the datanodes for all the tables in all the databases. For example to shift data of the table rr_tab_foo to the rest of the nodes we can use command: + </para> + <programlisting> + ALTER TABLE rr_tab_foo DELETE NODE (DATA_NODE_3); + </programlisting> + </listitem> + + <listitem> + <para> + Confirm that there is no data left on the datanode to be removed. For example to confirm that there is no data left on DATA_NODE_3, we can use the following command, it should return ZERO rows. In case of non-zero rows it returns the OIDs of the relations whose data still exists on DATA_NODE_3. + </para> + <programlisting> + SELECT c.pcrelid FROM pgxc_class c, pgxc_node n WHERE n.node_name = 'DATA_NODE_3' AND n.oid = ANY (c.nodeoids); + </programlisting> + </listitem> + + <listitem> + <para> + Stop the datanode server to be removed. Now any SELECTs or DMLs that involve the datanode to be removed would start failing. + </para> + </listitem> + + <listitem> + <para> + Connect to any of the coordinators. In our example assuming COORD_1 is running on port 5432, the following command would connect to COORD_1. + </para> + <programlisting> + psql postgres -p 5432 + </programlisting> + </listitem> + + <listitem> + <para> + Drop the datanode to be removed. For example to drop datanode DATA_NODE_3 use command. + </para> + <programlisting> + DROP NODE DATA_NODE_3; + </programlisting> + </listitem> + + <listitem> + <para> + Update the connection information cached in pooler. + </para> + <programlisting> + SELECT pgxc_pool_reload(); + </programlisting> + </listitem> + + <listitem> + <para> + Repeat the above three steps (4,5,6) for all the coordinators in the cluster. In our example we would need to repeat the above steps by connecting to COORD_2. + </para> + </listitem> + + </orderedlist> + </para> + <para> + DATA_NODE_3 is now removed from the cluster. + </para> + + </sect1> + +</chapter> diff --git a/doc/src/sgml/rules.sgml b/doc/src/sgml/rules.sgml index cb5c8fccae..803ec44384 100644 --- a/doc/src/sgml/rules.sgml +++ b/doc/src/sgml/rules.sgml @@ -657,6 +657,12 @@ SELECT shoe_ready.shoename, shoe_ready.sh_avail, <title>View Rules in Non-<command>SELECT</command> Statements</title> <para> + Non-<command>SELECT</command> statements in rules are not + supported by the current version + of <productname>Postgres-XL</productname>. +</para> + +<para> Two details of the query tree aren't touched in the description of view rules above. These are the command type and the result relation. In fact, the command type is not needed by view rules, but the result @@ -2249,6 +2255,12 @@ CREATE VIEW phone_number WITH (security_barrier) AS </indexterm> <para> + Trigger is not supported by the current release + of <productname>Postgres-XL</>. This may be supported in the + future releases. +</para> + +<para> Many things that can be done using triggers can also be implemented using the <productname>PostgreSQL</productname> rule system. One of the things that cannot be implemented by diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml index dacd3e1dfe..6260fe794c 100644 --- a/doc/src/sgml/runtime.sgml +++ b/doc/src/sgml/runtime.sgml @@ -47,6 +47,15 @@ </indexterm> <para> + The steps to configure a <productname>Postgres-XL</productname> + cluster can be cumbersome. It is highly recommended that you make use + of a utility called <xref linkend="pgxc-ctl"> to make this task easier, + and you can skip a lot of the content in this chapter. + That said, you may read on to understand how to manually configure + the cluster, which essentially describes the steps that <xref linkend="pgxc-ctl"> does. + </para> + + <para> Before you can do anything, you must initialize a database storage area on disk. We call this a <firstterm>database cluster</firstterm>. (<acronym>SQL</acronym> uses the term catalog cluster.) A @@ -74,15 +83,19 @@ <filename>/var/lib/pgsql/data</filename> are popular. To initialize a database cluster, use the command <xref linkend="app-initdb">,<indexterm><primary>initdb</></> which is - installed with <productname>PostgreSQL</productname>. The desired + installed with <productname>Postgres-XL</productname>. The desired file system location of your database cluster is indicated by the - <option>-D</option> option, for example: + <option>-D</option> option. You also need to define a node name for + the cluster element initialized, for example: <screen> -<prompt>$</> <userinput>initdb -D /usr/local/pgsql/data</userinput> +<prompt>$</> <userinput>initdb -D /usr/local/pgsql/data --nodename foo</userinput> </screen> Note that you must execute this command while logged into the - <productname>PostgreSQL</productname> user account, which is - described in the previous section. + <productname>Postgres-XL</productname> user account, which is + described in the previous section. You should assign + separate <firstterm>data directory</> to + each <firstterm>Coordinator</> and <firstterm>Datanode</> if you + are configuring them in a same server. </para> <tip> @@ -91,6 +104,16 @@ the environment variable <envar>PGDATA</envar>. <indexterm><primary><envar>PGDATA</envar></primary></indexterm> </para> + + <para> + In <productname>Postgres-XL</>, if you configure multiple <firstterm>Coordinator</> + and/or <firstterm>Datanode</>, you cannot + share <envar>PGDATA</envar> among them and you must + specify <firstterm>data directory</> explicitly. + </para> + <para> + <option>--nodename</option> is mandatory for all the nodes at initialization + </para> </tip> <para> @@ -98,7 +121,7 @@ the <xref linkend="app-pg-ctl"> program<indexterm><primary>pg_ctl</></> like so: <screen> -<prompt>$</> <userinput>pg_ctl -D /usr/local/pgsql/data initdb</userinput> +<prompt>$</> <userinput>pg_ctl -D /usr/local/pgsql/data -o '--nodename foo' initdb</userinput> </screen> This may be more intuitive if you are using <command>pg_ctl</command> for starting and stopping the @@ -211,7 +234,7 @@ postgres$ <userinput>initdb -D /usr/local/pgsql/data</userinput> </sect1> <sect1 id="server-start"> - <title>Starting the Database Server</title> + <title>Starting Postgres-XL Cluster</title> <para> Before anyone can access the database, you must start the database @@ -372,6 +395,570 @@ su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgs shutting down the server. </para> + <para> + As described in the previous chapter, <productname>XL</> consists of + various components. + Minimum set of components are GTM, GTM-Proxy, Coordinator and + Datanode. + You must configure and start each of them. + Following sections will give you how to configure and start them. + <filename>pgxc_clean</> and <filename>GTM-Standby</> are described in high-availability + sections. + </para> + + <sect2 id="creating-databases"> + <title>Creating Databases</title> + <para> + You should initialize each database which + composes <productname>Postgres-XL</> database cluster system. + Both Coordinator and Datanode has its own database and you should + initialize these database. + Coordinator holds just database catalog and temporary data store. + Datanode holds most of your data. + First of all, you should determine how many Coordinators/Datanodes + to run and where they should run. + It is a good convention that you run a Coordinator where you run a + Datanode. + In this case, you should run <filename>GTM-Proxy</> on the same + server too. + It simplifies <productname>XL</> configuration and help to make + workload of each servers even. + </para> + + <para> + Both Coordinator and Datanode have their + own databases, essentially <productname>PostgreSQL</> databases. + They are separate and you should initialize them separately. + </para> + </sect2> + + <sect2 id="gtm-start"> + <title>Starting GTM</title> + <para> + GTM provides global transaction management feature to all the + other components in <productname>Postgres-XL</> database cluster. + Because GTM handles transaction requirements from all + the Coordinators and Datanodes, it is highly advised to run this + in a separate server. + </para> + + <para> + Before you start GTM, you should decide followings: + </para> + + <variablelist> + + <varlistentry> + <term>Where to run GTM</term> + <listitem> + + <para> + Because GTM receives all the request to begin/end + transactions and to refer to sequence values, you should + run GTM in a separate server. If you + run GTM in the same server as Datanode or + Coordinator, it will become harder to make workload reasonably + balanced. + </para> + <para> + Then, you should determine GTM's working directory. + Please create this directory before you run GTM. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Listen address and port of GTM</term> + <listitem> + <para> + Next, you should determine listen address and port + of GTM. + Listen address can be either the IP address or host name which + receives request from other component, + typically <filename>GTM-Proxy</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>GTM id</term> + <listitem> + <para> + You have a chance to run more than one GTM in + one <productname>Postgres-XL</> cluster. + For example, if you need a backup of GTM in + high-availability environment, you need to run + two GTMs. + You should give unique GTM id to each of such GTMs. + GTM id value begins with one. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <para> + When this is determined, you can initialize GTM with the command <xref + linkend="app-initgtm">, + for example: +<screen> +<prompt>$</> <userinput>initgtm -Z gtm -D /usr/local/pgsql/data_gtm</userinput> +</screen> + </para> + + <para> + All the parameters related to GTM can be modified in <filename>gtm.conf</filename> + located in data folder initialized by <command>initgtm</command>. + </para> + + <para> + Then you can start GTM as follows: +<!-- Check precise parameters --> +<screen> +<prompt>$</> <userinput>gtm -D /usr/local/pgsql/data_gtm</userinput> +</screen> + where <option>-D</> option specifies working directory of GTM. + </para> + <para> + Alternatively, GTM can be started using <command>gtm_ctl</>, for example: +<screen> +<prompt>$</> <userinput>gtm_ctl -Z gtm start -D /usr/local/pgsql/data_gtm</userinput> +</screen> + </para> + + </sect2> + <sect2 id="gtm-proxy-start"> + <title>Starting GTM-Proxy</title> + <para> + GTM-Proxy is not a mandatory component of Postgres-XL cluster but + it can be used to group messages between GTM and cluster nodes, + reducing workload and the number of packages exchanged through network. + </para> + + <para> + As described in the previous section, <filename>GTM-Proxy</> needs + its own listen address, port, working directory and GTM-Proxy ID, + which should be unique and begins with one. + In addition, you should determine how many working threads to + run. + You should also use GTM's address and port to start <filename>GTM-Proxy</>. + </para> + + <para> + Then, you need first to initialize GTM-Proxy with <command>initgtm</command>, + for example: +<screen> +<prompt>$</> <userinput>initgtm -Z gtm_proxy -D /usr/local/pgsql/data_gtm_proxy</userinput> +</screen> + </para> + + <para> + All the parameters related to GTM-Proxy can be modified in <filename>gtm_proxy.conf</filename> + located in data folder initialized by <command>initgtm</command>. + </para> + + <para> + Then, you can start <filename>GTM-Proxy</> like: +<screen> +<prompt>$</> <userinput>gtm_proxy -D /usr/local/pgsql/data_gtm_proxy</userinput> +</screen> + where <option>-D</> specifies <filename>GTM-Proxy</>'s working + directory. + </para> + + <para> + Alternatively, you can start GTM-Proxy using <filename>gtm_ctl</> + as follows: +<screen> +<prompt>$</> <userinput>gtm_ctl start -Z gtm_proxy -D /usr/local/pgsql/data_gtm_proxy</userinput> +</screen> + </para> + + </sect2> + + <sect2 id="Datanode-configuration"> + <title>Configuring Datanodes</title> + <para> + Before starting Coordinator or Datanode, you must configure them. + You can configure Coordinator or Datanode by + editing <filename>postgresql.conf</> file located at their working + directory as you specified by <option>-D</> option + in <filename>initdb</> command. + </para> + + <para> + Datanode is almost native <productname>PostgreSQL</> with some + extension. + Additional options in <filename>postgresql.conf</> for the + Datanode are as follows: + </para> + + <variablelist> + + <varlistentry> + <term>max_connections</term> + <listitem> + + <para> + This value is not just a number of connections you expect to each + Coordinator. + Each Coordinator backend has a chance to connect to all the + Datanode. + You should specify number of total connections whole Coordinator + may accept. + For example, if you have five Coordinators and each of them may + accept forty connections, you should specify 200 as this + parameter value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>max_prepared_transactions</term> + <listitem> + <para> + Even though your application does not intend to + issue <command>PREPARE TRANSACTION</>, Coordinator may issue this + internally when more than one Datanode are involved. + You should specify this parameter the same value + as <filename>max_connections</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>pgxc_node_name</term> + <listitem> + <para> + GTM needs to identify each Datanode, as specified by + this parameter. + The value should be unique and start with one. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>port</term> + <listitem> + <para> + Because both Coordinator and Datanode may run on the same server, + you may want to assign separate port number to the Datanode. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>gtm_port</term> + <listitem> + <para> + Specify the port number of GTM-Proxy, as specified + in <option>-p</> option in <command>gtm_proxy</> + or <command>gtm_ctl</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>gtm_host</term> + <listitem> + <para> + Specify the host name or IP address of GTM-Proxy, as specified + in <option>-h</> option in <command>gtm_proxy</> + or <command>gtm_ctl</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>shared_queues</term> + <listitem> + <para> + For some joins that occur in queries, data from one Datanode may + need to be joined with data from another Datanode. + <productname>Postgres-XL</productname> uses shared queues for this purpose. + During execution each Datanode knows if it needs to produce or consume + tuples, or both. + </para> + <para> + Note that there may be mulitple shared_queues used even for a single + query. So a value should be set taking into account the number of + connections it can accept and expected number of such joins occurring + simultaneously. + </para> + + </listitem> + </varlistentry> + + <varlistentry> + <term>shared_queue_size</term> + <listitem> + <para> + This parameter sets the size of each each shared queue allocated. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </sect2> + + <sect2 id="Coordinator-configuration"> + <title>Configuring Coordinators</title> + <para> + Although Coordinator and Datanode shares the same binary, their + configuration is a little different due to their functionalities. + </para> + + <variablelist> + + <varlistentry> + <term>max_connections</term> + <listitem> + <para> + You don't have to take other Coordinator or Datanode into + account. + Just specify the number of connections the Coordinator accepts + from applications. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>max_prepared_transactions</term> + <listitem> + <para> + Specify at least total number of Coordinators in the cluster. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>pgxc_node_name</term> + <listitem> + <para> + GTM needs to identify each Datanode, as specified by + this parameter. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>port</term> + <listitem> + <para> + Because both Coordinator and Datanode may run on the same server, + you may want to assign separate port number to the Coordinator. + It may be convenient to use default value of PostgreSQL listen + port. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>gtm_port</term> + <listitem> + <para> + Specify the port number of GTM-Proxy, as specified + in <option>-p</> option in <command>gtm_proxy</> + or <command>gtm_ctl</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>gtm_host</term> + <listitem> + <para> + Specify the host name or IP address of GTM-Proxy, as specified + in <option>-h</> option in <command>gtm_proxy</> + or <command>gtm_ctl</>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>pooler_port</term> + <listitem> + <para> + Specify the port number that the pooler should use. This must not + conflict with any other server ports used on this host. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>max_pool_size</term> + <listitem> + <para> + Coordinator maintains connections to Datanode as a pool. + This parameter specifies max number of connections the + Coordinator maintains. + Specify <option>max_connection</> value of remote nodes as this + parameter value. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>min_pool_size</term> + <listitem> + <para> + This is the minimum number of Coordinator to remote node connections + maintained by the pooler. + Typically specify 1. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>pool_conn_keepalive</term> + <listitem> + <para> + This parameter specifies how long to keep the connection alive. + If older than this amount, the pooler discards the connection. + This parameter is useful in multi-tenant environments where + many connections to many different databases may be used, + so that idle connections may cleaned up. It is also useful + for automatically closing connections occasionally in case + there is some unknown memory leak so that this memory can be freed. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>pool_maintenance_timeout</term> + <listitem> + <para> + This parameter specifies how long to wait until pooler + maintenance is performed. During such maintenance, + old idle connections are discarded. + This parameter is useful in multi-tenant environments where + many connections to many different databases may be used, + so that idle connections may cleaned up. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>remote_query_cost</term> + <listitem> + <para> + This parameter specifies the cost overhead of setting up + a remote query to obtain remote data. It is used by + the planner in costing queries. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>network_byte_cost</term> + <listitem> + <para> + This parameter is used in query cost planning to + estimate the cost involved in row shipping and obtaining + remote data based on the expected data size. + Row shipping is expensive and adds latency, so this + setting helps to favor plans that minimizes row shipping. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>sequence_range</term> + <listitem> + <para> + This parameter is used to get several sequence values + at once from GTM. This greatly speeds up COPY and INSERT SELECT + operations where the target table uses sequences. + <productname>Postgres-XL</productname> will not use this entire + amount at once, but will increase the request size over + time if many requests are done in a short time frame in + the same session. + After a short time without any sequence requests, decreases back down to 1. + Note that any settings here are overriden if the CACHE clause was + used in <xref linkend='sql-createsequence'> or <xref linkend='sql-altersequence'>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>max_coordinators</term> + <listitem> + <para> + This is the maximum number of Coordinators that can be configured in the cluster. + Specify exact number if it is not planned to add more Coordinators + while cluster is running, or greater, if it is desired to dynamically + resize cluster. It costs about 140 bytes of shared memory per slot. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>max_datanodes</term> + <listitem> + <para> + This is the maximum number of Datanodes configured in the cluster. + Specify exact number if it is not planned to add more Datanodes + while cluster is running, or greater, if it is desired to dynamically + resize cluster. It costs about 140 bytes of shared memory per slot. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>enforce_two_phase_commit</term> + <listitem> + <para> + Enforce the usage of two-phase commit on transactions involving ON COMMIT actions + or temporary objects. Usage of autocommit instead of two-phase commit may break data + consistency so use at your own risk. + </para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + <sect2 id="starting-Datanode"> + <title>Starting Datanodes</title> + + <para> + Now you can start central component + of <productname>Postgres-XL</>, Datanode and Coordinator. + If you're familiar with starting <productname>PostgreSQL</> + database server, this step is very similar to <productname>PostgreSQL</>. + </para> + + <para> + You can start a Datanode as follows: +<screen> +<prompt>$</> <userinput>postgres --datanode -D /usr/local/pgsql/data</userinput> +</screen> + <option>--datanode</> specifies <command>postgres</> should run as a + Datanode. You may need to specify <option>-i</> <command>postgres</> to + accept connection from TCP/IP connections or edit <filename>pg_hba.conf</filename> + if cluster uses nodes among several servers. + </para> + + </sect2> + + <sect2 id="starting-Coordinator"> + <title>Starting Coordinators</title> + <para> + You can start a Coordinator as follows: +<screen> +<prompt>$</> <userinput>postgres --coordinator -D /usr/local/pgsql/Datanode</userinput> +</screen> + <option>--coordinator</> specifies <command>postgres</> should run as a + Coordinator. You may need to specify <option>-i</> <command>postgres</> to + accept connection from TCP/IP connections or edit <filename>pg_hba.conf</filename> + if cluster uses nodes among several servers. + </para> + + </sect2> + + + <sect2 id="server-start-failures"> <title>Server Start-up Failures</title> @@ -1396,6 +1983,16 @@ $ <userinput>sysctl -w vm.nr_hugepages=3170</userinput> </indexterm> <para> + This section describes how to shutdown each Coordinator and + Datanode. + Please note that you should shutdown Coordinator first and then + Datanodes, GTM-Proxy and GTM. + </para> + + <sect2 id="Coordinator-Datanode-shutdown"> + <title>Shutting Down Coordinators and Datanodes</title> + + <para> There are several ways to shut down the database server. You control the type of shutdown by sending different signals to the master <command>postgres</command> process. @@ -1488,6 +2085,31 @@ $ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput <systemitem>SIGTERM</> signal to the child process associated with the session. </para> + </sect2> + <sect2 id="shutting-down-gtm-proxy"> + <title>Shutting Down GTM-Proxies</title> + + <para> + You can shut down GTM-Proxies by sending SIGKILL, SIGTERM, SIGQUIT, + SIGINT or SIGHUP to each GTM-Proxy process. + GTM-Proxy process number is recorded in the + file <filename>gtm_proxy.pid</> located in each GTM-Proxy's working + directory. + </para> + + </sect2> + + <sect2 id="shutting-down-gtm"> + <title>Shutting Down GTM</title> + <para> + You can shut down GTM by sending SIGKILL, SIGTERM, SIGQUIT, + SIGINT or SIGHUP to GTM process. + GTM process number is recorded in the + file <filename>gtm.pid</> located in GTM's working + directory. + </para> + + </sect2> </sect1> <sect1 id="upgrading"> @@ -1508,6 +2130,13 @@ $ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput </para> <para> + Because <productname>Postgres-XL</>'s Coordinators and Datanodes + are essentially <productname>PostgreSQL</> servers, you can follw + the steps described below to upgrade each of them. Please note + that you should do this manually. + </para> + + <para> <productname>PostgreSQL</> major versions are represented by the first two digit groups of the version number, e.g., 8.4. <productname>PostgreSQL</> minor versions are represented by the @@ -1548,6 +2177,15 @@ $ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput following categories of possible changes: </para> + <note> + <para> + Support for <link linkend="pgupgrade">pg_upgrade</link> is not tested + in <productname>Postgres-XL</productname>. Users are advised to use other + methods for upgrading to next major version of + <productname>Postgres-XL</productname>. + </para> + </note> + <variablelist> <varlistentry> diff --git a/doc/src/sgml/sourcerepo.sgml b/doc/src/sgml/sourcerepo.sgml index d82706b40b..08cafe129b 100644 --- a/doc/src/sgml/sourcerepo.sgml +++ b/doc/src/sgml/sourcerepo.sgml @@ -52,7 +52,7 @@ To begin using the Git repository, make a clone of the official mirror: <programlisting> -git clone git://git.postgresql.org/git/postgresql.git +git clone git://git.postgresql.org/git/postgres-xl.git </programlisting> This will copy the full repository to your local machine, so it may take @@ -67,7 +67,7 @@ git clone git://git.postgresql.org/git/postgresql.git prefix to <literal>http</>, as in: <programlisting> -git clone http://git.postgresql.org/git/postgresql.git +git clone http://git.postgresql.org/git/postgres-xl.git </programlisting> The HTTP protocol is less efficient than the Git protocol, so it will be diff --git a/doc/src/sgml/sql.sgml b/doc/src/sgml/sql.sgml index 57396d7c24..315eb0b7f7 100644 --- a/doc/src/sgml/sql.sgml +++ b/doc/src/sgml/sql.sgml @@ -1468,6 +1468,11 @@ SELECT S.SNO, S.SNAME, COUNT(SE.PNO) <title>Subqueries</title> <para> + Subqueries in <productname>Postgres-XL</> have restrictions. + You may be warned if some of them cannot be handled. + </para> + + <para> In the WHERE and HAVING clauses the use of subqueries (subselects) is allowed in every place where a value is expected. In this case the value must be derived by evaluating the subquery first. The usage of diff --git a/doc/src/sgml/start.sgml b/doc/src/sgml/start.sgml index 342fdad91c..5df5daa93b 100644 --- a/doc/src/sgml/start.sgml +++ b/doc/src/sgml/start.sgml @@ -64,10 +64,77 @@ </para> <para> - In database jargon, <productname>PostgreSQL</productname> uses a - client/server model. A <productname>PostgreSQL</productname> - session consists of the following cooperating processes - (programs): + <productname>Postgres-XL</>, in short, is a collection + of <productname>PostgreSQL</> database clusters which act as if the whole + collection is a single database cluster. Based on your database design, + each table is replicated or distributed among member databases. + </para> + + <para> + To provide this capability, <productname>Postgres-XL</> is + composed of three major components called the GTM, Coordinator and + Datanode. The GTM is responsible to provide ACID property of + transactions. The Datanode stores table data and handle SQL statements + locally. The Coordinator handles each SQL statements from + applications, determines which Datanode to go, and sends plans on + to the appropriate Datanodes. + </para> + + <para> + You usually should run GTM on a separate server because GTM has to take + care of transaction requirements from all the Coordinators and + Datanodes. To group multiple requests and responses from + Coordinator and Datanode processes running on the same server, you can + configure GTM-Proxy. GTM-Proxy reduces the number of interactions + and the amount of data to GTM. GTM-Proxy also helps handle + GTM failures. + </para> + + <para> + It is often good practice to run both Coordinator and Datanode on the + same server because we don't have to worry about workload balance + between the two, and you can often get at data from replicated tables locally + without sending an additional request out on the network. + You can have any number of servers where these + two components are running. Because both Coordinator and Datanode + are essentially PostgreSQL instances, you should configure them to + avoid resource conflict. It is very important to assign them + different working directories and port numbers. + </para> + + <para> + Postgres-XL allows multiple Coordinators to accept statements + from applications independently but in an integrated way. Any + writes from any Coordinator is available from any other + Coordinators. They acts as if they are single database. + The Coordinator's role is to accept statements, find what Datanodes are + involved, send query plans on to the appropriate Datanodes if + needed, collect the results + and write them back to applications. + </para> + + <para> + The Coordinator does not store any user data. It stores only catalog + data to determine how to process statements, where the target + Datanodes are, among others. Therefore, you don't have to worry + about Coordinator failure much. When the Coordinator fails, you + can just switch to the other one. + </para> + + <para> + The GTM could be single point of failure (SPOF). To prevent this, you + can run another GTM as GTM-Standby to backup GTM's status. When + GTM fails, GTM-Proxy can switch to the standby on the fly. This + will be described in detail in high-availability sections. + </para> + + <para> + As described above, the Coordinators and Datanodes + of <productname>Postgres-XL</> are + essentially <productname>PostgreSQL</> database servers. In database + jargon, <productname>PostgreSQL</productname> uses a client/server + model. A <productname>PostgreSQL</productname> session consists + of the following cooperating processes (programs): <itemizedlist> <listitem> diff --git a/doc/src/sgml/trigger.sgml b/doc/src/sgml/trigger.sgml index 52f28bca7a..77b9e007dc 100644 --- a/doc/src/sgml/trigger.sgml +++ b/doc/src/sgml/trigger.sgml @@ -8,6 +8,10 @@ </indexterm> <para> + In <productname>Postgres-XL</>, triggers are not currently supported. + </para> + + <para> This chapter provides general information about writing trigger functions. Trigger functions can be written in most of the available procedural languages, including diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml index e3941c9391..ddcdfdc2d6 100644 --- a/doc/src/sgml/wal.sgml +++ b/doc/src/sgml/wal.sgml @@ -785,4 +785,70 @@ seem to be a problem in practice. </para> </sect1> + <sect1 id="barriers"> + <title>Barrier</title> + <para> + Its important to ensure that any global transactions in the cluster + must either be committed on all the nodes or none of the nodes. If + global recovery is done in a way such that on one node the + <acronym>WAL</acronym> recovery stops after a commit record of some + global transaction is processed, but stops before the commit record for + the same transaction is processed on some other node, the cluster may + be left in an inconsistent state. Since the commit messages of global + transactions can arrive out-of-order on different nodes, its very hard + to find a common synchronization point. For example, for two global + transactions T1 and T2, the commit messages for these transactions + can arrive on nodes N1 and N2 such that N1 receives commit message + for T1 first and N2 receives commit message T2 first. In this case, + during <acronym>PITR</acronym>, irrespective of whether we stop at + T1 or T2, the cluster would lose its consistency since at least one + transaction will be marked as committed on one node and as aborted + on the other node. + </para> + + <para> + During recovery, it can be hard and even impossible to find + cluster-wide consistent synchronization points. In fact, such + synchronization points may not exists at all. Postgres-XL provides + a mechanism to create such synchronization points, called barriers, + during normal operation. A barrier can be created by using a SQL + command <command>BARRIER</command> + </para> + + <para> + The user must connect to one of the Coordinators and issue the + <command>BARRIER</command> command, optionally followed by an + identifier. If the user does not specify an identifier, an unique + identifier will be generated and returned to the caller. Upon + receiving the <command>BARRIER</command> command, the Coordinator + temporarily pauses any new two-phase commits. It also communicates with + other Coordinators to ensure that there are no in-progress two-phase + commits in the cluster. At that point, a barrier <acronym>WAL</acronym> + record along with the user-given or system-generated BARRIER identifier + is writtent to the <acronym>WAL</acronym> stream of all Datanodes and + the Coordinators. + </para> + + <para> + The user can create as many barriers as she wants to. At the time of + point-in-time-recovery, the same barrier id must be specified in the + <filename>recovery.conf</filename> files of all the Coordinators and + Datanodes. When every node in the cluster recovers to the same barrier + id, a cluster-wide consistent state is reached. Its important that + the recovery must be started from a backup taken before the barrier + was generated. If no matching barrier record is found, either because + the barrier was created before the base backup used for the recovery + or the said barrier was never created, the recovery is run to the end. + </para> + + <para> + Note that in the current release, Postgres-XL has no means to detect + if another barrier with the same name already exists. So the user + must be careful while choosing a barrier identifier and ensure + that an unique identifier is used for every barrier. Similarly, at + the time of recovery, the user must specify the same barrier id in the + <filename>recovery.conf</filename> file on all the nodes. Otherwise, + the cluster would not recover to a global consistent image. + </para> + </sect1> </chapter> diff --git a/doc/src/sgml/xaggr.sgml b/doc/src/sgml/xaggr.sgml index ef7cff4879..782c86942b 100644 --- a/doc/src/sgml/xaggr.sgml +++ b/doc/src/sgml/xaggr.sgml @@ -9,27 +9,27 @@ </indexterm> <para> - Aggregate functions in <productname>PostgreSQL</productname> - are defined in terms of <firstterm>state values</firstterm> - and <firstterm>state transition functions</firstterm>. - That is, an aggregate operates using a state value that is updated - as each successive input row is processed. + Aggregate functions in <productname>Postgres-XL</productname> + are expressed in terms of <firstterm>state values</firstterm> + and <firstterm>state functions namely transition function and collection + function</firstterm>. + That is, an aggregate operates using 1. a transition state value that is updated + as each successive input row, in a given set of rows, is processed, and + 2. (optionally) collection state value that is updated as each successive + transition state value is processed. To define a new aggregate function, one selects a data type for the state value, - an initial value for the state, and a state transition - function. The state transition function takes the previous state - value and the aggregate's input value(s) for the current row, and - returns a new state value. - A <firstterm>final function</firstterm> + an initial value for the transition state, and a state transition + function. The state transition function is just an + ordinary function that could also be used outside the + context of the aggregate. A collection function and an initial value for + the collection state can also be specified, if one wants to take advantage of + distributed aggregation. Similar to transition function, a collection + function can be an ordinary function that could also be used outside the + context of the aggregate. A <firstterm>final function</firstterm> can also be specified, in case the desired result of the aggregate is different from the data that needs to be kept in the running - state value. The final function takes the last state value - and returns whatever is wanted as the aggregate result. - In principle, the transition and final functions are just ordinary - functions that could also be used outside the context of the - aggregate. (In practice, it's often helpful for performance reasons - to create specialized transition functions that can only work when - called as part of an aggregate.) + state (either collection or transition) value. </para> <para> @@ -93,11 +93,60 @@ SELECT sum(a) FROM test_complex; </para> <para> - Another bit of default behavior for a <quote>strict</> transition function + In <productname>Postgres-XL</productname>, a user can provide collection + function if distributed aggregation is expected for improving performance. The + collection function essentially combines the state transition results produced + at different Datanodes. Without a final function the result produced by the + collection function is the result of aggregate. Above the definition of aggregate + <function>sum</> for complex numeric data types can be modified to have a + collection function as follows + +<screen> +CREATE AGGREGATE sum (complex) +( + sfunc = complex_add, + stype = complex, +cfunc = complex_add, + initcond = '(0,0)' +initcollect = '(0.0)' +); + +SELECT sum(a) FROM test_complex; + + sum +----------- + (34,53.9) +</screen> + + Notice that both the CREATE AGGREGATE commands work in + <productname>Postgres-XL</productname>. An aggregate created by either command + produces the same results. + </para> + + <para> + The above definitions of <function>sum</function> will return zero (the initial + state condition) if there are no nonnull input values. + Perhaps we want to return null in that case instead — the SQL standard + expects <function>sum</function> to behave that way. We can do this simply by + omitting the <literal>initcond</literal> phrase, so that the initial state + condition is null. Ordinarily this would mean that the <literal>sfunc</literal> + would need to check for a null state-condition input, but for + <function>sum</function> and some other simple aggregates like + <function>max</> and <function>min</>, + it is sufficient to insert the first nonnull input value into + the state variable and then start applying the transition function + at the second nonnull input value. <productname>Postgres-XL</productname> + will do that automatically if the initial condition is null and + the transition function is marked <quote>strict</> (i.e., not to be called + for null inputs). + </para> + + <para> + Another bit of default behavior for a <quote>strict</> transition/collection function is that the previous state value is retained unchanged whenever a null input value is encountered. Thus, null values are ignored. If you need some other behavior for null inputs, do not declare your - transition function as strict; instead code it to test for null inputs and + transition/collection function as strict; instead code it to test for null inputs and do whatever is needed. </para> @@ -116,15 +165,18 @@ CREATE AGGREGATE avg (float8) ( sfunc = float8_accum, stype = float8[], + cfunc = float8_collect, finalfunc = float8_avg, - initcond = '{0,0,0}' + initcond = '{0,0}' + initcollect = '{0,0,0}' ); </programlisting> </para> <note> <para> - <function>float8_accum</> requires a three-element array, not just + <function>float8_accum</> and <function>float8_collect</> + require a three-element array, not just two elements, because it accumulates the sum of squares as well as the sum and count of the inputs. This is so that it can be used for some other aggregates as well as <function>avg</>. diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 9c15950ccd..ca64ccc948 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -705,6 +705,12 @@ DROP FUNCTION sum_n_product (int, int); </indexterm> <para> + It is highly recommended not to use <literal>VARIADIC</> in the + current <productname>Postgres-XL</>. It is not well reviewed and + may return wrong result. + </para> + + <para> <acronym>SQL</acronym> functions can be declared to accept variable numbers of arguments, so long as all the <quote>optional</> arguments are of the same data type. The optional arguments will be |