From 185364b161512806d23ca390f5b615666079699b Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Tue, 30 May 2017 11:47:19 -0400 Subject: doc: Fix ALTER PUBLICATION details Some of the text was made nonsensical by commit e9500240661c03750923e6f539bfa2d75cfaa32a. Fix that and make some other minor changes. Reported-by: Jeff Janes --- doc/src/sgml/ref/alter_publication.sgml | 34 ++++++++++++++++++++++---------- doc/src/sgml/ref/alter_subscription.sgml | 3 +++ 2 files changed, 27 insertions(+), 10 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/alter_publication.sgml b/doc/src/sgml/ref/alter_publication.sgml index 7b8f114f54..cc5d92f041 100644 --- a/doc/src/sgml/ref/alter_publication.sgml +++ b/doc/src/sgml/ref/alter_publication.sgml @@ -34,28 +34,40 @@ ALTER PUBLICATION name RENAME TO Description - The first variant of this command listed in the synopsis can change + The command ALTER PUBLICATION can change the attributes + of a publication. + + + + The first three variants change which tables are part of the publication. + The SET TABLE clause will replace the list of tables in + the publication with the specified one. The ADD TABLE + and DROP TABLE clauses will add and remove one or more + tables from the publication. Note that adding tables to a publication that + is already subscribed to will require a ALTER SUBSCRIPTION + ... REFRESH PUBLICATION action on the subscribing side in order + to become effective. + + + + The fourth variant of this command listed in the synopsis can change all of the publication properties specified in . Properties not mentioned in the command retain their previous settings. + The remaining variants change the owner and the name of the publication. + + + + You must own the publication to use ALTER PUBLICATION. To alter the owner, you must also be a direct or indirect member of the new owning role. The new owner must have CREATE privilege on the database. Also, the new owner of a FOR ALL TABLES publication must be a superuser. However, a superuser can change the ownership of a publication while circumventing these restrictions. - - - The other variants of this command deal with the table membership of the - publication. The SET TABLE clause will replace the - list of tables in the publication with the specified one. - The ADD TABLE and - DROP TABLE will add and remove one or more tables from - the publication. - @@ -147,6 +159,8 @@ ALTER PUBLICATION mypublication ADD TABLE users, departments; + + diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml index 113e32bfd0..a3471a0442 100644 --- a/doc/src/sgml/ref/alter_subscription.sgml +++ b/doc/src/sgml/ref/alter_subscription.sgml @@ -42,8 +42,11 @@ ALTER SUBSCRIPTION name RENAME TO < + You must own the subscription to use ALTER SUBSCRIPTION. To alter the owner, you must also be a direct or indirect member of the new owning role. The new owner has to be a superuser. + (Currently, all subscription owners must be superusers, so the owner checks + will be bypassed in practice. But this might change in the future.) -- cgit v1.2.3 From 3e6d2fabccef5ed602cd248bfbedf4dc9a57eb09 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 31 May 2017 13:39:28 -0400 Subject: doc: Add another migration item to release notes --- doc/src/sgml/release-10.sgml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) (limited to 'doc/src') diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml index 3ccac676ad..b10086bdb0 100644 --- a/doc/src/sgml/release-10.sgml +++ b/doc/src/sgml/release-10.sgml @@ -165,6 +165,26 @@ + + Change how logical replication + uses pg_hba.conf. + + + + In previous releases, a logical replication connection required + the replication keyword in the database column. As + of this release, logical replication matches a normal entry with a + database name or keywords such as all. Physical + replication continues to use the replication keyword. + Since built-in logical replication is new in this release as well, this + change only affects users of third-party logical replication plugins. + + + + + -- cgit v1.2.3 From e9a3c047a5fc701d2efb776be2b351645ea001c8 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Thu, 1 Jun 2017 13:32:55 -0400 Subject: Always use -fPIC, not -fpic, when building shared libraries with gcc. On some platforms, -fpic fails for sufficiently large shared libraries. We've mostly not hit that boundary yet, but there are some extensions such as Citus and pglogical where it's becoming a problem. A bit of research suggests that the penalty for -fPIC is small, in the single-digit-percentage range --- and there's none at all on popular platforms such as x86_64. So let's just default to -fPIC everywhere and provide one less thing for extension developers to worry about. Per complaint from Christoph Berg. Back-patch to all supported branches. (I did not bother to touch the recently-removed Makefiles for sco and unixware in the back branches, though. We'd have no way to test that it doesn't break anything on those platforms.) Discussion: https://postgr.es/m/20170529155850.qojdfrwkkqnjb3ap@msg.df7cb.de --- doc/src/sgml/dfunc.sgml | 26 ++++++++++++-------------- src/makefiles/Makefile.linux | 7 +++---- src/makefiles/Makefile.netbsd | 4 ---- src/makefiles/Makefile.openbsd | 4 ---- 4 files changed, 15 insertions(+), 26 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/dfunc.sgml b/doc/src/sgml/dfunc.sgml index 6a4b7d6e97..23af270e32 100644 --- a/doc/src/sgml/dfunc.sgml +++ b/doc/src/sgml/dfunc.sgml @@ -63,10 +63,10 @@ The compiler flag to create PIC is - . To create shared libraries the compiler + . To create shared libraries the compiler flag is . -gcc -fpic -c foo.c +gcc -fPIC -c foo.c gcc -shared -o foo.so foo.o This is applicable as of version 3.0 of @@ -84,14 +84,14 @@ gcc -shared -o foo.so foo.o The compiler flag of the system compiler to create PIC is . When using - GCC it's . The + GCC it's . The linker flag for shared libraries is . So: cc +z -c foo.c or: -gcc -fpic -c foo.c +gcc -fPIC -c foo.c and then: @@ -112,13 +112,11 @@ ld -b -o foo.sl foo.o The compiler flag to create PIC is - . On some platforms in some situations - must be used if - does not work. Refer to the GCC manual for more information. + . The compiler flag to create a shared library is . A complete example looks like this: -cc -fpic -c foo.c +cc -fPIC -c foo.c cc -shared -o foo.so foo.o @@ -149,12 +147,12 @@ cc -bundle -flat_namespace -undefined suppress -o foo.so foo.o The compiler flag to create PIC is - . For ELF systems, the + . For ELF systems, the compiler with the flag is used to link shared libraries. On the older non-ELF systems, ld -Bshareable is used. -gcc -fpic -c foo.c +gcc -fPIC -c foo.c gcc -shared -o foo.so foo.o @@ -169,10 +167,10 @@ gcc -shared -o foo.so foo.o The compiler flag to create PIC is - . ld -Bshareable is + . ld -Bshareable is used to link shared libraries. -gcc -fpic -c foo.c +gcc -fPIC -c foo.c ld -Bshareable -o foo.so foo.o @@ -188,7 +186,7 @@ ld -Bshareable -o foo.so foo.o The compiler flag to create PIC is with the Sun compiler and - with GCC. To + with GCC. To link shared libraries, the compiler option is with either compiler or alternatively with GCC. @@ -198,7 +196,7 @@ cc -G -o foo.so foo.o or -gcc -fpic -c foo.c +gcc -fPIC -c foo.c gcc -G -o foo.so foo.o diff --git a/src/makefiles/Makefile.linux b/src/makefiles/Makefile.linux index 52bf0b1e2b..f4f091caef 100644 --- a/src/makefiles/Makefile.linux +++ b/src/makefiles/Makefile.linux @@ -1,15 +1,14 @@ AROPT = crs + export_dynamic = -Wl,-E # Use --enable-new-dtags to generate DT_RUNPATH instead of DT_RPATH. # This allows LD_LIBRARY_PATH to still work when needed. rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtags + DLSUFFIX = .so -ifeq "$(findstring sparc,$(host_cpu))" "sparc" CFLAGS_SL = -fPIC -else -CFLAGS_SL = -fpic -endif + # Rule for building a shared library from a single .o file %.so: %.o diff --git a/src/makefiles/Makefile.netbsd b/src/makefiles/Makefile.netbsd index 31a52601af..43841c1597 100644 --- a/src/makefiles/Makefile.netbsd +++ b/src/makefiles/Makefile.netbsd @@ -9,11 +9,7 @@ endif DLSUFFIX = .so -ifeq ($(findstring sparc,$(host_cpu)), sparc) CFLAGS_SL = -fPIC -DPIC -else -CFLAGS_SL = -fpic -DPIC -endif # Rule for building a shared library from a single .o file diff --git a/src/makefiles/Makefile.openbsd b/src/makefiles/Makefile.openbsd index 7bf5493309..d8fde49d5c 100644 --- a/src/makefiles/Makefile.openbsd +++ b/src/makefiles/Makefile.openbsd @@ -7,11 +7,7 @@ endif DLSUFFIX = .so -ifeq ($(findstring sparc,$(host_cpu)), sparc) CFLAGS_SL = -fPIC -DPIC -else -CFLAGS_SL = -fpic -DPIC -endif # Rule for building a shared library from a single .o file -- cgit v1.2.3 From 3d79013b970d4cc336c06eb77ed526b44308c03e Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 31 May 2017 16:39:27 -0700 Subject: Make ALTER SEQUENCE, including RESTART, fully transactional. Previously the changes to the "data" part of the sequence, i.e. the one containing the current value, were not transactional, whereas the definition, including minimum and maximum value were. That leads to odd behaviour if a schema change is rolled back, with the potential that out-of-bound sequence values can be returned. To avoid the issue create a new relfilenode fork whenever ALTER SEQUENCE is executed, similar to how TRUNCATE ... RESTART IDENTITY already is already handled. This commit also makes ALTER SEQUENCE RESTART transactional, as it seems to be too confusing to have some forms of ALTER SEQUENCE behave transactionally, some forms not. This way setval() and nextval() are not transactional, but DDL is, which seems to make sense. This commit also rolls back parts of the changes made in 3d092fe540 and f8dc1985f as they're now not needed anymore. Author: Andres Freund Discussion: https://postgr.es/m/20170522154227.nvafbsm62sjpbxvd@alap3.anarazel.de Backpatch: Bug is in master/v10 only --- doc/src/sgml/ref/alter_sequence.sgml | 15 ++-- src/backend/commands/sequence.c | 123 +++++++-------------------- src/test/isolation/expected/sequence-ddl.out | 30 +++---- src/test/isolation/specs/sequence-ddl.spec | 19 +++-- 4 files changed, 65 insertions(+), 122 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/alter_sequence.sgml b/doc/src/sgml/ref/alter_sequence.sgml index 30e5316b8c..3a04d07ecc 100644 --- a/doc/src/sgml/ref/alter_sequence.sgml +++ b/doc/src/sgml/ref/alter_sequence.sgml @@ -171,7 +171,7 @@ ALTER SEQUENCE [ IF EXISTS ] name S The optional clause RESTART [ WITH restart ] changes the - current value of the sequence. This is equivalent to calling the + current value of the sequence. This is similar to calling the setval function with is_called = false: the specified value will be returned by the next call of nextval. @@ -182,11 +182,11 @@ ALTER SEQUENCE [ IF EXISTS ] name S - Like a setval call, a RESTART - operation on a sequence is never rolled back, to avoid blocking of - concurrent transactions that obtain numbers from the same sequence. - (The other clauses cause ordinary catalog updates that can be rolled - back.) + In contrast to a setval call, + a RESTART operation on a sequence is transactional + and blocks concurrent transactions from obtaining numbers from the + same sequence. If that's not the desired mode of + operation, setval should be used. @@ -307,8 +307,7 @@ ALTER SEQUENCE [ IF EXISTS ] name S ALTER SEQUENCE blocks concurrent nextval, currval, - lastval, and setval calls, except - if only the RESTART clause is used. + lastval, and setval calls. diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index 568b3022f2..4a56f03e8b 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -98,11 +98,9 @@ static void create_seq_hashtable(void); static void init_sequence(Oid relid, SeqTable *p_elm, Relation *p_rel); static Form_pg_sequence_data read_seq_tuple(Relation rel, Buffer *buf, HeapTuple seqdatatuple); -static LOCKMODE alter_sequence_get_lock_level(List *options); static void init_params(ParseState *pstate, List *options, bool for_identity, bool isInit, Form_pg_sequence seqform, - bool *changed_seqform, Form_pg_sequence_data seqdataform, List **owned_by); static void do_setval(Oid relid, int64 next, bool iscalled); static void process_owned_by(Relation seqrel, List *owned_by, bool for_identity); @@ -117,7 +115,6 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq) { FormData_pg_sequence seqform; FormData_pg_sequence_data seqdataform; - bool changed_seqform = false; /* not used here */ List *owned_by; CreateStmt *stmt = makeNode(CreateStmt); Oid seqoid; @@ -156,7 +153,7 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq) } /* Check and set all option values */ - init_params(pstate, seq->options, seq->for_identity, true, &seqform, &changed_seqform, &seqdataform, &owned_by); + init_params(pstate, seq->options, seq->for_identity, true, &seqform, &seqdataform, &owned_by); /* * Create relation (and fill value[] and null[] for the tuple) @@ -417,19 +414,18 @@ AlterSequence(ParseState *pstate, AlterSeqStmt *stmt) SeqTable elm; Relation seqrel; Buffer buf; - HeapTupleData seqdatatuple; + HeapTupleData datatuple; Form_pg_sequence seqform; - Form_pg_sequence_data seqdata; - FormData_pg_sequence_data newseqdata; - bool changed_seqform = false; + Form_pg_sequence_data newdataform; List *owned_by; ObjectAddress address; Relation rel; - HeapTuple tuple; + HeapTuple seqtuple; + HeapTuple newdatatuple; /* Open and lock sequence. */ relid = RangeVarGetRelid(stmt->sequence, - alter_sequence_get_lock_level(stmt->options), + ShareRowExclusiveLock, stmt->missing_ok); if (relid == InvalidOid) { @@ -447,22 +443,26 @@ AlterSequence(ParseState *pstate, AlterSeqStmt *stmt) stmt->sequence->relname); rel = heap_open(SequenceRelationId, RowExclusiveLock); - tuple = SearchSysCacheCopy1(SEQRELID, - ObjectIdGetDatum(relid)); - if (!HeapTupleIsValid(tuple)) + seqtuple = SearchSysCacheCopy1(SEQRELID, + ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(seqtuple)) elog(ERROR, "cache lookup failed for sequence %u", relid); - seqform = (Form_pg_sequence) GETSTRUCT(tuple); + seqform = (Form_pg_sequence) GETSTRUCT(seqtuple); /* lock page's buffer and read tuple into new sequence structure */ - seqdata = read_seq_tuple(seqrel, &buf, &seqdatatuple); + (void) read_seq_tuple(seqrel, &buf, &datatuple); - /* Copy old sequence data into workspace */ - memcpy(&newseqdata, seqdata, sizeof(FormData_pg_sequence_data)); + /* copy the existing sequence data tuple, so it can be modified localy */ + newdatatuple = heap_copytuple(&datatuple); + newdataform = (Form_pg_sequence_data) GETSTRUCT(newdatatuple); + + UnlockReleaseBuffer(buf); /* Check and set new values */ - init_params(pstate, stmt->options, stmt->for_identity, false, seqform, &changed_seqform, &newseqdata, &owned_by); + init_params(pstate, stmt->options, stmt->for_identity, false, seqform, + newdataform, &owned_by); /* Clear local cache so that we don't think we have cached numbers */ /* Note that we do not change the currval() state */ @@ -472,36 +472,19 @@ AlterSequence(ParseState *pstate, AlterSeqStmt *stmt) if (RelationNeedsWAL(seqrel)) GetTopTransactionId(); - /* Now okay to update the on-disk tuple */ - START_CRIT_SECTION(); - - memcpy(seqdata, &newseqdata, sizeof(FormData_pg_sequence_data)); - - MarkBufferDirty(buf); - - /* XLOG stuff */ - if (RelationNeedsWAL(seqrel)) - { - xl_seq_rec xlrec; - XLogRecPtr recptr; - Page page = BufferGetPage(buf); - - XLogBeginInsert(); - XLogRegisterBuffer(0, buf, REGBUF_WILL_INIT); - - xlrec.node = seqrel->rd_node; - XLogRegisterData((char *) &xlrec, sizeof(xl_seq_rec)); - - XLogRegisterData((char *) seqdatatuple.t_data, seqdatatuple.t_len); - - recptr = XLogInsert(RM_SEQ_ID, XLOG_SEQ_LOG); - - PageSetLSN(page, recptr); - } - - END_CRIT_SECTION(); + /* + * Create a new storage file for the sequence, making the state changes + * transactional. We want to keep the sequence's relfrozenxid at 0, since + * it won't contain any unfrozen XIDs. Same with relminmxid, since a + * sequence will never contain multixacts. + */ + RelationSetNewRelfilenode(seqrel, seqrel->rd_rel->relpersistence, + InvalidTransactionId, InvalidMultiXactId); - UnlockReleaseBuffer(buf); + /* + * Insert the modified tuple into the new storage file. + */ + fill_seq_with_data(seqrel, newdatatuple); /* process OWNED BY if given */ if (owned_by) @@ -511,10 +494,9 @@ AlterSequence(ParseState *pstate, AlterSeqStmt *stmt) ObjectAddressSet(address, RelationRelationId, relid); - if (changed_seqform) - CatalogTupleUpdate(rel, &tuple->t_self, tuple); - heap_close(rel, RowExclusiveLock); + CatalogTupleUpdate(rel, &seqtuple->t_self, seqtuple); + heap_close(rel, RowExclusiveLock); relation_close(seqrel, NoLock); return address; @@ -1219,30 +1201,6 @@ read_seq_tuple(Relation rel, Buffer *buf, HeapTuple seqdatatuple) return seq; } -/* - * Check the sequence options list and return the appropriate lock level for - * ALTER SEQUENCE. - * - * Most sequence option changes require a self-exclusive lock and should block - * concurrent nextval() et al. But RESTART does not, because it's not - * transactional. Also take a lower lock if no option at all is present. - */ -static LOCKMODE -alter_sequence_get_lock_level(List *options) -{ - ListCell *option; - - foreach(option, options) - { - DefElem *defel = (DefElem *) lfirst(option); - - if (strcmp(defel->defname, "restart") != 0) - return ShareRowExclusiveLock; - } - - return RowExclusiveLock; -} - /* * init_params: process the options list of CREATE or ALTER SEQUENCE, and * store the values into appropriate fields of seqform, for changes that go @@ -1258,7 +1216,6 @@ static void init_params(ParseState *pstate, List *options, bool for_identity, bool isInit, Form_pg_sequence seqform, - bool *changed_seqform, Form_pg_sequence_data seqdataform, List **owned_by) { @@ -1378,8 +1335,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, defel->defname); } - *changed_seqform = false; - /* * We must reset log_cnt when isInit or when changing any parameters that * would affect future nextval allocations. @@ -1420,19 +1375,16 @@ init_params(ParseState *pstate, List *options, bool for_identity, } seqform->seqtypid = newtypid; - *changed_seqform = true; } else if (isInit) { seqform->seqtypid = INT8OID; - *changed_seqform = true; } /* INCREMENT BY */ if (increment_by != NULL) { seqform->seqincrement = defGetInt64(increment_by); - *changed_seqform = true; if (seqform->seqincrement == 0) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), @@ -1442,28 +1394,24 @@ init_params(ParseState *pstate, List *options, bool for_identity, else if (isInit) { seqform->seqincrement = 1; - *changed_seqform = true; } /* CYCLE */ if (is_cycled != NULL) { seqform->seqcycle = intVal(is_cycled->arg); - *changed_seqform = true; Assert(BoolIsValid(seqform->seqcycle)); seqdataform->log_cnt = 0; } else if (isInit) { seqform->seqcycle = false; - *changed_seqform = true; } /* MAXVALUE (null arg means NO MAXVALUE) */ if (max_value != NULL && max_value->arg) { seqform->seqmax = defGetInt64(max_value); - *changed_seqform = true; seqdataform->log_cnt = 0; } else if (isInit || max_value != NULL || reset_max_value) @@ -1480,7 +1428,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, } else seqform->seqmax = -1; /* descending seq */ - *changed_seqform = true; seqdataform->log_cnt = 0; } @@ -1502,7 +1449,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, if (min_value != NULL && min_value->arg) { seqform->seqmin = defGetInt64(min_value); - *changed_seqform = true; seqdataform->log_cnt = 0; } else if (isInit || min_value != NULL || reset_min_value) @@ -1519,7 +1465,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, } else seqform->seqmin = 1; /* ascending seq */ - *changed_seqform = true; seqdataform->log_cnt = 0; } @@ -1555,7 +1500,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, if (start_value != NULL) { seqform->seqstart = defGetInt64(start_value); - *changed_seqform = true; } else if (isInit) { @@ -1563,7 +1507,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, seqform->seqstart = seqform->seqmin; /* ascending seq */ else seqform->seqstart = seqform->seqmax; /* descending seq */ - *changed_seqform = true; } /* crosscheck START */ @@ -1638,7 +1581,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, if (cache_value != NULL) { seqform->seqcache = defGetInt64(cache_value); - *changed_seqform = true; if (seqform->seqcache <= 0) { char buf[100]; @@ -1654,7 +1596,6 @@ init_params(ParseState *pstate, List *options, bool for_identity, else if (isInit) { seqform->seqcache = 1; - *changed_seqform = true; } } diff --git a/src/test/isolation/expected/sequence-ddl.out b/src/test/isolation/expected/sequence-ddl.out index 6b7119738f..6766c0aff6 100644 --- a/src/test/isolation/expected/sequence-ddl.out +++ b/src/test/isolation/expected/sequence-ddl.out @@ -13,15 +13,13 @@ step s1commit: COMMIT; step s2nv: <... completed> error in steps s1commit s2nv: ERROR: nextval: reached maximum value of sequence "seq1" (10) -starting permutation: s2begin s2nv s1alter2 s2commit s1commit -step s2begin: BEGIN; -step s2nv: SELECT nextval('seq1') FROM generate_series(1, 15); +starting permutation: s1restart s2nv s1commit +step s1restart: ALTER SEQUENCE seq1 RESTART WITH 5; +step s2nv: SELECT nextval('seq1') FROM generate_series(1, 15); +step s1commit: COMMIT; +step s2nv: <... completed> nextval -1 -2 -3 -4 5 6 7 @@ -33,14 +31,16 @@ nextval 13 14 15 -step s1alter2: ALTER SEQUENCE seq1 MAXVALUE 20; -step s2commit: COMMIT; -step s1alter2: <... completed> -step s1commit: COMMIT; +16 +17 +18 +19 starting permutation: s1restart s2nv s1commit step s1restart: ALTER SEQUENCE seq1 RESTART WITH 5; -step s2nv: SELECT nextval('seq1') FROM generate_series(1, 15); +step s2nv: SELECT nextval('seq1') FROM generate_series(1, 15); +step s1commit: COMMIT; +step s2nv: <... completed> nextval 5 @@ -58,9 +58,8 @@ nextval 17 18 19 -step s1commit: COMMIT; -starting permutation: s2begin s2nv s1restart s2commit s1commit +starting permutation: s2begin s2nv s1alter2 s2commit s1commit step s2begin: BEGIN; step s2nv: SELECT nextval('seq1') FROM generate_series(1, 15); nextval @@ -80,6 +79,7 @@ nextval 13 14 15 -step s1restart: ALTER SEQUENCE seq1 RESTART WITH 5; +step s1alter2: ALTER SEQUENCE seq1 MAXVALUE 20; step s2commit: COMMIT; +step s1alter2: <... completed> step s1commit: COMMIT; diff --git a/src/test/isolation/specs/sequence-ddl.spec b/src/test/isolation/specs/sequence-ddl.spec index 42ee3b0615..5c51fcdae6 100644 --- a/src/test/isolation/specs/sequence-ddl.spec +++ b/src/test/isolation/specs/sequence-ddl.spec @@ -15,6 +15,7 @@ setup { BEGIN; } step "s1alter" { ALTER SEQUENCE seq1 MAXVALUE 10; } step "s1alter2" { ALTER SEQUENCE seq1 MAXVALUE 20; } step "s1restart" { ALTER SEQUENCE seq1 RESTART WITH 5; } +step "s1setval" { SELECT setval('seq1', 5); } step "s1commit" { COMMIT; } session "s2" @@ -24,16 +25,18 @@ step "s2commit" { COMMIT; } permutation "s1alter" "s1commit" "s2nv" -# Prior to PG10, the s2nv would see the uncommitted s1alter change, -# but now it waits. +# Prior to PG10, the s2nv step would see the uncommitted s1alter +# change, but now it waits. permutation "s1alter" "s2nv" "s1commit" -# nextval doesn't release lock until transaction end, so s1alter2 has -# to wait for s2commit. -permutation "s2begin" "s2nv" "s1alter2" "s2commit" "s1commit" +# Prior to PG10, the s2nv step would see the uncommitted s1reset +# change, but now it waits. +permutation "s1restart" "s2nv" "s1commit" -# RESTART is nontransactional, so s2nv sees it right away +# In contrast to ALTER setval() is non-transactional, so it doesn't +# have to wait. permutation "s1restart" "s2nv" "s1commit" -# RESTART does not wait -permutation "s2begin" "s2nv" "s1restart" "s2commit" "s1commit" +# nextval doesn't release lock until transaction end, so s1alter2 has +# to wait for s2commit. +permutation "s2begin" "s2nv" "s1alter2" "s2commit" "s1commit" -- cgit v1.2.3 From f112f175a464697dd7ff5280de40dcc08d75f995 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Thu, 1 Jun 2017 17:45:53 -0400 Subject: Fix typo Reported by: Tim Goodaire Discussion: https://postgr.es/m/20170601182230.1487.26008@wrigleys.postgresql.org --- doc/src/sgml/config.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/src') diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index b51826dc68..ac1cf471f2 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -3495,7 +3495,7 @@ ANY num_sync ( ), running manually, increasing the value of the name + + Notes + + + When dropping a subscription that is associated with a replication slot on + the remote host (the normal state), DROP SUBSCRIPTION + will connect to the remote host and try to drop the replication slot as + part of its operation. This is necessary so that the resources allocated + for the subscription on the remote host are released. If this fails, + either because the remote host is not reachable or because the remote + replication slot cannot be dropped or does not exist or never existed, + the DROP SUBSCRIPTION command will fail. To proceed in + this situation, disassociate the subscription from the replication slot by + executing ALTER SUBSCRIPTION ... SET (slot_name = NONE). + After that, DROP SUBSCRIPTION will no longer attempt any + actions on a remote host. Note that if the remote replication slot still + exists, it should then be dropped manually; otherwise it will continue to + reserve WAL and might eventually cause the disk to fill up. See + also . + + + Examples -- cgit v1.2.3 From 9907b55ceb17f55bb508a1f24027a57530d84442 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 5 Jun 2017 21:37:00 -0400 Subject: Fix ALTER SUBSCRIPTION grammar ambiguity There was a grammar ambiguity between SET PUBLICATION name REFRESH and SET PUBLICATION SKIP REFRESH, because SKIP is not a reserved word. To resolve that, fold the refresh choice into the WITH options. Refreshing is the default now. Reported-by: tushar --- doc/src/sgml/catalogs.sgml | 2 +- doc/src/sgml/ref/alter_subscription.sgml | 35 ++++++++++++++++++++---------- src/backend/commands/subscriptioncmds.c | 34 +++++++++++++++++++++-------- src/backend/parser/gram.y | 14 ++---------- src/include/nodes/parsenodes.h | 1 - src/test/regress/expected/subscription.out | 2 +- src/test/regress/sql/subscription.sql | 2 +- src/test/subscription/t/001_rep_changes.pl | 2 +- 8 files changed, 54 insertions(+), 38 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index b2fae027f5..5723be744d 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -6609,7 +6609,7 @@ SCRAM-SHA-256$<iteration count>:<salt>< This catalog only contains tables known to the subscription after running either CREATE SUBSCRIPTION or - ALTER SUBSCRIPTION ... REFRESH. + ALTER SUBSCRIPTION ... REFRESH PUBLICATION. diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml index a3471a0442..bead99622e 100644 --- a/doc/src/sgml/ref/alter_subscription.sgml +++ b/doc/src/sgml/ref/alter_subscription.sgml @@ -22,7 +22,7 @@ PostgreSQL documentation ALTER SUBSCRIPTION name CONNECTION 'conninfo' -ALTER SUBSCRIPTION name SET PUBLICATION publication_name [, ...] { REFRESH [ WITH ( refresh_option [= value] [, ... ] ) ] | SKIP REFRESH } +ALTER SUBSCRIPTION name SET PUBLICATION publication_name [, ...] [ WITH ( set_publication_option [= value] [, ... ] ) ] ALTER SUBSCRIPTION name REFRESH PUBLICATION [ WITH ( refresh_option [= value] [, ... ] ) ] ALTER SUBSCRIPTION name ENABLE ALTER SUBSCRIPTION name DISABLE @@ -80,18 +80,29 @@ ALTER SUBSCRIPTION name RENAME TO < Changes list of subscribed publications. See for more information. + By default this command will also act like REFRESH + PUBLICATION. - When REFRESH is specified, this command will also act - like REFRESH - PUBLICATION. refresh_option specifies - additional options for the refresh operation, as described - under REFRESH PUBLICATION. When - SKIP REFRESH is specified, the command will not try - to refresh table information. Note that - either REFRESH or SKIP REFRESH - must be specified. + set_publication_option specifies additional + options for this operation. The supported options are: + + + + refresh (boolean) + + + When false, the command will not try to refresh table information. + REFRESH PUBLICATION should then be executed separately. + The default is true. + + + + + + Additionally, refresh options as described + under REFRESH PUBLICATION may be specified. @@ -107,7 +118,7 @@ ALTER SUBSCRIPTION name RENAME TO < - refresh_option specifies additional options for the + refresh_option specifies additional options for the refresh operation. The supported options are: @@ -185,7 +196,7 @@ ALTER SUBSCRIPTION name RENAME TO < Change the publication subscribed by a subscription to insert_only: -ALTER SUBSCRIPTION mysub SET PUBLICATION insert_only REFRESH; +ALTER SUBSCRIPTION mysub SET PUBLICATION insert_only; diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c index 86eb31df93..ad98b38efe 100644 --- a/src/backend/commands/subscriptioncmds.c +++ b/src/backend/commands/subscriptioncmds.c @@ -64,12 +64,14 @@ static void parse_subscription_options(List *options, bool *connect, bool *enabled_given, bool *enabled, bool *create_slot, bool *slot_name_given, char **slot_name, - bool *copy_data, char **synchronous_commit) + bool *copy_data, char **synchronous_commit, + bool *refresh) { ListCell *lc; bool connect_given = false; bool create_slot_given = false; bool copy_data_given = false; + bool refresh_given = false; /* If connect is specified, the others also need to be. */ Assert(!connect || (enabled && create_slot && copy_data)); @@ -92,6 +94,8 @@ parse_subscription_options(List *options, bool *connect, bool *enabled_given, *copy_data = true; if (synchronous_commit) *synchronous_commit = NULL; + if (refresh) + *refresh = true; /* Parse options */ foreach(lc, options) @@ -167,6 +171,16 @@ parse_subscription_options(List *options, bool *connect, bool *enabled_given, PGC_BACKEND, PGC_S_TEST, GUC_ACTION_SET, false, 0, false); } + else if (strcmp(defel->defname, "refresh") == 0 && refresh) + { + if (refresh_given) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("conflicting or redundant options"))); + + refresh_given = true; + *refresh = defGetBoolean(defel); + } else ereport(ERROR, (errcode(ERRCODE_SYNTAX_ERROR), @@ -315,7 +329,8 @@ CreateSubscription(CreateSubscriptionStmt *stmt, bool isTopLevel) */ parse_subscription_options(stmt->options, &connect, &enabled_given, &enabled, &create_slot, &slotname_given, - &slotname, ©_data, &synchronous_commit); + &slotname, ©_data, &synchronous_commit, + NULL); /* * Since creating a replication slot is not transactional, rolling back @@ -645,7 +660,7 @@ AlterSubscription(AlterSubscriptionStmt *stmt) parse_subscription_options(stmt->options, NULL, NULL, NULL, NULL, &slotname_given, &slotname, - NULL, &synchronous_commit); + NULL, &synchronous_commit, NULL); if (slotname_given) { @@ -680,7 +695,7 @@ AlterSubscription(AlterSubscriptionStmt *stmt) parse_subscription_options(stmt->options, NULL, &enabled_given, &enabled, NULL, - NULL, NULL, NULL, NULL); + NULL, NULL, NULL, NULL, NULL); Assert(enabled_given); if (!sub->slotname && enabled) @@ -712,13 +727,13 @@ AlterSubscription(AlterSubscriptionStmt *stmt) break; case ALTER_SUBSCRIPTION_PUBLICATION: - case ALTER_SUBSCRIPTION_PUBLICATION_REFRESH: { bool copy_data; + bool refresh; parse_subscription_options(stmt->options, NULL, NULL, NULL, NULL, NULL, NULL, ©_data, - NULL); + NULL, &refresh); values[Anum_pg_subscription_subpublications - 1] = publicationListToArray(stmt->publication); @@ -727,12 +742,13 @@ AlterSubscription(AlterSubscriptionStmt *stmt) update_tuple = true; /* Refresh if user asked us to. */ - if (stmt->kind == ALTER_SUBSCRIPTION_PUBLICATION_REFRESH) + if (refresh) { if (!sub->enabled) ereport(ERROR, (errcode(ERRCODE_SYNTAX_ERROR), - errmsg("ALTER SUBSCRIPTION ... REFRESH is not allowed for disabled subscriptions"))); + errmsg("ALTER SUBSCRIPTION with refresh is not allowed for disabled subscriptions"), + errhint("Use ALTER SUBSCRIPTION ... SET PUBLICATION ... WITH (refresh = false)."))); /* Make sure refresh sees the new list of publications. */ sub->publications = stmt->publication; @@ -754,7 +770,7 @@ AlterSubscription(AlterSubscriptionStmt *stmt) parse_subscription_options(stmt->options, NULL, NULL, NULL, NULL, NULL, NULL, ©_data, - NULL); + NULL, NULL); AlterSubscription_refresh(sub, copy_data); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 7e03624eb4..ada95e5bc3 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -9279,24 +9279,14 @@ AlterSubscriptionStmt: n->options = $6; $$ = (Node *)n; } - | ALTER SUBSCRIPTION name SET PUBLICATION publication_name_list REFRESH opt_definition - { - AlterSubscriptionStmt *n = - makeNode(AlterSubscriptionStmt); - n->kind = ALTER_SUBSCRIPTION_PUBLICATION_REFRESH; - n->subname = $3; - n->publication = $6; - n->options = $8; - $$ = (Node *)n; - } - | ALTER SUBSCRIPTION name SET PUBLICATION publication_name_list SKIP REFRESH + | ALTER SUBSCRIPTION name SET PUBLICATION publication_name_list opt_definition { AlterSubscriptionStmt *n = makeNode(AlterSubscriptionStmt); n->kind = ALTER_SUBSCRIPTION_PUBLICATION; n->subname = $3; n->publication = $6; - n->options = NIL; + n->options = $7; $$ = (Node *)n; } | ALTER SUBSCRIPTION name ENABLE_P diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 8720e713c4..2d2e2c0fbc 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -3382,7 +3382,6 @@ typedef enum AlterSubscriptionType ALTER_SUBSCRIPTION_OPTIONS, ALTER_SUBSCRIPTION_CONNECTION, ALTER_SUBSCRIPTION_PUBLICATION, - ALTER_SUBSCRIPTION_PUBLICATION_REFRESH, ALTER_SUBSCRIPTION_REFRESH, ALTER_SUBSCRIPTION_ENABLED } AlterSubscriptionType; diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out index 91ba8ab95a..4fcbf7efe9 100644 --- a/src/test/regress/expected/subscription.out +++ b/src/test/regress/expected/subscription.out @@ -82,7 +82,7 @@ ERROR: invalid connection string syntax: missing "=" after "foobar" in connecti testsub | regress_subscription_user | f | {testpub} | off | dbname=doesnotexist (1 row) -ALTER SUBSCRIPTION testsub SET PUBLICATION testpub2, testpub3 SKIP REFRESH; +ALTER SUBSCRIPTION testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false); ALTER SUBSCRIPTION testsub CONNECTION 'dbname=doesnotexist2'; ALTER SUBSCRIPTION testsub SET (slot_name = 'newname'); -- fail diff --git a/src/test/regress/sql/subscription.sql b/src/test/regress/sql/subscription.sql index 4b694a357e..36fa1bbac8 100644 --- a/src/test/regress/sql/subscription.sql +++ b/src/test/regress/sql/subscription.sql @@ -61,7 +61,7 @@ ALTER SUBSCRIPTION testsub CONNECTION 'foobar'; \dRs+ -ALTER SUBSCRIPTION testsub SET PUBLICATION testpub2, testpub3 SKIP REFRESH; +ALTER SUBSCRIPTION testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false); ALTER SUBSCRIPTION testsub CONNECTION 'dbname=doesnotexist2'; ALTER SUBSCRIPTION testsub SET (slot_name = 'newname'); diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index e5638d3322..f9cf5e4392 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -143,7 +143,7 @@ $oldpid = $node_publisher->safe_psql('postgres', "SELECT pid FROM pg_stat_replication WHERE application_name = '$appname';" ); $node_subscriber->safe_psql('postgres', -"ALTER SUBSCRIPTION tap_sub SET PUBLICATION tap_pub_ins_only REFRESH WITH (copy_data = false)" +"ALTER SUBSCRIPTION tap_sub SET PUBLICATION tap_pub_ins_only WITH (copy_data = false)" ); $node_publisher->poll_query_until('postgres', "SELECT pid != $oldpid FROM pg_stat_replication WHERE application_name = '$appname';" -- cgit v1.2.3 From 703f148e98ecb4b299fdad403fc5a1de51220714 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Mon, 5 Jun 2017 18:53:41 -0700 Subject: Revert "Prevent panic during shutdown checkpoint" This reverts commit 086221cf6b1727c2baed4703c582f657b7c5350e, which was made to master only. The approach implemented in the above commit has some issues. While those could easily be fixed incrementally, doing so would make backpatching considerably harder, so instead first revert this patch. Discussion: https://postgr.es/m/20170602002912.tqlwn4gymzlxpvs2@alap3.anarazel.de --- doc/src/sgml/monitoring.sgml | 5 - src/backend/access/transam/xlog.c | 6 -- src/backend/postmaster/postmaster.c | 7 +- src/backend/replication/walsender.c | 143 ++++------------------------ src/include/replication/walsender.h | 1 - src/include/replication/walsender_private.h | 3 +- 6 files changed, 24 insertions(+), 141 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 79ca45a156..5640c0d84a 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1690,11 +1690,6 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i backup: This WAL sender is sending a backup. - - - stopping: This WAL sender is stopping. - - diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 399822d3fe..35ee7d1cb6 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8324,12 +8324,6 @@ ShutdownXLOG(int code, Datum arg) ereport(IsPostmasterEnvironment ? LOG : NOTICE, (errmsg("shutting down"))); - /* - * Wait for WAL senders to be in stopping state. This prevents commands - * from writing new WAL. - */ - WalSndWaitStopping(); - if (RecoveryInProgress()) CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE); else diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c index 35b4ec88d3..5c79b1e40d 100644 --- a/src/backend/postmaster/postmaster.c +++ b/src/backend/postmaster/postmaster.c @@ -2918,7 +2918,7 @@ reaper(SIGNAL_ARGS) * Waken walsenders for the last time. No regular backends * should be around anymore. */ - SignalChildren(SIGINT); + SignalChildren(SIGUSR2); pmState = PM_SHUTDOWN_2; @@ -3656,9 +3656,7 @@ PostmasterStateMachine(void) /* * If we get here, we are proceeding with normal shutdown. All * the regular children are gone, and it's time to tell the - * checkpointer to do a shutdown checkpoint. All WAL senders - * are told to switch to a stopping state so that the shutdown - * checkpoint can go ahead. + * checkpointer to do a shutdown checkpoint. */ Assert(Shutdown > NoShutdown); /* Start the checkpointer if not running */ @@ -3667,7 +3665,6 @@ PostmasterStateMachine(void) /* And tell it to shut down */ if (CheckpointerPID != 0) { - SignalSomeChildren(SIGUSR2, BACKEND_TYPE_WALSND); signal_child(CheckpointerPID, SIGUSR2); pmState = PM_SHUTDOWN; } diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 49cce38880..aa705e5b35 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -24,14 +24,11 @@ * are treated as not a crash but approximately normal termination; * the walsender will exit quickly without sending any more XLOG records. * - * If the server is shut down, postmaster sends us SIGUSR2 after all regular - * backends have exited. This causes the walsender to switch to the "stopping" - * state. In this state, the walsender will reject any replication command - * that may generate WAL activity. The checkpointer begins the shutdown - * checkpoint once all walsenders are confirmed as stopping. When the shutdown - * checkpoint finishes, the postmaster sends us SIGINT. This instructs - * walsender to send any outstanding WAL, including the shutdown checkpoint - * record, wait for it to be replicated to the standby, and then exit. + * If the server is shut down, postmaster sends us SIGUSR2 after all + * regular backends have exited and the shutdown checkpoint has been written. + * This instructs walsender to send any outstanding WAL, including the + * shutdown checkpoint record, wait for it to be replicated to the standby, + * and then exit. * * * Portions Copyright (c) 2010-2017, PostgreSQL Global Development Group @@ -180,14 +177,13 @@ static bool WalSndCaughtUp = false; /* Flags set by signal handlers for later service in main loop */ static volatile sig_atomic_t got_SIGHUP = false; -static volatile sig_atomic_t got_SIGINT = false; -static volatile sig_atomic_t got_SIGUSR2 = false; +static volatile sig_atomic_t walsender_ready_to_stop = false; /* - * This is set while we are streaming. When not set, SIGINT signal will be + * This is set while we are streaming. When not set, SIGUSR2 signal will be * handled like SIGTERM. When set, the main loop is responsible for checking - * got_SIGINT and terminating when it's set (after streaming any remaining - * WAL). + * walsender_ready_to_stop and terminating when it's set (after streaming any + * remaining WAL). */ static volatile sig_atomic_t replication_active = false; @@ -217,7 +213,6 @@ static struct /* Signal handlers */ static void WalSndSigHupHandler(SIGNAL_ARGS); static void WalSndXLogSendHandler(SIGNAL_ARGS); -static void WalSndSwitchStopping(SIGNAL_ARGS); static void WalSndLastCycleHandler(SIGNAL_ARGS); /* Prototypes for private functions */ @@ -306,14 +301,11 @@ WalSndErrorCleanup(void) ReplicationSlotCleanup(); replication_active = false; - if (got_SIGINT) + if (walsender_ready_to_stop) proc_exit(0); /* Revert back to startup state */ WalSndSetState(WALSNDSTATE_STARTUP); - - if (got_SIGUSR2) - WalSndSetState(WALSNDSTATE_STOPPING); } /* @@ -686,7 +678,7 @@ StartReplication(StartReplicationCmd *cmd) WalSndLoop(XLogSendPhysical); replication_active = false; - if (got_SIGINT) + if (walsender_ready_to_stop) proc_exit(0); WalSndSetState(WALSNDSTATE_STARTUP); @@ -1064,7 +1056,7 @@ StartLogicalReplication(StartReplicationCmd *cmd) { ereport(LOG, (errmsg("terminating walsender process after promotion"))); - got_SIGINT = true; + walsender_ready_to_stop = true; } WalSndSetState(WALSNDSTATE_CATCHUP); @@ -1115,7 +1107,7 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotRelease(); replication_active = false; - if (got_SIGINT) + if (walsender_ready_to_stop) proc_exit(0); WalSndSetState(WALSNDSTATE_STARTUP); @@ -1326,14 +1318,6 @@ WalSndWaitForWal(XLogRecPtr loc) else RecentFlushPtr = GetXLogReplayRecPtr(NULL); - /* - * If postmaster asked us to switch to the stopping state, do so. - * Shutdown is in progress and this will allow the checkpointer to - * move on with the shutdown checkpoint. - */ - if (got_SIGUSR2) - WalSndSetState(WALSNDSTATE_STOPPING); - /* * If postmaster asked us to stop, don't wait here anymore. This will * cause the xlogreader to return without reading a full record, which @@ -1343,7 +1327,7 @@ WalSndWaitForWal(XLogRecPtr loc) * RecentFlushPtr, so we can send all remaining data before shutting * down. */ - if (got_SIGINT) + if (walsender_ready_to_stop) break; /* @@ -1417,22 +1401,6 @@ exec_replication_command(const char *cmd_string) MemoryContext cmd_context; MemoryContext old_context; - /* - * If WAL sender has been told that shutdown is getting close, switch its - * status accordingly to handle the next replication commands correctly. - */ - if (got_SIGUSR2) - WalSndSetState(WALSNDSTATE_STOPPING); - - /* - * Throw error if in stopping mode. We need prevent commands that could - * generate WAL while the shutdown checkpoint is being written. To be - * safe, we just prohibit all new commands. - */ - if (MyWalSnd->state == WALSNDSTATE_STOPPING) - ereport(ERROR, - (errmsg("cannot execute new commands while WAL sender is in stopping mode"))); - /* * CREATE_REPLICATION_SLOT ... LOGICAL exports a snapshot until the next * command arrives. Clean up the old stuff if there's anything. @@ -2155,20 +2123,13 @@ WalSndLoop(WalSndSendDataCallback send_data) } /* - * At the reception of SIGUSR2, switch the WAL sender to the - * stopping state. - */ - if (got_SIGUSR2) - WalSndSetState(WALSNDSTATE_STOPPING); - - /* - * When SIGINT arrives, we send any outstanding logs up to the + * When SIGUSR2 arrives, we send any outstanding logs up to the * shutdown checkpoint record (i.e., the latest record), wait for * them to be replicated to the standby, and exit. This may be a * normal termination at shutdown, or a promotion, the walsender * is not sure which. */ - if (got_SIGINT) + if (walsender_ready_to_stop) WalSndDone(send_data); } @@ -2907,23 +2868,7 @@ WalSndXLogSendHandler(SIGNAL_ARGS) errno = save_errno; } -/* SIGUSR2: set flag to switch to stopping state */ -static void -WalSndSwitchStopping(SIGNAL_ARGS) -{ - int save_errno = errno; - - got_SIGUSR2 = true; - SetLatch(MyLatch); - - errno = save_errno; -} - -/* - * SIGINT: set flag to do a last cycle and shut down afterwards. The WAL - * sender should already have been switched to WALSNDSTATE_STOPPING at - * this point. - */ +/* SIGUSR2: set flag to do a last cycle and shut down afterwards */ static void WalSndLastCycleHandler(SIGNAL_ARGS) { @@ -2938,7 +2883,7 @@ WalSndLastCycleHandler(SIGNAL_ARGS) if (!replication_active) kill(MyProcPid, SIGTERM); - got_SIGINT = true; + walsender_ready_to_stop = true; SetLatch(MyLatch); errno = save_errno; @@ -2951,14 +2896,14 @@ WalSndSignals(void) /* Set up signal handlers */ pqsignal(SIGHUP, WalSndSigHupHandler); /* set flag to read config * file */ - pqsignal(SIGINT, WalSndLastCycleHandler); /* request a last cycle and - * shutdown */ + pqsignal(SIGINT, SIG_IGN); /* not used */ pqsignal(SIGTERM, die); /* request shutdown */ pqsignal(SIGQUIT, quickdie); /* hard crash time */ InitializeTimeouts(); /* establishes SIGALRM handler */ pqsignal(SIGPIPE, SIG_IGN); pqsignal(SIGUSR1, WalSndXLogSendHandler); /* request WAL sending */ - pqsignal(SIGUSR2, WalSndSwitchStopping); /* switch to stopping state */ + pqsignal(SIGUSR2, WalSndLastCycleHandler); /* request a last cycle and + * shutdown */ /* Reset some signals that are accepted by postmaster but not here */ pqsignal(SIGCHLD, SIG_DFL); @@ -3036,50 +2981,6 @@ WalSndWakeup(void) } } -/* - * Wait that all the WAL senders have reached the stopping state. This is - * used by the checkpointer to control when shutdown checkpoints can - * safely begin. - */ -void -WalSndWaitStopping(void) -{ - for (;;) - { - int i; - bool all_stopped = true; - - for (i = 0; i < max_wal_senders; i++) - { - WalSndState state; - WalSnd *walsnd = &WalSndCtl->walsnds[i]; - - SpinLockAcquire(&walsnd->mutex); - - if (walsnd->pid == 0) - { - SpinLockRelease(&walsnd->mutex); - continue; - } - - state = walsnd->state; - SpinLockRelease(&walsnd->mutex); - - if (state != WALSNDSTATE_STOPPING) - { - all_stopped = false; - break; - } - } - - /* safe to leave if confirmation is done for all WAL senders */ - if (all_stopped) - return; - - pg_usleep(10000L); /* wait for 10 msec */ - } -} - /* Set state for current walsender (only called in walsender) */ void WalSndSetState(WalSndState state) @@ -3113,8 +3014,6 @@ WalSndGetStateString(WalSndState state) return "catchup"; case WALSNDSTATE_STREAMING: return "streaming"; - case WALSNDSTATE_STOPPING: - return "stopping"; } return "UNKNOWN"; } diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 99f12377e0..2ca903872e 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -44,7 +44,6 @@ extern void WalSndSignals(void); extern Size WalSndShmemSize(void); extern void WalSndShmemInit(void); extern void WalSndWakeup(void); -extern void WalSndWaitStopping(void); extern void WalSndRqstFileReload(void); /* diff --git a/src/include/replication/walsender_private.h b/src/include/replication/walsender_private.h index 36311e124c..2c59056cef 100644 --- a/src/include/replication/walsender_private.h +++ b/src/include/replication/walsender_private.h @@ -24,8 +24,7 @@ typedef enum WalSndState WALSNDSTATE_STARTUP = 0, WALSNDSTATE_BACKUP, WALSNDSTATE_CATCHUP, - WALSNDSTATE_STREAMING, - WALSNDSTATE_STOPPING + WALSNDSTATE_STREAMING } WalSndState; /* -- cgit v1.2.3 From c6c333436491a292d56044ed6e167e2bdee015a2 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Mon, 5 Jun 2017 18:53:41 -0700 Subject: Prevent possibility of panics during shutdown checkpoint. When the checkpointer writes the shutdown checkpoint, it checks afterwards whether any WAL has been written since it started and throws a PANIC if so. At that point, only walsenders are still active, so one might think this could not happen, but walsenders can also generate WAL, for instance in BASE_BACKUP and logical decoding related commands (e.g. via hint bits). So they can trigger this panic if such a command is run while the shutdown checkpoint is being written. To fix this, divide the walsender shutdown into two phases. First, checkpointer, itself triggered by postmaster, sends a PROCSIG_WALSND_INIT_STOPPING signal to all walsenders. If the backend is idle or runs an SQL query this causes the backend to shutdown, if logical replication is in progress all existing WAL records are processed followed by a shutdown. Otherwise this causes the walsender to switch to the "stopping" state. In this state, the walsender will reject any further replication commands. The checkpointer begins the shutdown checkpoint once all walsenders are confirmed as stopping. When the shutdown checkpoint finishes, the postmaster sends us SIGUSR2. This instructs walsender to send any outstanding WAL, including the shutdown checkpoint record, wait for it to be replicated to the standby, and then exit. Author: Andres Freund, based on an earlier patch by Michael Paquier Reported-By: Fujii Masao, Andres Freund Reviewed-By: Michael Paquier Discussion: https://postgr.es/m/20170602002912.tqlwn4gymzlxpvs2@alap3.anarazel.de Backpatch: 9.4, where logical decoding was introduced --- doc/src/sgml/monitoring.sgml | 5 + src/backend/access/transam/xlog.c | 11 ++ src/backend/replication/walsender.c | 187 ++++++++++++++++++++++++---- src/backend/storage/ipc/procsignal.c | 4 + src/include/replication/walsender.h | 3 + src/include/replication/walsender_private.h | 3 +- src/include/storage/procsignal.h | 1 + 7 files changed, 186 insertions(+), 28 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 5640c0d84a..79ca45a156 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1690,6 +1690,11 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i backup: This WAL sender is sending a backup. + + + stopping: This WAL sender is stopping. + + diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 35ee7d1cb6..70d2570dc2 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8324,6 +8324,17 @@ ShutdownXLOG(int code, Datum arg) ereport(IsPostmasterEnvironment ? LOG : NOTICE, (errmsg("shutting down"))); + /* + * Signal walsenders to move to stopping state. + */ + WalSndInitStopping(); + + /* + * Wait for WAL senders to be in stopping state. This prevents commands + * from writing new WAL. + */ + WalSndWaitStopping(); + if (RecoveryInProgress()) CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE); else diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 27aa3e6bc7..c9a6f7019d 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -24,11 +24,17 @@ * are treated as not a crash but approximately normal termination; * the walsender will exit quickly without sending any more XLOG records. * - * If the server is shut down, postmaster sends us SIGUSR2 after all - * regular backends have exited and the shutdown checkpoint has been written. - * This instructs walsender to send any outstanding WAL, including the - * shutdown checkpoint record, wait for it to be replicated to the standby, - * and then exit. + * If the server is shut down, checkpointer sends us + * PROCSIG_WALSND_INIT_STOPPING after all regular backends have exited. If + * the backend is idle or runs an SQL query this causes the backend to + * shutdown, if logical replication is in progress all existing WAL records + * are processed followed by a shutdown. Otherwise this causes the walsender + * to switch to the "stopping" state. In this state, the walsender will reject + * any further replication commands. The checkpointer begins the shutdown + * checkpoint once all walsenders are confirmed as stopping. When the shutdown + * checkpoint finishes, the postmaster sends us SIGUSR2. This instructs + * walsender to send any outstanding WAL, including the shutdown checkpoint + * record, wait for it to be replicated to the standby, and then exit. * * * Portions Copyright (c) 2010-2017, PostgreSQL Global Development Group @@ -177,13 +183,14 @@ static bool WalSndCaughtUp = false; /* Flags set by signal handlers for later service in main loop */ static volatile sig_atomic_t got_SIGHUP = false; -static volatile sig_atomic_t walsender_ready_to_stop = false; +static volatile sig_atomic_t got_SIGUSR2 = false; +static volatile sig_atomic_t got_STOPPING = false; /* - * This is set while we are streaming. When not set, SIGUSR2 signal will be - * handled like SIGTERM. When set, the main loop is responsible for checking - * walsender_ready_to_stop and terminating when it's set (after streaming any - * remaining WAL). + * This is set while we are streaming. When not set + * PROCSIG_WALSND_INIT_STOPPING signal will be handled like SIGTERM. When set, + * the main loop is responsible for checking got_STOPPING and terminating when + * it's set (after streaming any remaining WAL). */ static volatile sig_atomic_t replication_active = false; @@ -300,7 +307,8 @@ WalSndErrorCleanup(void) ReplicationSlotCleanup(); replication_active = false; - if (walsender_ready_to_stop) + + if (got_STOPPING || got_SIGUSR2) proc_exit(0); /* Revert back to startup state */ @@ -677,7 +685,7 @@ StartReplication(StartReplicationCmd *cmd) WalSndLoop(XLogSendPhysical); replication_active = false; - if (walsender_ready_to_stop) + if (got_STOPPING) proc_exit(0); WalSndSetState(WALSNDSTATE_STARTUP); @@ -1055,7 +1063,7 @@ StartLogicalReplication(StartReplicationCmd *cmd) { ereport(LOG, (errmsg("terminating walsender process after promotion"))); - walsender_ready_to_stop = true; + got_STOPPING = true; } WalSndSetState(WALSNDSTATE_CATCHUP); @@ -1106,7 +1114,7 @@ StartLogicalReplication(StartReplicationCmd *cmd) ReplicationSlotRelease(); replication_active = false; - if (walsender_ready_to_stop) + if (got_STOPPING) proc_exit(0); WalSndSetState(WALSNDSTATE_STARTUP); @@ -1311,6 +1319,14 @@ WalSndWaitForWal(XLogRecPtr loc) /* Check for input from the client */ ProcessRepliesIfAny(); + /* + * If we're shutting down, trigger pending WAL to be written out, + * otherwise we'd possibly end up waiting for WAL that never gets + * written, because walwriter has shut down already. + */ + if (got_STOPPING) + XLogBackgroundFlush(); + /* Update our idea of the currently flushed position. */ if (!RecoveryInProgress()) RecentFlushPtr = GetFlushRecPtr(); @@ -1326,7 +1342,7 @@ WalSndWaitForWal(XLogRecPtr loc) * RecentFlushPtr, so we can send all remaining data before shutting * down. */ - if (walsender_ready_to_stop) + if (got_STOPPING) break; /* @@ -1400,6 +1416,22 @@ exec_replication_command(const char *cmd_string) MemoryContext cmd_context; MemoryContext old_context; + /* + * If WAL sender has been told that shutdown is getting close, switch its + * status accordingly to handle the next replication commands correctly. + */ + if (got_STOPPING) + WalSndSetState(WALSNDSTATE_STOPPING); + + /* + * Throw error if in stopping mode. We need prevent commands that could + * generate WAL while the shutdown checkpoint is being written. To be + * safe, we just prohibit all new commands. + */ + if (MyWalSnd->state == WALSNDSTATE_STOPPING) + ereport(ERROR, + (errmsg("cannot execute new commands while WAL sender is in stopping mode"))); + /* * CREATE_REPLICATION_SLOT ... LOGICAL exports a snapshot until the next * command arrives. Clean up the old stuff if there's anything. @@ -2128,7 +2160,7 @@ WalSndLoop(WalSndSendDataCallback send_data) * normal termination at shutdown, or a promotion, the walsender * is not sure which. */ - if (walsender_ready_to_stop) + if (got_SIGUSR2) WalSndDone(send_data); } @@ -2443,6 +2475,10 @@ XLogSendPhysical(void) XLogRecPtr endptr; Size nbytes; + /* If requested switch the WAL sender to the stopping state. */ + if (got_STOPPING) + WalSndSetState(WALSNDSTATE_STOPPING); + if (streamingDoneSending) { WalSndCaughtUp = true; @@ -2733,7 +2769,16 @@ XLogSendLogical(void) * point, then we're caught up. */ if (logical_decoding_ctx->reader->EndRecPtr >= GetFlushRecPtr()) + { WalSndCaughtUp = true; + + /* + * Have WalSndLoop() terminate the connection in an orderly + * manner, after writing out all the pending data. + */ + if (got_STOPPING) + got_SIGUSR2 = true; + } } /* Update shared memory status */ @@ -2843,6 +2888,26 @@ WalSndRqstFileReload(void) } } +/* + * Handle PROCSIG_WALSND_INIT_STOPPING signal. + */ +void +HandleWalSndInitStopping(void) +{ + Assert(am_walsender); + + /* + * If replication has not yet started, die like with SIGTERM. If + * replication is active, only set a flag and wake up the main loop. It + * will send any outstanding WAL, wait for it to be replicated to the + * standby, and then exit gracefully. + */ + if (!replication_active) + kill(MyProcPid, SIGTERM); + else + got_STOPPING = true; +} + /* SIGHUP: set flag to re-read config file at next convenient time */ static void WalSndSigHupHandler(SIGNAL_ARGS) @@ -2856,22 +2921,17 @@ WalSndSigHupHandler(SIGNAL_ARGS) errno = save_errno; } -/* SIGUSR2: set flag to do a last cycle and shut down afterwards */ +/* + * SIGUSR2: set flag to do a last cycle and shut down afterwards. The WAL + * sender should already have been switched to WALSNDSTATE_STOPPING at + * this point. + */ static void WalSndLastCycleHandler(SIGNAL_ARGS) { int save_errno = errno; - /* - * If replication has not yet started, die like with SIGTERM. If - * replication is active, only set a flag and wake up the main loop. It - * will send any outstanding WAL, wait for it to be replicated to the - * standby, and then exit gracefully. - */ - if (!replication_active) - kill(MyProcPid, SIGTERM); - - walsender_ready_to_stop = true; + got_SIGUSR2 = true; SetLatch(MyLatch); errno = save_errno; @@ -2969,6 +3029,77 @@ WalSndWakeup(void) } } +/* + * Signal all walsenders to move to stopping state. + * + * This will trigger walsenders to move to a state where no further WAL can be + * generated. See this file's header for details. + */ +void +WalSndInitStopping(void) +{ + int i; + + for (i = 0; i < max_wal_senders; i++) + { + WalSnd *walsnd = &WalSndCtl->walsnds[i]; + pid_t pid; + + SpinLockAcquire(&walsnd->mutex); + pid = walsnd->pid; + SpinLockRelease(&walsnd->mutex); + + if (pid == 0) + continue; + + SendProcSignal(pid, PROCSIG_WALSND_INIT_STOPPING, InvalidBackendId); + } +} + +/* + * Wait that all the WAL senders have quit or reached the stopping state. This + * is used by the checkpointer to control when the shutdown checkpoint can + * safely be performed. + */ +void +WalSndWaitStopping(void) +{ + for (;;) + { + int i; + bool all_stopped = true; + + for (i = 0; i < max_wal_senders; i++) + { + WalSndState state; + WalSnd *walsnd = &WalSndCtl->walsnds[i]; + + SpinLockAcquire(&walsnd->mutex); + + if (walsnd->pid == 0) + { + SpinLockRelease(&walsnd->mutex); + continue; + } + + state = walsnd->state; + SpinLockRelease(&walsnd->mutex); + + if (state != WALSNDSTATE_STOPPING) + { + all_stopped = false; + break; + } + } + + /* safe to leave if confirmation is done for all WAL senders */ + if (all_stopped) + return; + + pg_usleep(10000L); /* wait for 10 msec */ + } +} + /* Set state for current walsender (only called in walsender) */ void WalSndSetState(WalSndState state) @@ -3002,6 +3133,8 @@ WalSndGetStateString(WalSndState state) return "catchup"; case WALSNDSTATE_STREAMING: return "streaming"; + case WALSNDSTATE_STOPPING: + return "stopping"; } return "UNKNOWN"; } diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 4a21d5512d..b9302ac630 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -20,6 +20,7 @@ #include "access/parallel.h" #include "commands/async.h" #include "miscadmin.h" +#include "replication/walsender.h" #include "storage/latch.h" #include "storage/ipc.h" #include "storage/proc.h" @@ -270,6 +271,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) if (CheckProcSignal(PROCSIG_PARALLEL_MESSAGE)) HandleParallelMessageInterrupt(); + if (CheckProcSignal(PROCSIG_WALSND_INIT_STOPPING)) + HandleWalSndInitStopping(); + if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE)) RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE); diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 2ca903872e..c50e450ec2 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -44,6 +44,9 @@ extern void WalSndSignals(void); extern Size WalSndShmemSize(void); extern void WalSndShmemInit(void); extern void WalSndWakeup(void); +extern void WalSndInitStopping(void); +extern void WalSndWaitStopping(void); +extern void HandleWalSndInitStopping(void); extern void WalSndRqstFileReload(void); /* diff --git a/src/include/replication/walsender_private.h b/src/include/replication/walsender_private.h index 2c59056cef..36311e124c 100644 --- a/src/include/replication/walsender_private.h +++ b/src/include/replication/walsender_private.h @@ -24,7 +24,8 @@ typedef enum WalSndState WALSNDSTATE_STARTUP = 0, WALSNDSTATE_BACKUP, WALSNDSTATE_CATCHUP, - WALSNDSTATE_STREAMING + WALSNDSTATE_STREAMING, + WALSNDSTATE_STOPPING } WalSndState; /* diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index d068dde5d7..553f0f43f7 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -32,6 +32,7 @@ typedef enum PROCSIG_CATCHUP_INTERRUPT, /* sinval catchup interrupt */ PROCSIG_NOTIFY_INTERRUPT, /* listen/notify interrupt */ PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */ + PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */ /* Recovery conflict reasons */ PROCSIG_RECOVERY_CONFLICT_DATABASE, -- cgit v1.2.3 From 0f33ee0e3b7527fb0c88abf0ae8a49a9c38d9c0e Mon Sep 17 00:00:00 2001 From: Michael Meskes Date: Tue, 6 Jun 2017 12:19:28 +0200 Subject: Fix docs to not claim ECPG's SET CONNECTION is not thread-aware. Changed by: Tsunakawa, Takayuki --- doc/src/sgml/ecpg.sgml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ecpg.sgml b/doc/src/sgml/ecpg.sgml index dead4c3f86..f13a0e999f 100644 --- a/doc/src/sgml/ecpg.sgml +++ b/doc/src/sgml/ecpg.sgml @@ -240,8 +240,7 @@ EXEC SQL AT connection-name SELECT ...; If your application uses multiple threads of execution, they cannot share a connection concurrently. You must either explicitly control access to the connection - (using mutexes) or use a connection for each thread. If each thread uses its own connection, - you will need to use the AT clause to specify which connection the thread will use. + (using mutexes) or use a connection for each thread. @@ -251,7 +250,7 @@ EXEC SQL AT connection-name SELECT ...; EXEC SQL SET CONNECTION connection-name; This option is particularly convenient if many statements are to be - executed on the same connection. It is not thread-aware. + executed on the same connection. -- cgit v1.2.3 From 2186b608b3cb859fe0ec04015a5c4e4cbf69caed Mon Sep 17 00:00:00 2001 From: Robert Haas Date: Tue, 6 Jun 2017 11:07:20 -0400 Subject: Clean up partcollation handling for OID 0. Consistent with what we do for indexes, we shouldn't try to record dependencies on collation OID 0 or the default collation OID (which is pinned). Also, the fact that indcollation and partcollation can contain zero OIDs when the data type is not collatable should be documented. Amit Langote, per a complaint from me. Discussion: http://postgr.es/m/CA+Tgmoba5mtPgM3NKfG06vv8na5gGbVOj0h4zvivXQwLw8wXXQ@mail.gmail.com --- doc/src/sgml/catalogs.sgml | 6 ++++-- src/backend/catalog/heap.c | 11 ++++++++--- 2 files changed, 12 insertions(+), 5 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 5723be744d..61ce12c560 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -3783,7 +3783,8 @@ SCRAM-SHA-256$<iteration count>:<salt>< pg_collation.oid For each column in the index key, this contains the OID of the - collation to use for the index. + the collation to use for the index, or zero if the column is not + of a collatable data type. @@ -4770,7 +4771,8 @@ SCRAM-SHA-256$<iteration count>:<salt>< pg_opclass.oid For each column in the partition key, this contains the OID of the - the collation to use for partitioning. + the collation to use for partitioning, or zero if the column is not + of a collatable data type. diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index 0ce94f346f..4e5b79ef94 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -3160,9 +3160,14 @@ StorePartitionKey(Relation rel, recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); - referenced.classId = CollationRelationId; - referenced.objectId = partcollation[i]; - referenced.objectSubId = 0; + /* The default collation is pinned, so don't bother recording it */ + if (OidIsValid(partcollation[i]) && + partcollation[i] != DEFAULT_COLLATION_OID) + { + referenced.classId = CollationRelationId; + referenced.objectId = partcollation[i]; + referenced.objectSubId = 0; + } recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); } -- cgit v1.2.3 From 79c4fa0f62291fd30d126a12f35b9ce71f06b3c0 Mon Sep 17 00:00:00 2001 From: Robert Haas Date: Tue, 6 Jun 2017 12:24:44 -0400 Subject: Fix some cases of "the the" split across two lines. Kevin Grittner observed that 2186b608b3cb859fe0ec04015a5c4e4cbf69caed introduced a new occurence of this by copying existing text, and I found a few more cases using grep. Discussion: http://postgr.es/m/CADAecHWfG-K+YvocHCkrXV-ycm+eUOaaUVfYZNOnwf0pSmuQCw@mail.gmail.com --- doc/src/sgml/catalogs.sgml | 4 ++-- doc/src/sgml/ddl.sgml | 2 +- src/bin/pg_basebackup/receivelog.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 61ce12c560..ea655a10a8 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -3783,7 +3783,7 @@ SCRAM-SHA-256$<iteration count>:<salt>< pg_collation.oid For each column in the index key, this contains the OID of the - the collation to use for the index, or zero if the column is not + collation to use for the index, or zero if the column is not of a collatable data type. @@ -4771,7 +4771,7 @@ SCRAM-SHA-256$<iteration count>:<salt>< pg_opclass.oid For each column in the partition key, this contains the OID of the - the collation to use for partitioning, or zero if the column is not + collation to use for partitioning, or zero if the column is not of a collatable data type. diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml index 84c4f20990..ec015e93d0 100644 --- a/doc/src/sgml/ddl.sgml +++ b/doc/src/sgml/ddl.sgml @@ -3165,7 +3165,7 @@ CREATE INDEX ON measurement_y2008m01 (logdate); Normally the set of partitions established when initially defining the - the table are not intended to remain static. It is common to want to + table are not intended to remain static. It is common to want to remove old partitions of data and periodically add new partitions for new data. One of the most important advantages of partitioning is precisely that it allows this otherwise painful task to be executed diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c index cf730da283..52aa274bb8 100644 --- a/src/bin/pg_basebackup/receivelog.c +++ b/src/bin/pg_basebackup/receivelog.c @@ -629,7 +629,7 @@ ReceiveXlogStream(PGconn *conn, StreamCtl *stream) * server had sent us half of a WAL record, when it was promoted. * The new timeline will begin at the end of the last complete * record in that case, overlapping the partial WAL record on the - * the old timeline. + * old timeline. */ uint32 newtimeline; bool parsed; -- cgit v1.2.3 From 0198c277a29a035aa8a4e6767967201135f6caa9 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Wed, 7 Jun 2017 17:23:38 -0400 Subject: Docs: improve CREATE TABLE ref page's discussion of partition bounds. Clarify in the syntax synopsis that partition bound values must be exactly numeric literals or string literals; previously it said "bound_literal" which was defined nowhere. Replace confusing --- and, I think, incorrect in detail --- definition of how range bounds work with a reference to row-wise comparison plus a concrete example (which I stole from Robert Haas). Minor copy-editing in the same area. Discussion: https://postgr.es/m/30475.1496005465@sss.pgh.pa.us Discussion: https://postgr.es/m/28106.1496041449@sss.pgh.pa.us --- doc/src/sgml/ref/create_table.sgml | 72 ++++++++++++++++++++++---------------- 1 file changed, 41 insertions(+), 31 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 0478e40447..bc014d0879 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -86,8 +86,9 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI and partition_bound_spec is: -{ IN ( { bound_literal | NULL } [, ...] ) | - FROM ( { bound_literal | UNBOUNDED } [, ...] ) TO ( { bound_literal | UNBOUNDED } [, ...] ) } +IN ( { numeric_literal | string_literal | NULL } [, ...] ) | +FROM ( { numeric_literal | string_literal | UNBOUNDED } [, ...] ) + TO ( { numeric_literal | string_literal | UNBOUNDED } [, ...] ) index_parameters in UNIQUE, PRIMARY KEY, and EXCLUDE constraints are: @@ -252,21 +253,34 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI PARTITION OF parent_table FOR VALUES partition_bound_spec - Creates the table as partition of the specified + Creates the table as a partition of the specified parent table. - The partition bound specification must correspond to the partitioning - method and partition key of the parent table, and must not overlap with - any existing partition of that parent. + The partition_bound_spec + must correspond to the partitioning method and partition key of the + parent table, and must not overlap with any existing partition of that + parent. The form with IN is used for list partitioning, + while the form with FROM and TO is used for + range partitioning. - Each of the values specified in the partition bound specification is + Each of the values specified in + the partition_bound_spec is a literal, NULL, or UNBOUNDED. - A literal is either a numeric constant or a string constant that is - coercible to the corresponding partition key column's type. + Each literal value must be either a numeric constant that is coercible + to the corresponding partition key column's type, or a string literal + that is valid input for that type. + + + + When creating a list partition, NULL can be + specified to signify that the partition allows the partition key + column to be null. However, there cannot be more than one such + list partition for a given parent table. NULL + cannot be specified for range partitions. @@ -274,30 +288,25 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI FROM is an inclusive bound, whereas the upper bound specified with TO is an exclusive bound. That is, the values specified in the FROM list - are accepted values of the corresponding partition key columns in a - given partition, whereas those in the TO list are - not. To be precise, this applies only to the first of the partition - key columns for which the corresponding values in the FROM - and TO lists are not equal. All rows in a given - partition contain the same values for all preceding columns, equal to - those specified in FROM and TO - lists. On the other hand, any subsequent columns are insignificant - as far as implicit partition constraint is concerned. + are valid values of the corresponding partition key columns for this + partition, whereas those in the TO list are + not. Note that this statement must be understood according to the + rules of row-wise comparison (). + For example, given PARTITION BY RANGE (x,y), a partition + bound FROM (1, 2) TO (3, 4) + allows x=1 with any y>=2, + x=2 with any non-null y, + and x=3 with any y<4. - Specifying UNBOUNDED in FROM + Writing UNBOUNDED in FROM signifies -infinity as the lower bound of the - corresponding column, whereas it signifies +infinity - as the upper bound when specified in TO. - - - - When creating a list partition, NULL can be - specified to signify that the partition allows the partition key - column to be null. However, there cannot be more than one such - list partition for a given parent table. NULL - cannot be specified for range partitions. + corresponding column, whereas when written in TO, + it signifies +infinity as the upper bound. + All items following an UNBOUNDED item within + a FROM or TO list must also + be UNBOUNDED. @@ -318,8 +327,9 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI Rows inserted into a partitioned table will be automatically routed to the correct partition. If no suitable partition exists, an error will - occur. Also, if updating a row in a given partition causes it to move - to another partition due to the new partition key, an error will occur. + occur. Also, if updating a row in a given partition would require it + to move to another partition due to new partition key values, an error + will occur. -- cgit v1.2.3 From 7ff9812f9aef584b6ee076378d77a09a68f12d97 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 8 Jun 2017 12:12:31 -0400 Subject: Put new command-line options in alphabetical order --- doc/src/sgml/ref/pg_dumpall.sgml | 16 +++++------ doc/src/sgml/ref/pg_recvlogical.sgml | 52 +++++++++++++++++----------------- doc/src/sgml/ref/pgbench.sgml | 20 ++++++------- src/bin/pg_basebackup/pg_recvlogical.c | 4 +-- src/bin/pg_dump/pg_dumpall.c | 4 +-- src/bin/pgbench/pgbench.c | 4 +-- 6 files changed, 50 insertions(+), 50 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml index b45e813486..2ab570ad4c 100644 --- a/doc/src/sgml/ref/pg_dumpall.sgml +++ b/doc/src/sgml/ref/pg_dumpall.sgml @@ -333,23 +333,23 @@ PostgreSQL documentation - + - Do not dump passwords for roles. When restored, roles will have a NULL - password and authentication will always fail until the password is reset. - Since password values aren't needed when this option is specified we - use the catalog view pg_roles in preference to pg_authid, since access - to pg_authid may be restricted by security policy. + Do not dump publications. - + - Do not dump publications. + Do not dump passwords for roles. When restored, roles will have a NULL + password and authentication will always fail until the password is reset. + Since password values aren't needed when this option is specified we + use the catalog view pg_roles in preference to pg_authid, since access + to pg_authid may be restricted by security policy. diff --git a/doc/src/sgml/ref/pg_recvlogical.sgml b/doc/src/sgml/ref/pg_recvlogical.sgml index eaea94df8b..9c7bb1907b 100644 --- a/doc/src/sgml/ref/pg_recvlogical.sgml +++ b/doc/src/sgml/ref/pg_recvlogical.sgml @@ -113,6 +113,32 @@ PostgreSQL documentation output and other replication behavior: + + + + + + In mode, automatically stop replication + and exit with normal exit status 0 when receiving reaches the + specified LSN. If specified when not in + mode, an error is raised. + + + + If there's a record with LSN exactly equal to lsn, + the record will be output. + + + + The option is not aware of transaction + boundaries and may truncate output partway through a transaction. + Any partially output transaction will not be consumed and will be + replayed again when the slot is next read from. Individual messages + are never truncated. + + + + @@ -162,32 +188,6 @@ PostgreSQL documentation - - - - - - In mode, automatically stop replication - and exit with normal exit status 0 when receiving reaches the - specified LSN. If specified when not in - mode, an error is raised. - - - - If there's a record with LSN exactly equal to lsn, - the record will be output. - - - - The option is not aware of transaction - boundaries and may truncate output partway through a transaction. - Any partially output transaction will not be consumed and will be - replayed again when the slot is next read from. Individual messages - are never truncated. - - - - diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 5735c48717..64b043b48a 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -579,6 +579,16 @@ pgbench options dbname + + + + + Set the filename prefix for the log files created by + + + + @@ -610,16 +620,6 @@ pgbench options dbname - - - - - Set the filename prefix for the log files created by - - - - diff --git a/src/bin/pg_basebackup/pg_recvlogical.c b/src/bin/pg_basebackup/pg_recvlogical.c index 6b081bd737..5f7412e9d5 100644 --- a/src/bin/pg_basebackup/pg_recvlogical.c +++ b/src/bin/pg_basebackup/pg_recvlogical.c @@ -81,12 +81,12 @@ usage(void) printf(_(" --drop-slot drop the replication slot (for the slot's name see --slot)\n")); printf(_(" --start start streaming in a replication slot (for the slot's name see --slot)\n")); printf(_("\nOptions:\n")); + printf(_(" -E, --endpos=LSN exit after receiving the specified LSN\n")); printf(_(" -f, --file=FILE receive log into this file, - for stdout\n")); printf(_(" -F --fsync-interval=SECS\n" " time between fsyncs to the output file (default: %d)\n"), (fsync_interval / 1000)); printf(_(" --if-not-exists do not error if slot already exists when creating a slot\n")); printf(_(" -I, --startpos=LSN where in an existing slot should the streaming start\n")); - printf(_(" -E, --endpos=LSN exit after receiving the specified LSN\n")); printf(_(" -n, --no-loop do not loop on connection lost\n")); printf(_(" -o, --option=NAME[=VALUE]\n" " pass option NAME with optional value VALUE to the\n" @@ -725,7 +725,7 @@ main(int argc, char **argv) } } - while ((c = getopt_long(argc, argv, "f:F:nvd:h:p:U:wWI:E:o:P:s:S:", + while ((c = getopt_long(argc, argv, "E:f:F:nvd:h:p:U:wWI:o:P:s:S:", long_options, &option_index)) != -1) { switch (c) diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c index 68003c3533..2a08abfba5 100644 --- a/src/bin/pg_dump/pg_dumpall.c +++ b/src/bin/pg_dump/pg_dumpall.c @@ -131,11 +131,11 @@ main(int argc, char *argv[]) {"role", required_argument, NULL, 3}, {"use-set-session-authorization", no_argument, &use_setsessauth, 1}, {"no-publications", no_argument, &no_publications, 1}, + {"no-role-passwords", no_argument, &no_role_passwords, 1}, {"no-security-labels", no_argument, &no_security_labels, 1}, {"no-subscriptions", no_argument, &no_subscriptions, 1}, {"no-sync", no_argument, NULL, 4}, {"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1}, - {"no-role-passwords", no_argument, &no_role_passwords, 1}, {NULL, 0, NULL, 0} }; @@ -599,12 +599,12 @@ help(void) printf(_(" --if-exists use IF EXISTS when dropping objects\n")); printf(_(" --inserts dump data as INSERT commands, rather than COPY\n")); printf(_(" --no-publications do not dump publications\n")); + printf(_(" --no-role-passwords do not dump passwords for roles\n")); printf(_(" --no-security-labels do not dump security label assignments\n")); printf(_(" --no-subscriptions do not dump subscriptions\n")); printf(_(" --no-sync do not wait for changes to be written safely to disk\n")); printf(_(" --no-tablespaces do not dump tablespace assignments\n")); printf(_(" --no-unlogged-table-data do not dump unlogged table data\n")); - printf(_(" --no-role-passwords do not dump passwords for roles\n")); printf(_(" --quote-all-identifiers quote all identifiers, even if not key words\n")); printf(_(" --use-set-session-authorization\n" " use SET SESSION AUTHORIZATION commands instead of\n" diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index ae3624721e..14aa587a27 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -511,10 +511,10 @@ usage(void) " -T, --time=NUM duration of benchmark test in seconds\n" " -v, --vacuum-all vacuum all four standard tables before tests\n" " --aggregate-interval=NUM aggregate data over NUM seconds\n" - " --progress-timestamp use Unix epoch timestamps for progress\n" - " --sampling-rate=NUM fraction of transactions to log (e.g., 0.01 for 1%%)\n" " --log-prefix=PREFIX prefix for transaction time log file\n" " (default: \"pgbench_log\")\n" + " --progress-timestamp use Unix epoch timestamps for progress\n" + " --sampling-rate=NUM fraction of transactions to log (e.g., 0.01 for 1%%)\n" "\nCommon options:\n" " -d, --debug print debugging output\n" " -h, --host=HOSTNAME database server host or socket directory\n" -- cgit v1.2.3 From 5c4109f2c8c2027114cfdc7c0617f81928a0b10e Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 8 Jun 2017 17:19:50 -0400 Subject: Update key words table for version 10 --- doc/src/sgml/keywords.sgml | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/keywords.sgml b/doc/src/sgml/keywords.sgml index 4407fb928b..01bc9b47b1 100644 --- a/doc/src/sgml/keywords.sgml +++ b/doc/src/sgml/keywords.sgml @@ -778,7 +778,7 @@ COLUMNS - + non-reserved non-reserved non-reserved @@ -1989,7 +1989,7 @@ GENERATED - + non-reserved non-reserved non-reserved @@ -2568,13 +2568,6 @@ non-reserved - - LIST - non-reserved - - - - LISTEN non-reserved @@ -2906,7 +2899,7 @@ NEW - + non-reserved reserved reserved @@ -3130,7 +3123,7 @@ OLD - + non-reserved reserved reserved @@ -3256,7 +3249,7 @@ OVERRIDING - + non-reserved non-reserved non-reserved @@ -3590,6 +3583,13 @@ non-reserved reserved + + PUBLICATION + non-reserved + + + + QUOTE non-reserved @@ -3676,7 +3676,7 @@ REFERENCING - + non-reserved reserved reserved @@ -4017,6 +4017,13 @@ non-reserved reserved + + SCHEMAS + non-reserved + + + + SCHEMA_NAME @@ -4458,6 +4465,13 @@ reserved + + SUBSCRIPTION + non-reserved + + + + SUBSTRING non-reserved (cannot be function or type) @@ -5293,7 +5307,7 @@ XMLNAMESPACES - + non-reserved (cannot be function or type) reserved reserved @@ -5342,7 +5356,7 @@ XMLTABLE - + non-reserved (cannot be function or type) reserved reserved -- cgit v1.2.3 From 0933fcee9851eb2afcd41db1ee4425153f4cbdd3 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Fri, 9 Jun 2017 17:11:46 -0400 Subject: doc: Document that subscriptions to same server might hang --- doc/src/sgml/ref/create_subscription.sgml | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml index 2c91eb6f50..dd5eda043a 100644 --- a/doc/src/sgml/ref/create_subscription.sgml +++ b/doc/src/sgml/ref/create_subscription.sgml @@ -226,6 +226,19 @@ CREATE SUBSCRIPTION subscription_name + + + Creating a subscription that connects to the same database cluster (for + example, to replicate between databases in the same cluster or to replicate + within the same database) will only succeed if the replication slot is not + created as part of the same command. Otherwise, the CREATE + SUBSCRIPTION call will hang. To make this work, create the + replication slot separately (using the + function pg_create_logical_replication_slot with the + plugin name pgoutput) and create the subscription using + the parameter create_slot = false. This is an + implementation restriction that might be lifted in a future release. + -- cgit v1.2.3 From 0332993c4e14f13b211f41535f77aadb305fd354 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Fri, 9 Jun 2017 19:49:18 -0400 Subject: doc: Improve types in example Reported-by: Nikolaus Thiel --- doc/src/sgml/xfunc.sgml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 10e3c0cef9..05f4312bf3 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -302,7 +302,7 @@ SELECT add_em(1, 2) AS answer; bank account: -CREATE FUNCTION tf1 (accountno integer, debit numeric) RETURNS integer AS $$ +CREATE FUNCTION tf1 (accountno integer, debit numeric) RETURNS numeric AS $$ UPDATE bank SET balance = balance - debit WHERE accountno = tf1.accountno; @@ -333,7 +333,7 @@ SELECT tf1(17, 100.0); is: -CREATE FUNCTION tf1 (accountno integer, debit numeric) RETURNS integer AS $$ +CREATE FUNCTION tf1 (accountno integer, debit numeric) RETURNS numeric AS $$ UPDATE bank SET balance = balance - debit WHERE accountno = tf1.accountno; @@ -345,7 +345,7 @@ $$ LANGUAGE SQL; The same thing could be done in one command using RETURNING: -CREATE FUNCTION tf1 (accountno integer, debit numeric) RETURNS integer AS $$ +CREATE FUNCTION tf1 (accountno integer, debit numeric) RETURNS numeric AS $$ UPDATE bank SET balance = balance - debit WHERE accountno = tf1.accountno -- cgit v1.2.3 From e20f679f66fb7930215a1b59f13b5b1c06bfc456 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Fri, 9 Jun 2017 20:34:27 -0400 Subject: doc: Add Node.js and Go drivers to client interfaces Also, fix client interface JDBC language name to Java. Author: Sehrope Sarkuni --- doc/src/sgml/external-projects.sgml | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) (limited to 'doc/src') diff --git a/doc/src/sgml/external-projects.sgml b/doc/src/sgml/external-projects.sgml index 8457426545..2c41512de5 100644 --- a/doc/src/sgml/external-projects.sgml +++ b/doc/src/sgml/external-projects.sgml @@ -70,7 +70,7 @@ JDBC - JDBC + Java Type 4 JDBC driver @@ -82,6 +82,13 @@ + + node-postgres + JavaScript + Node.js driver + + + Npgsql .NET @@ -96,6 +103,13 @@ + + pq + Go + Pure Go driver for Go's database/sql + + + psqlODBC ODBC -- cgit v1.2.3 From eab86897bd8cbeb21ae8959ca9a095ce7cb663df Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii Date: Mon, 12 Jun 2017 14:49:25 +0900 Subject: Fix ALTER TABLE doc examples. Patch by Yugo Nagata . Confirmed by Amit Langote, who is the original author of the document part. --- doc/src/sgml/ref/alter_table.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 56ea830d41..4c61c44265 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -1398,7 +1398,7 @@ ALTER TABLE cities Detach a partition from partitioned table: -ALTER TABLE cities +ALTER TABLE measurement DETACH PARTITION measurement_y2015m12; -- cgit v1.2.3 From bf6e4c3c82d349dc311ef795cc8eb7a9badf49eb Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 12 Jun 2017 09:51:18 -0400 Subject: Trim trailing whitespace --- doc/src/sgml/func.sgml | 2 +- doc/src/sgml/high-availability.sgml | 2 +- doc/src/sgml/libpq.sgml | 8 ++++---- doc/src/sgml/monitoring.sgml | 4 ++-- doc/src/sgml/parallel.sgml | 2 +- doc/src/sgml/pgstattuple.sgml | 14 +++++++------- doc/src/sgml/postgres-fdw.sgml | 2 +- doc/src/sgml/protocol.sgml | 4 ++-- doc/src/sgml/ref/alter_table.sgml | 4 ++-- doc/src/sgml/ref/copy.sgml | 2 +- 10 files changed, 22 insertions(+), 22 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 14aae736c3..7452d7cfa6 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -4361,7 +4361,7 @@ SELECT (regexp_match('foobarbequebaz', 'bar.*que'))[1]; Some examples: - SELECT regexp_matches('foo', 'not there'); +SELECT regexp_matches('foo', 'not there'); regexp_matches ---------------- (0 rows) diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml index 92e3b454fb..ce67d4a5e3 100644 --- a/doc/src/sgml/high-availability.sgml +++ b/doc/src/sgml/high-availability.sgml @@ -1175,7 +1175,7 @@ synchronous_standby_names = 'FIRST 2 (s1, s2, s3)' An example of synchronous_standby_names for a quorum-based multiple synchronous standbys is: - synchronous_standby_names = 'ANY 2 (s1, s2, s3)' +synchronous_standby_names = 'ANY 2 (s1, s2, s3)' In this example, if four standby servers s1, s2, s3 and s4 are running, transaction commits will diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 23e6f1c3dc..c3bd4f9b9b 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -5941,12 +5941,12 @@ char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, Prepares the md5-encrypted form of a PostgreSQL password. - + char *PQencryptPassword(const char *passwd, const char *user); - - PQencryptPassword is an older, deprecated version of + + PQencryptPassword is an older, deprecated version of PQencryptPasswodConn. The difference is that - PQencryptPassword does not + PQencryptPassword does not require a connection object, and md5 is always used as the encryption algorithm. diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 79ca45a156..9ff5eea038 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -802,7 +802,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser backend_type text - Type of current backend. Possible types are + Type of current backend. Possible types are autovacuum launcher, autovacuum worker, background worker, background writer, client backend, checkpointer, @@ -1827,7 +1827,7 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i the standby to catch up with the sending server assuming the current rate of replay. Such a system would show similar times while new WAL is being generated, but would differ when the sender becomes idle. In - particular, when the standby has caught up completely, + particular, when the standby has caught up completely, pg_stat_replication shows the time taken to write, flush and replay the most recent reported WAL location rather than zero as some users might expect. This is consistent with the goal of diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml index a65129078c..ff31e7537e 100644 --- a/doc/src/sgml/parallel.sgml +++ b/doc/src/sgml/parallel.sgml @@ -275,7 +275,7 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%'; In a parallel sequential scan, the table's blocks will be divided among the cooperating processes. Blocks are handed out one - at a time, so that access to the table remains sequential. + at a time, so that access to the table remains sequential. diff --git a/doc/src/sgml/pgstattuple.sgml b/doc/src/sgml/pgstattuple.sgml index b6a5f19e70..e98e04fa2f 100644 --- a/doc/src/sgml/pgstattuple.sgml +++ b/doc/src/sgml/pgstattuple.sgml @@ -119,13 +119,13 @@ free_percent | 1.95
- - The table_len will always be greater than the sum - of the tuple_len, dead_tuple_len - and free_space. The difference is accounted for by - fixed page overhead, the per-page table of pointers to tuples, and - padding to ensure that tuples are correctly aligned. - + + The table_len will always be greater than the sum + of the tuple_len, dead_tuple_len + and free_space. The difference is accounted for by + fixed page overhead, the per-page table of pointers to tuples, and + padding to ensure that tuples are correctly aligned. + diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml index 3dfc0f84ed..23558e7ec0 100644 --- a/doc/src/sgml/postgres-fdw.sgml +++ b/doc/src/sgml/postgres-fdw.sgml @@ -566,7 +566,7 @@ - These are less likely to be problematic than search_path, but + These are less likely to be problematic than search_path, but can be handled with function SET options if the need arises. diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 4837be5016..a7a3d3b2f9 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -1352,7 +1352,7 @@ general, while the next subsection gives more details on SCRAM-SHA-256. SASL Authentication Message Flow - + To begin a SASL authentication exchange, the server sends an AuthenticationSASL message. It includes a list of SASL authentication mechanisms that the server can accept, in the server's preferred order. @@ -1401,7 +1401,7 @@ ErrorMessage. SCRAM-SHA-256 (called just SCRAM from now on) is the only implemented SASL mechanism, at the moment. It is described in detail - in RFC 7677 and RFC 5802. + in RFC 7677 and RFC 5802. diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 4c61c44265..69600321e6 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -185,7 +185,7 @@ ALTER TABLE [ IF EXISTS ] name table. Even if there is no NOT NULL constraint on the parent, such a constraint can still be added to individual partitions, if desired; that is, the children can disallow nulls even if the parent - allows them, but not the other way around. + allows them, but not the other way around. @@ -617,7 +617,7 @@ ALTER TABLE [ IF EXISTS ] name - SHARE UPDATE EXCLUSIVE lock will be taken for + SHARE UPDATE EXCLUSIVE lock will be taken for fillfactor and autovacuum storage parameters, as well as the following planner related parameters: effective_io_concurrency, parallel_workers, seq_page_cost diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgml index 215efcd69d..8de1150dfb 100644 --- a/doc/src/sgml/ref/copy.sgml +++ b/doc/src/sgml/ref/copy.sgml @@ -425,7 +425,7 @@ COPY count - If row-level security is enabled for the table, the relevant + If row-level security is enabled for the table, the relevant SELECT policies will apply to COPY table TO statements. Currently, COPY FROM is not supported for tables -- cgit v1.2.3 From 5d8beac8bee344bdf4b05a63b93f06e05d999b39 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 12 Jun 2017 16:20:12 -0400 Subject: doc: Update information_schema documentation for identity columns This was apparently forgotten in the original patch. --- doc/src/sgml/information_schema.sgml | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/information_schema.sgml b/doc/src/sgml/information_schema.sgml index 02f7927436..b85849b258 100644 --- a/doc/src/sgml/information_schema.sgml +++ b/doc/src/sgml/information_schema.sgml @@ -1602,31 +1602,47 @@ identity_start character_data - Applies to a feature not available in PostgreSQL + + If the column is an identity column, then the start value of the + internal sequence, else null. + identity_increment character_data - Applies to a feature not available in PostgreSQL + + If the column is an identity column, then the increment of the internal + sequence, else null. + identity_maximum character_data - Applies to a feature not available in PostgreSQL + + If the column is an identity column, then the maximum value of the + internal sequence, else null. + identity_minimum character_data - Applies to a feature not available in PostgreSQL + + If the column is an identity column, then the minimum value of the + internal sequence, else null. + identity_cycle yes_or_no - Applies to a feature not available in PostgreSQL + + If the column is an identity column, then YES if the + internal sequence cycles or NO if it does not; + otherwise null. + -- cgit v1.2.3 From 2440c442d167d9d081a3e69c4fa78f3b6f8932e9 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 12 Jun 2017 22:34:04 -0400 Subject: doc: Update external PL list Add PL/Lua, PL/v8. Remove stale/unmaintained PL/PHP, PL/Py, PL/Ruby, PL/Scheme. Reported-by: Adam Sah --- doc/src/sgml/external-projects.sgml | 32 ++++++++++---------------------- 1 file changed, 10 insertions(+), 22 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/external-projects.sgml b/doc/src/sgml/external-projects.sgml index 2c41512de5..82aaad4e4b 100644 --- a/doc/src/sgml/external-projects.sgml +++ b/doc/src/sgml/external-projects.sgml @@ -186,19 +186,13 @@ PL/Java Java - + - PL/PHP - PHP - - - - - PL/Py - Python - + PL/Lua + Lua + @@ -207,23 +201,17 @@ - - PL/Ruby - Ruby - - - - - PL/Scheme - Scheme - - - PL/sh Unix shell + + + PL/v8 + JavaScript + + -- cgit v1.2.3 From 506b565831e6c88666c18a467377c2f98066ccac Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Tue, 13 Jun 2017 11:28:52 -0400 Subject: doc: Fix typo Author: Julien Rouhaud --- doc/src/sgml/brin.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/src') diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml index ad11109775..8dcc29925b 100644 --- a/doc/src/sgml/brin.sgml +++ b/doc/src/sgml/brin.sgml @@ -81,7 +81,7 @@ occur. (This last trigger is disabled by default and can be enabled with the autosummarize parameter.) Conversely, a range can be de-summarized using the - brin_desummarize_range(regclass, bigint) range, + brin_desummarize_range(regclass, bigint) function, which is useful when the index tuple is no longer a very good representation because the existing values have changed. -- cgit v1.2.3 From 39da0f709db4d9f16f46be56ae401df72aab93c0 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Tue, 13 Jun 2017 16:10:11 -0400 Subject: doc: Update example version numbers in pg_upgrade documentation The exact numbers don't matter, since they are examples, but it was looking quite dated. For the target version, we now automatically substitute the current major version. The updated example source version should be good for a couple of years. --- doc/src/sgml/ref/pgupgrade.sgml | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml index bf58a0a7d8..40c978730c 100644 --- a/doc/src/sgml/ref/pgupgrade.sgml +++ b/doc/src/sgml/ref/pgupgrade.sgml @@ -38,9 +38,9 @@ pg_upgrade (formerly called pg_migrator) allows data stored in PostgreSQL data files to be upgraded to a later PostgreSQL major version without the data dump/reload typically required for - major version upgrades, e.g. from 8.4.7 to the current major release + major version upgrades, e.g. from 9.6.3 to the current major release of PostgreSQL. It is not required for minor version upgrades, e.g. from - 9.0.1 to 9.0.4. + 9.6.2 to 9.6.3. @@ -66,7 +66,7 @@ pg_upgrade supports upgrades from 8.4.X and later to the current - major release of PostgreSQL, including snapshot and alpha releases. + major release of PostgreSQL, including snapshot and beta releases. @@ -207,7 +207,7 @@ If you are using a version-specific installation directory, e.g. - /opt/PostgreSQL/9.1, you do not need to move the old cluster. The + /opt/PostgreSQL/&majorversion;, you do not need to move the old cluster. The graphical installers all use version-specific installation directories. @@ -298,15 +298,15 @@ make prefix=/usr/local/pgsql.new install Make sure both database servers are stopped using, on Unix, e.g.: -pg_ctl -D /opt/PostgreSQL/8.4 stop -pg_ctl -D /opt/PostgreSQL/9.0 stop +pg_ctl -D /opt/PostgreSQL/9.6 stop +pg_ctl -D /opt/PostgreSQL/&majorversion; stop or on Windows, using the proper service names: -NET STOP postgresql-8.4 -NET STOP postgresql-9.0 +NET STOP postgresql-9.6 +NET STOP postgresql-&majorversion; @@ -366,17 +366,17 @@ NET STOP postgresql-9.0 RUNAS /USER:postgres "CMD.EXE" -SET PATH=%PATH%;C:\Program Files\PostgreSQL\9.0\bin; +SET PATH=%PATH%;C:\Program Files\PostgreSQL\&majorversion;\bin; and then run pg_upgrade with quoted directories, e.g.: pg_upgrade.exe - --old-datadir "C:/Program Files/PostgreSQL/8.4/data" - --new-datadir "C:/Program Files/PostgreSQL/9.0/data" - --old-bindir "C:/Program Files/PostgreSQL/8.4/bin" - --new-bindir "C:/Program Files/PostgreSQL/9.0/bin" + --old-datadir "C:/Program Files/PostgreSQL/9.6/data" + --new-datadir "C:/Program Files/PostgreSQL/&majorversion;/data" + --old-bindir "C:/Program Files/PostgreSQL/9.6/bin" + --new-bindir "C:/Program Files/PostgreSQL/&majorversion;/bin" Once started, pg_upgrade will verify the two clusters are compatible -- cgit v1.2.3 From 0436f6bde8848b7135f19dd7f8548b8c2ae89a34 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Tue, 13 Jun 2017 23:46:39 -0400 Subject: Disallow set-returning functions inside CASE or COALESCE. When we reimplemented SRFs in commit 69f4b9c85, our initial choice was to allow the behavior to vary from historical practice in cases where a SRF call appeared within a conditional-execution construct (currently, only CASE or COALESCE). But that was controversial to begin with, and subsequent discussion has resulted in a consensus that it's better to throw an error instead of executing the query differently from before, so long as we can provide a reasonably clear error message and a way to rewrite the query. Hence, add a parser mechanism to allow detection of such cases during parse analysis. The mechanism just requires storing, in the ParseState, a pointer to the set-returning FuncExpr or OpExpr most recently emitted by parse analysis. Then the parsing functions for CASE and COALESCE can detect the presence of a SRF in their arguments by noting whether this pointer changes while analyzing their arguments. Furthermore, if it does, it provides a suitable error cursor location for the complaint. (This means that if there's more than one SRF in the arguments, the error will point at the last one to be analyzed not the first. While connoisseurs of parsing behavior might find that odd, it's unlikely the average user would ever notice.) While at it, we can also provide more specific error messages than before about some pre-existing restrictions, such as no-SRFs-within-aggregates. Also, reject at parse time cases where a NULLIF or IS DISTINCT FROM construct would need to return a set. We've never supported that, but the restriction is depended on in more subtle ways now, so it seems wise to detect it at the start. Also, provide some documentation about how to rewrite a SRF-within-CASE query using a custom wrapper SRF. It turns out that the information_schema.user_mapping_options view contained an instance of exactly the behavior we're now forbidding; but rewriting it makes it more clear and safer too. initdb forced because of user_mapping_options change. Patch by me, with error message suggestions from Alvaro Herrera and Andres Freund, pursuant to a complaint from Regina Obe. Discussion: https://postgr.es/m/000001d2d5de$d8d66170$8a832450$@pcorp.us --- doc/src/sgml/xfunc.sgml | 83 +++++++++++++++++++++--------- src/backend/catalog/information_schema.sql | 8 +-- src/backend/executor/functions.c | 1 + src/backend/parser/parse_agg.c | 8 +++ src/backend/parser/parse_clause.c | 38 +++++++++++--- src/backend/parser/parse_expr.c | 79 +++++++++++++++++++++------- src/backend/parser/parse_func.c | 41 +++++++++++++-- src/backend/parser/parse_oper.c | 14 +++-- src/include/catalog/catversion.h | 2 +- src/include/parser/parse_func.h | 5 +- src/include/parser/parse_node.h | 5 ++ src/include/parser/parse_oper.h | 2 +- src/test/regress/expected/create_table.out | 2 +- src/test/regress/expected/rangefuncs.out | 12 ----- src/test/regress/expected/tsrf.out | 24 +++++++-- src/test/regress/sql/rangefuncs.sql | 9 ---- src/test/regress/sql/tsrf.sql | 7 +++ 17 files changed, 249 insertions(+), 91 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 05f4312bf3..1a6c3b9bc2 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -997,6 +997,29 @@ SELECT name, listchildren(name) FROM nodes; the LATERAL syntax. + + PostgreSQL's behavior for a set-returning function in a + query's select list is almost exactly the same as if the set-returning + function had been written in a LATERAL FROM-clause item + instead. For example, + +SELECT x, generate_series(1,5) AS g FROM tab; + + is almost equivalent to + +SELECT x, g FROM tab, LATERAL generate_series(1,5) AS g; + + It would be exactly the same, except that in this specific example, + the planner could choose to put g on the outside of the + nestloop join, since g has no actual lateral dependency + on tab. That would result in a different output row + order. Set-returning functions in the select list are always evaluated + as though they are on the inside of a nestloop join with the rest of + the FROM clause, so that the function(s) are run to + completion before the next row from the FROM clause is + considered. + + If there is more than one set-returning function in the query's select list, the behavior is similar to what you get from putting the functions @@ -1028,32 +1051,19 @@ SELECT srf1(srf2(x), srf3(y)), srf4(srf5(z)) FROM tab; - This behavior also means that set-returning functions will be evaluated - even when it might appear that they should be skipped because of a - conditional-evaluation construct, such as CASE - or COALESCE. For example, consider + Set-returning functions cannot be used within conditional-evaluation + constructs, such as CASE or COALESCE. For + example, consider SELECT x, CASE WHEN x > 0 THEN generate_series(1, 5) ELSE 0 END FROM tab; - It might seem that this should produce five repetitions of input - rows that have x > 0, and a single repetition of those - that do not; but actually it will produce five repetitions of every - input row. This is because generate_series() is run first, - and then the CASE expression is applied to its result rows. - The behavior is thus comparable to - -SELECT x, CASE WHEN x > 0 THEN g ELSE 0 END - FROM tab, LATERAL generate_series(1,5) AS g; - - It would be exactly the same, except that in this specific example, - the planner could choose to put g on the outside of the - nestloop join, since g has no actual lateral dependency - on tab. That would result in a different output row - order. Set-returning functions in the select list are always evaluated - as though they are on the inside of a nestloop join with the rest of - the FROM clause, so that the function(s) are run to - completion before the next row from the FROM clause is - considered. + It might seem that this should produce five repetitions of input rows + that have x > 0, and a single repetition of those that do + not; but actually, because generate_series(1, 5) would be + run in an implicit LATERAL FROM item before + the CASE expression is ever evaluated, it would produce five + repetitions of every input row. To reduce confusion, such cases produce + a parse-time error instead. @@ -1078,11 +1088,34 @@ SELECT x, CASE WHEN x > 0 THEN g ELSE 0 END functions. Also, nested set-returning functions did not work as described above; instead, a set-returning function could have at most one set-returning argument, and each nest of set-returning functions - was run independently. The behavior for conditional execution - (set-returning functions inside CASE etc) was different too. + was run independently. Also, conditional execution (set-returning + functions inside CASE etc) was previously allowed, + complicating things even more. Use of the LATERAL syntax is recommended when writing queries that need to work in older PostgreSQL versions, because that will give consistent results across different versions. + If you have a query that is relying on conditional execution of a + set-returning function, you may be able to fix it by moving the + conditional test into a custom set-returning function. For example, + +SELECT x, CASE WHEN y > 0 THEN generate_series(1, z) ELSE 5 END FROM tab; + + could become + +CREATE FUNCTION case_generate_series(cond bool, start int, fin int, els int) + RETURNS SETOF int AS $$ +BEGIN + IF cond THEN + RETURN QUERY SELECT generate_series(start, fin); + ELSE + RETURN QUERY SELECT els; + END IF; +END$$ LANGUAGE plpgsql; + +SELECT x, case_generate_series(y > 0, 1, z, 5) FROM tab; + + This formulation will work the same in all versions + of PostgreSQL. diff --git a/src/backend/catalog/information_schema.sql b/src/backend/catalog/information_schema.sql index cbcd6cfbc1..98bcfa08c6 100644 --- a/src/backend/catalog/information_schema.sql +++ b/src/backend/catalog/information_schema.sql @@ -2936,12 +2936,14 @@ CREATE VIEW user_mapping_options AS SELECT authorization_identifier, foreign_server_catalog, foreign_server_name, - CAST((pg_options_to_table(um.umoptions)).option_name AS sql_identifier) AS option_name, + CAST(opts.option_name AS sql_identifier) AS option_name, CAST(CASE WHEN (umuser <> 0 AND authorization_identifier = current_user) OR (umuser = 0 AND pg_has_role(srvowner, 'USAGE')) - OR (SELECT rolsuper FROM pg_authid WHERE rolname = current_user) THEN (pg_options_to_table(um.umoptions)).option_value + OR (SELECT rolsuper FROM pg_authid WHERE rolname = current_user) + THEN opts.option_value ELSE NULL END AS character_data) AS option_value - FROM _pg_user_mappings um; + FROM _pg_user_mappings um, + pg_options_to_table(um.umoptions) opts; GRANT SELECT ON user_mapping_options TO PUBLIC; diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index a35ba32e6d..89aea2fe5c 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -388,6 +388,7 @@ sql_fn_post_column_ref(ParseState *pstate, ColumnRef *cref, Node *var) param = ParseFuncOrColumn(pstate, list_make1(subfield), list_make1(param), + pstate->p_last_srf, NULL, cref->location); } diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c index efe1c371ef..6218ee94fa 100644 --- a/src/backend/parser/parse_agg.c +++ b/src/backend/parser/parse_agg.c @@ -705,6 +705,14 @@ check_agg_arguments_walker(Node *node, } /* Continue and descend into subtree */ } + /* We can throw error on sight for a set-returning function */ + if ((IsA(node, FuncExpr) &&((FuncExpr *) node)->funcretset) || + (IsA(node, OpExpr) &&((OpExpr *) node)->opretset)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("aggregate function calls cannot contain set-returning function calls"), + errhint("You might be able to move the set-returning function into a LATERAL FROM item."), + parser_errposition(context->pstate, exprLocation(node)))); /* We can throw error on sight for a window function */ if (IsA(node, WindowFunc)) ereport(ERROR, diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c index 27dd49d301..3d5b20836f 100644 --- a/src/backend/parser/parse_clause.c +++ b/src/backend/parser/parse_clause.c @@ -572,6 +572,8 @@ transformRangeFunction(ParseState *pstate, RangeFunction *r) List *pair = (List *) lfirst(lc); Node *fexpr; List *coldeflist; + Node *newfexpr; + Node *last_srf; /* Disassemble the function-call/column-def-list pairs */ Assert(list_length(pair) == 2); @@ -618,13 +620,25 @@ transformRangeFunction(ParseState *pstate, RangeFunction *r) Node *arg = (Node *) lfirst(lc); FuncCall *newfc; + last_srf = pstate->p_last_srf; + newfc = makeFuncCall(SystemFuncName("unnest"), list_make1(arg), fc->location); - funcexprs = lappend(funcexprs, - transformExpr(pstate, (Node *) newfc, - EXPR_KIND_FROM_FUNCTION)); + newfexpr = transformExpr(pstate, (Node *) newfc, + EXPR_KIND_FROM_FUNCTION); + + /* nodeFunctionscan.c requires SRFs to be at top level */ + if (pstate->p_last_srf != last_srf && + pstate->p_last_srf != newfexpr) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("set-returning functions must appear at top level of FROM"), + parser_errposition(pstate, + exprLocation(pstate->p_last_srf)))); + + funcexprs = lappend(funcexprs, newfexpr); funcnames = lappend(funcnames, FigureColname((Node *) newfc)); @@ -638,9 +652,21 @@ transformRangeFunction(ParseState *pstate, RangeFunction *r) } /* normal case ... */ - funcexprs = lappend(funcexprs, - transformExpr(pstate, fexpr, - EXPR_KIND_FROM_FUNCTION)); + last_srf = pstate->p_last_srf; + + newfexpr = transformExpr(pstate, fexpr, + EXPR_KIND_FROM_FUNCTION); + + /* nodeFunctionscan.c requires SRFs to be at top level */ + if (pstate->p_last_srf != last_srf && + pstate->p_last_srf != newfexpr) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("set-returning functions must appear at top level of FROM"), + parser_errposition(pstate, + exprLocation(pstate->p_last_srf)))); + + funcexprs = lappend(funcexprs, newfexpr); funcnames = lappend(funcnames, FigureColname(fexpr)); diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index 92101c9103..65bd51e1fd 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -118,8 +118,7 @@ static Node *transformCurrentOfExpr(ParseState *pstate, CurrentOfExpr *cexpr); static Node *transformColumnRef(ParseState *pstate, ColumnRef *cref); static Node *transformWholeRowRef(ParseState *pstate, RangeTblEntry *rte, int location); -static Node *transformIndirection(ParseState *pstate, Node *basenode, - List *indirection); +static Node *transformIndirection(ParseState *pstate, A_Indirection *ind); static Node *transformTypeCast(ParseState *pstate, TypeCast *tc); static Node *transformCollateClause(ParseState *pstate, CollateClause *c); static Node *make_row_comparison_op(ParseState *pstate, List *opname, @@ -192,14 +191,8 @@ transformExprRecurse(ParseState *pstate, Node *expr) } case T_A_Indirection: - { - A_Indirection *ind = (A_Indirection *) expr; - - result = transformExprRecurse(pstate, ind->arg); - result = transformIndirection(pstate, result, - ind->indirection); - break; - } + result = transformIndirection(pstate, (A_Indirection *) expr); + break; case T_A_ArrayExpr: result = transformArrayExpr(pstate, (A_ArrayExpr *) expr, @@ -439,11 +432,12 @@ unknown_attribute(ParseState *pstate, Node *relref, char *attname, } static Node * -transformIndirection(ParseState *pstate, Node *basenode, List *indirection) +transformIndirection(ParseState *pstate, A_Indirection *ind) { - Node *result = basenode; + Node *last_srf = pstate->p_last_srf; + Node *result = transformExprRecurse(pstate, ind->arg); List *subscripts = NIL; - int location = exprLocation(basenode); + int location = exprLocation(result); ListCell *i; /* @@ -451,7 +445,7 @@ transformIndirection(ParseState *pstate, Node *basenode, List *indirection) * subscripting. Adjacent A_Indices nodes have to be treated as a single * multidimensional subscript operation. */ - foreach(i, indirection) + foreach(i, ind->indirection) { Node *n = lfirst(i); @@ -484,6 +478,7 @@ transformIndirection(ParseState *pstate, Node *basenode, List *indirection) newresult = ParseFuncOrColumn(pstate, list_make1(n), list_make1(result), + last_srf, NULL, location); if (newresult == NULL) @@ -632,6 +627,7 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref) node = ParseFuncOrColumn(pstate, list_make1(makeString(colname)), list_make1(node), + pstate->p_last_srf, NULL, cref->location); } @@ -678,6 +674,7 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref) node = ParseFuncOrColumn(pstate, list_make1(makeString(colname)), list_make1(node), + pstate->p_last_srf, NULL, cref->location); } @@ -737,6 +734,7 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref) node = ParseFuncOrColumn(pstate, list_make1(makeString(colname)), list_make1(node), + pstate->p_last_srf, NULL, cref->location); } @@ -927,6 +925,8 @@ transformAExprOp(ParseState *pstate, A_Expr *a) else { /* Ordinary scalar operator */ + Node *last_srf = pstate->p_last_srf; + lexpr = transformExprRecurse(pstate, lexpr); rexpr = transformExprRecurse(pstate, rexpr); @@ -934,6 +934,7 @@ transformAExprOp(ParseState *pstate, A_Expr *a) a->name, lexpr, rexpr, + last_srf, a->location); } @@ -1053,6 +1054,7 @@ transformAExprNullIf(ParseState *pstate, A_Expr *a) a->name, lexpr, rexpr, + pstate->p_last_srf, a->location); /* @@ -1063,6 +1065,12 @@ transformAExprNullIf(ParseState *pstate, A_Expr *a) (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("NULLIF requires = operator to yield boolean"), parser_errposition(pstate, a->location))); + if (result->opretset) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + /* translator: %s is name of a SQL construct, eg NULLIF */ + errmsg("%s must not return a set", "NULLIF"), + parser_errposition(pstate, a->location))); /* * ... but the NullIfExpr will yield the first operand's type. @@ -1266,6 +1274,7 @@ transformAExprIn(ParseState *pstate, A_Expr *a) a->name, copyObject(lexpr), rexpr, + pstate->p_last_srf, a->location); } @@ -1430,6 +1439,7 @@ transformBoolExpr(ParseState *pstate, BoolExpr *a) static Node * transformFuncCall(ParseState *pstate, FuncCall *fn) { + Node *last_srf = pstate->p_last_srf; List *targs; ListCell *args; @@ -1465,6 +1475,7 @@ transformFuncCall(ParseState *pstate, FuncCall *fn) return ParseFuncOrColumn(pstate, fn->funcname, targs, + last_srf, fn, fn->location); } @@ -1619,7 +1630,8 @@ transformMultiAssignRef(ParseState *pstate, MultiAssignRef *maref) static Node * transformCaseExpr(ParseState *pstate, CaseExpr *c) { - CaseExpr *newc; + CaseExpr *newc = makeNode(CaseExpr); + Node *last_srf = pstate->p_last_srf; Node *arg; CaseTestExpr *placeholder; List *newargs; @@ -1628,8 +1640,6 @@ transformCaseExpr(ParseState *pstate, CaseExpr *c) Node *defresult; Oid ptype; - newc = makeNode(CaseExpr); - /* transform the test expression, if any */ arg = transformExprRecurse(pstate, (Node *) c->arg); @@ -1741,6 +1751,17 @@ transformCaseExpr(ParseState *pstate, CaseExpr *c) "CASE/WHEN"); } + /* if any subexpression contained a SRF, complain */ + if (pstate->p_last_srf != last_srf) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + /* translator: %s is name of a SQL construct, eg GROUP BY */ + errmsg("set-returning functions are not allowed in %s", + "CASE"), + errhint("You might be able to move the set-returning function into a LATERAL FROM item."), + parser_errposition(pstate, + exprLocation(pstate->p_last_srf)))); + newc->location = c->location; return (Node *) newc; @@ -2177,6 +2198,7 @@ static Node * transformCoalesceExpr(ParseState *pstate, CoalesceExpr *c) { CoalesceExpr *newc = makeNode(CoalesceExpr); + Node *last_srf = pstate->p_last_srf; List *newargs = NIL; List *newcoercedargs = NIL; ListCell *args; @@ -2205,6 +2227,17 @@ transformCoalesceExpr(ParseState *pstate, CoalesceExpr *c) newcoercedargs = lappend(newcoercedargs, newe); } + /* if any subexpression contained a SRF, complain */ + if (pstate->p_last_srf != last_srf) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + /* translator: %s is name of a SQL construct, eg GROUP BY */ + errmsg("set-returning functions are not allowed in %s", + "COALESCE"), + errhint("You might be able to move the set-returning function into a LATERAL FROM item."), + parser_errposition(pstate, + exprLocation(pstate->p_last_srf)))); + newc->args = newcoercedargs; newc->location = c->location; return (Node *) newc; @@ -2793,7 +2826,8 @@ make_row_comparison_op(ParseState *pstate, List *opname, Node *rarg = (Node *) lfirst(r); OpExpr *cmp; - cmp = castNode(OpExpr, make_op(pstate, opname, larg, rarg, location)); + cmp = castNode(OpExpr, make_op(pstate, opname, larg, rarg, + pstate->p_last_srf, location)); /* * We don't use coerce_to_boolean here because we insist on the @@ -3000,12 +3034,19 @@ make_distinct_op(ParseState *pstate, List *opname, Node *ltree, Node *rtree, { Expr *result; - result = make_op(pstate, opname, ltree, rtree, location); + result = make_op(pstate, opname, ltree, rtree, + pstate->p_last_srf, location); if (((OpExpr *) result)->opresulttype != BOOLOID) ereport(ERROR, (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("IS DISTINCT FROM requires = operator to yield boolean"), parser_errposition(pstate, location))); + if (((OpExpr *) result)->opretset) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + /* translator: %s is name of a SQL construct, eg NULLIF */ + errmsg("%s must not return a set", "IS DISTINCT FROM"), + parser_errposition(pstate, location))); /* * We rely on DistinctExpr and OpExpr being same struct diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c index 55853c20bb..34f1cf82ee 100644 --- a/src/backend/parser/parse_func.c +++ b/src/backend/parser/parse_func.c @@ -64,10 +64,14 @@ static Node *ParseComplexProjection(ParseState *pstate, char *funcname, * * The argument expressions (in fargs) must have been transformed * already. However, nothing in *fn has been transformed. + * + * last_srf should be a copy of pstate->p_last_srf from just before we + * started transforming fargs. If the caller knows that fargs couldn't + * contain any SRF calls, last_srf can just be pstate->p_last_srf. */ Node * ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs, - FuncCall *fn, int location) + Node *last_srf, FuncCall *fn, int location) { bool is_column = (fn == NULL); List *agg_order = (fn ? fn->agg_order : NIL); @@ -628,7 +632,7 @@ ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs, /* if it returns a set, check that's OK */ if (retset) - check_srf_call_placement(pstate, location); + check_srf_call_placement(pstate, last_srf, location); /* build the appropriate output structure */ if (fdresult == FUNCDETAIL_NORMAL) @@ -759,6 +763,17 @@ ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs, errmsg("FILTER is not implemented for non-aggregate window functions"), parser_errposition(pstate, location))); + /* + * Window functions can't either take or return sets + */ + if (pstate->p_last_srf != last_srf) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("window function calls cannot contain set-returning function calls"), + errhint("You might be able to move the set-returning function into a LATERAL FROM item."), + parser_errposition(pstate, + exprLocation(pstate->p_last_srf)))); + if (retset) ereport(ERROR, (errcode(ERRCODE_INVALID_FUNCTION_DEFINITION), @@ -771,6 +786,10 @@ ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs, retval = (Node *) wfunc; } + /* if it returns a set, remember it for error checks at higher levels */ + if (retset) + pstate->p_last_srf = retval; + return retval; } @@ -2083,9 +2102,13 @@ LookupAggWithArgs(ObjectWithArgs *agg, bool noError) * and throw a nice error if not. * * A side-effect is to set pstate->p_hasTargetSRFs true if appropriate. + * + * last_srf should be a copy of pstate->p_last_srf from just before we + * started transforming the function's arguments. This allows detection + * of whether the SRF's arguments contain any SRFs. */ void -check_srf_call_placement(ParseState *pstate, int location) +check_srf_call_placement(ParseState *pstate, Node *last_srf, int location) { const char *err; bool errkind; @@ -2121,7 +2144,15 @@ check_srf_call_placement(ParseState *pstate, int location) errkind = true; break; case EXPR_KIND_FROM_FUNCTION: - /* okay ... but we can't check nesting here */ + /* okay, but we don't allow nested SRFs here */ + /* errmsg is chosen to match transformRangeFunction() */ + /* errposition should point to the inner SRF */ + if (pstate->p_last_srf != last_srf) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("set-returning functions must appear at top level of FROM"), + parser_errposition(pstate, + exprLocation(pstate->p_last_srf)))); break; case EXPR_KIND_WHERE: errkind = true; @@ -2202,7 +2233,7 @@ check_srf_call_placement(ParseState *pstate, int location) err = _("set-returning functions are not allowed in trigger WHEN conditions"); break; case EXPR_KIND_PARTITION_EXPRESSION: - err = _("set-returning functions are not allowed in partition key expression"); + err = _("set-returning functions are not allowed in partition key expressions"); break; /* diff --git a/src/backend/parser/parse_oper.c b/src/backend/parser/parse_oper.c index e40b10d4f6..4b1db76e19 100644 --- a/src/backend/parser/parse_oper.c +++ b/src/backend/parser/parse_oper.c @@ -735,12 +735,14 @@ op_error(ParseState *pstate, List *op, char oprkind, * Transform operator expression ensuring type compatibility. * This is where some type conversion happens. * - * As with coerce_type, pstate may be NULL if no special unknown-Param - * processing is wanted. + * last_srf should be a copy of pstate->p_last_srf from just before we + * started transforming the operator's arguments; this is used for nested-SRF + * detection. If the caller will throw an error anyway for a set-returning + * expression, it's okay to cheat and just pass pstate->p_last_srf. */ Expr * make_op(ParseState *pstate, List *opname, Node *ltree, Node *rtree, - int location) + Node *last_srf, int location) { Oid ltypeId, rtypeId; @@ -843,7 +845,11 @@ make_op(ParseState *pstate, List *opname, Node *ltree, Node *rtree, /* if it returns a set, check that's OK */ if (result->opretset) - check_srf_call_placement(pstate, location); + { + check_srf_call_placement(pstate, last_srf, location); + /* ... and remember it for error checks at higher levels */ + pstate->p_last_srf = (Node *) result; + } ReleaseSysCache(tup); diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h index ae34f75a84..8b2ec21509 100644 --- a/src/include/catalog/catversion.h +++ b/src/include/catalog/catversion.h @@ -53,6 +53,6 @@ */ /* yyyymmddN */ -#define CATALOG_VERSION_NO 201706131 +#define CATALOG_VERSION_NO 201706141 #endif diff --git a/src/include/parser/parse_func.h b/src/include/parser/parse_func.h index 5be1812242..68572e9d2a 100644 --- a/src/include/parser/parse_func.h +++ b/src/include/parser/parse_func.h @@ -31,7 +31,7 @@ typedef enum extern Node *ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs, - FuncCall *fn, int location); + Node *last_srf, FuncCall *fn, int location); extern FuncDetailCode func_get_detail(List *funcname, List *fargs, List *fargnames, @@ -67,6 +67,7 @@ extern Oid LookupFuncWithArgs(ObjectWithArgs *func, extern Oid LookupAggWithArgs(ObjectWithArgs *agg, bool noError); -extern void check_srf_call_placement(ParseState *pstate, int location); +extern void check_srf_call_placement(ParseState *pstate, Node *last_srf, + int location); #endif /* PARSE_FUNC_H */ diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h index 0b54840e29..6a3507f3b1 100644 --- a/src/include/parser/parse_node.h +++ b/src/include/parser/parse_node.h @@ -157,6 +157,9 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param, * p_hasAggs, p_hasWindowFuncs, etc: true if we've found any of the indicated * constructs in the query. * + * p_last_srf: the set-returning FuncExpr or OpExpr most recently found in + * the query, or NULL if none. + * * p_pre_columnref_hook, etc: optional parser hook functions for modifying the * interpretation of ColumnRefs and ParamRefs. * @@ -199,6 +202,8 @@ struct ParseState bool p_hasSubLinks; bool p_hasModifyingCTE; + Node *p_last_srf; /* most recent set-returning func/op found */ + /* * Optional hook functions for parser callbacks. These are null unless * set up by the caller of make_parsestate. diff --git a/src/include/parser/parse_oper.h b/src/include/parser/parse_oper.h index d783b37f0f..ab3c4aa62f 100644 --- a/src/include/parser/parse_oper.h +++ b/src/include/parser/parse_oper.h @@ -59,7 +59,7 @@ extern Oid oprfuncid(Operator op); /* Build expression tree for an operator invocation */ extern Expr *make_op(ParseState *pstate, List *opname, - Node *ltree, Node *rtree, int location); + Node *ltree, Node *rtree, Node *last_srf, int location); extern Expr *make_scalar_array_op(ParseState *pstate, List *opname, bool useOr, Node *ltree, Node *rtree, int location); diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out index 5136506dff..fb8745be04 100644 --- a/src/test/regress/expected/create_table.out +++ b/src/test/regress/expected/create_table.out @@ -315,7 +315,7 @@ CREATE FUNCTION retset (a int) RETURNS SETOF int AS $$ SELECT 1; $$ LANGUAGE SQL CREATE TABLE partitioned ( a int ) PARTITION BY RANGE (retset(a)); -ERROR: set-returning functions are not allowed in partition key expression +ERROR: set-returning functions are not allowed in partition key expressions DROP FUNCTION retset(int); CREATE TABLE partitioned ( a int diff --git a/src/test/regress/expected/rangefuncs.out b/src/test/regress/expected/rangefuncs.out index 4b6170dea5..5c82614e05 100644 --- a/src/test/regress/expected/rangefuncs.out +++ b/src/test/regress/expected/rangefuncs.out @@ -1969,18 +1969,6 @@ select * from foobar(); -- fail ERROR: function return row and query-specified return row do not match DETAIL: Returned row contains 3 attributes, but query expects 2. drop function foobar(); --- check behavior when a function's input sometimes returns a set (bug #8228) -SELECT *, - lower(CASE WHEN id = 2 THEN (regexp_matches(str, '^0*([1-9]\d+)$'))[1] - ELSE str - END) -FROM - (VALUES (1,''), (2,'0000000049404'), (3,'FROM 10000000876')) v(id, str); - id | str | lower -----+---------------+------- - 2 | 0000000049404 | 49404 -(1 row) - -- check whole-row-Var handling in nested lateral functions (bug #11703) create function extractq2(t int8_tbl) returns int8 as $$ select t.q2 diff --git a/src/test/regress/expected/tsrf.out b/src/test/regress/expected/tsrf.out index c8ae361e75..ef02ba7465 100644 --- a/src/test/regress/expected/tsrf.out +++ b/src/test/regress/expected/tsrf.out @@ -41,6 +41,11 @@ SELECT generate_series(1, generate_series(1, 3)); 3 (6 rows) +-- but we've traditionally rejected the same in FROM +SELECT * FROM generate_series(1, generate_series(1, 3)); +ERROR: set-returning functions must appear at top level of FROM +LINE 1: SELECT * FROM generate_series(1, generate_series(1, 3)); + ^ -- srf, with two SRF arguments SELECT generate_series(generate_series(1,3), generate_series(2, 4)); generate_series @@ -190,16 +195,29 @@ SELECT few.dataa, count(*) FROM few WHERE dataa = 'a' GROUP BY few.dataa, unnest a | 4 (2 rows) +-- SRFs are not allowed if they'd need to be conditionally executed +SELECT q1, case when q1 > 0 then generate_series(1,3) else 0 end FROM int8_tbl; +ERROR: set-returning functions are not allowed in CASE +LINE 1: SELECT q1, case when q1 > 0 then generate_series(1,3) else 0... + ^ +HINT: You might be able to move the set-returning function into a LATERAL FROM item. +SELECT q1, coalesce(generate_series(1,3), 0) FROM int8_tbl; +ERROR: set-returning functions are not allowed in COALESCE +LINE 1: SELECT q1, coalesce(generate_series(1,3), 0) FROM int8_tbl; + ^ +HINT: You might be able to move the set-returning function into a LATERAL FROM item. -- SRFs are not allowed in aggregate arguments SELECT min(generate_series(1, 3)) FROM few; -ERROR: set-valued function called in context that cannot accept a set +ERROR: aggregate function calls cannot contain set-returning function calls LINE 1: SELECT min(generate_series(1, 3)) FROM few; ^ +HINT: You might be able to move the set-returning function into a LATERAL FROM item. -- SRFs are not allowed in window function arguments, either SELECT min(generate_series(1, 3)) OVER() FROM few; -ERROR: set-valued function called in context that cannot accept a set +ERROR: window function calls cannot contain set-returning function calls LINE 1: SELECT min(generate_series(1, 3)) OVER() FROM few; ^ +HINT: You might be able to move the set-returning function into a LATERAL FROM item. -- SRFs are normally computed after window functions SELECT id,lag(id) OVER(), count(*) OVER(), generate_series(1,3) FROM few; id | lag | count | generate_series @@ -427,7 +445,7 @@ SELECT int4mul(generate_series(1,2), 10); -- but SRFs in function RTEs must be at top level (annoying restriction) SELECT * FROM int4mul(generate_series(1,2), 10); -ERROR: set-valued function called in context that cannot accept a set +ERROR: set-returning functions must appear at top level of FROM LINE 1: SELECT * FROM int4mul(generate_series(1,2), 10); ^ -- DISTINCT ON is evaluated before tSRF evaluation if SRF is not diff --git a/src/test/regress/sql/rangefuncs.sql b/src/test/regress/sql/rangefuncs.sql index 4ed84b1f2b..442d397d4a 100644 --- a/src/test/regress/sql/rangefuncs.sql +++ b/src/test/regress/sql/rangefuncs.sql @@ -600,15 +600,6 @@ select * from foobar(); -- fail drop function foobar(); --- check behavior when a function's input sometimes returns a set (bug #8228) - -SELECT *, - lower(CASE WHEN id = 2 THEN (regexp_matches(str, '^0*([1-9]\d+)$'))[1] - ELSE str - END) -FROM - (VALUES (1,''), (2,'0000000049404'), (3,'FROM 10000000876')) v(id, str); - -- check whole-row-Var handling in nested lateral functions (bug #11703) create function extractq2(t int8_tbl) returns int8 as $$ diff --git a/src/test/regress/sql/tsrf.sql b/src/test/regress/sql/tsrf.sql index 417e78c53d..45de099e4d 100644 --- a/src/test/regress/sql/tsrf.sql +++ b/src/test/regress/sql/tsrf.sql @@ -14,6 +14,9 @@ SELECT generate_series(1, 2), generate_series(1,4); -- srf, with SRF argument SELECT generate_series(1, generate_series(1, 3)); +-- but we've traditionally rejected the same in FROM +SELECT * FROM generate_series(1, generate_series(1, 3)); + -- srf, with two SRF arguments SELECT generate_series(generate_series(1,3), generate_series(2, 4)); @@ -51,6 +54,10 @@ SELECT dataa, generate_series(1,1), count(*) FROM few GROUP BY 1, 2 HAVING count SELECT few.dataa, count(*) FROM few WHERE dataa = 'a' GROUP BY few.dataa ORDER BY 2; SELECT few.dataa, count(*) FROM few WHERE dataa = 'a' GROUP BY few.dataa, unnest('{1,1,3}'::int[]) ORDER BY 2; +-- SRFs are not allowed if they'd need to be conditionally executed +SELECT q1, case when q1 > 0 then generate_series(1,3) else 0 end FROM int8_tbl; +SELECT q1, coalesce(generate_series(1,3), 0) FROM int8_tbl; + -- SRFs are not allowed in aggregate arguments SELECT min(generate_series(1, 3)) FROM few; -- cgit v1.2.3 From 0d9bdbcaae00dac89a82c25e66e4a859130e2fe8 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 14 Jun 2017 11:09:33 -0400 Subject: Put documentation of options and commands in more alphabetical order --- doc/src/sgml/ref/initdb.sgml | 4 +- doc/src/sgml/ref/pg_dump.sgml | 28 ++++++------- doc/src/sgml/ref/psql-ref.sgml | 89 +++++++++++++++++++++--------------------- src/bin/psql/help.c | 14 +++---- 4 files changed, 68 insertions(+), 67 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml index 3ce99e248b..6efb2e442d 100644 --- a/doc/src/sgml/ref/initdb.sgml +++ b/doc/src/sgml/ref/initdb.sgml @@ -276,8 +276,8 @@ PostgreSQL documentation - - + + Sets the default text search configuration. diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml index bb0bf5d566..bafa031e1a 100644 --- a/doc/src/sgml/ref/pg_dump.sgml +++ b/doc/src/sgml/ref/pg_dump.sgml @@ -816,6 +816,20 @@ PostgreSQL documentation + + + + + By default, pg_dump will wait for all files + to be written safely to disk. This option causes + pg_dump to return without waiting, which is + faster, but means that a subsequent operating system crash can leave + the dump corrupt. Generally, this option is useful for testing + but should not be used when dumping data from production installation. + + + + @@ -856,20 +870,6 @@ PostgreSQL documentation - - - - - By default, pg_dump will wait for all files - to be written safely to disk. This option causes - pg_dump to return without waiting, which is - faster, but means that a subsequent operating system crash can leave - the dump corrupt. Generally, this option is useful for testing - but should not be used when dumping data from production installation. - - - - diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index 3b86612862..e6eba21eda 100644 --- a/doc/src/sgml/ref/psql-ref.sgml +++ b/doc/src/sgml/ref/psql-ref.sgml @@ -1250,6 +1250,23 @@ testdb=> + + \dD[S+] [ pattern ] + + + Lists domains. If pattern + is specified, only domains whose names match the pattern are shown. + By default, only user-created objects are shown; supply a + pattern or the S modifier to include system + objects. + If + is appended to the command name, each object + is listed with its associated permissions and description. + + + + + \ddp [ pattern ] @@ -1272,23 +1289,6 @@ testdb=> - - \dD[S+] [ pattern ] - - - Lists domains. If pattern - is specified, only domains whose names match the pattern are shown. - By default, only user-created objects are shown; supply a - pattern or the S modifier to include system - objects. - If + is appended to the command name, each object - is listed with its associated permissions and description. - - - - - \dE[S+] [ pattern ] \di[S+] [ pattern ] @@ -1944,18 +1944,6 @@ Tue Oct 26 21:40:57 CEST 1999 - - \gx [ filename ] - \gx [ |command ] - - - \gx is equivalent to \g, but - forces expanded output mode for this query. See \x. - - - - - \gexec @@ -2046,6 +2034,19 @@ hello 10 + + + \gx [ filename ] + \gx [ |command ] + + + \gx is equivalent to \g, but + forces expanded output mode for this query. See \x. + + + + + \h or \help [ command ] @@ -2117,21 +2118,6 @@ hello 10 - - \ir or \include_relative filename - - - The \ir command is similar to \i, but resolves - relative file names differently. When executing in interactive mode, - the two commands behave identically. However, when invoked from a - script, \ir interprets file names relative to the - directory in which the script is located, rather than the current - working directory. - - - - - \if expression \elif expression @@ -2221,6 +2207,21 @@ SELECT + + \ir or \include_relative filename + + + The \ir command is similar to \i, but resolves + relative file names differently. When executing in interactive mode, + the two commands behave identically. However, when invoked from a + script, \ir interprets file names relative to the + directory in which the script is located, rather than the current + working directory. + + + + + \l[+] or \list[+] [ pattern ] diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c index ac435220e6..f097b06594 100644 --- a/src/bin/psql/help.c +++ b/src/bin/psql/help.c @@ -171,13 +171,13 @@ slashUsage(unsigned short int pager) fprintf(output, _("General\n")); fprintf(output, _(" \\copyright show PostgreSQL usage and distribution terms\n")); + fprintf(output, _(" \\crosstabview [COLUMNS] execute query and display results in crosstab\n")); fprintf(output, _(" \\errverbose show most recent error message at maximum verbosity\n")); fprintf(output, _(" \\g [FILE] or ; execute query (and send results to file or |pipe)\n")); - fprintf(output, _(" \\gx [FILE] as \\g, but forces expanded output mode\n")); fprintf(output, _(" \\gexec execute query, then execute each value in its result\n")); fprintf(output, _(" \\gset [PREFIX] execute query and store results in psql variables\n")); + fprintf(output, _(" \\gx [FILE] as \\g, but forces expanded output mode\n")); fprintf(output, _(" \\q quit psql\n")); - fprintf(output, _(" \\crosstabview [COLUMNS] execute query and display results in crosstab\n")); fprintf(output, _(" \\watch [SEC] execute query every SEC seconds\n")); fprintf(output, "\n"); @@ -227,8 +227,9 @@ slashUsage(unsigned short int pager) fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n")); fprintf(output, _(" \\dC[+] [PATTERN] list casts\n")); fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n")); - fprintf(output, _(" \\ddp [PATTERN] list default privileges\n")); fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n")); + fprintf(output, _(" \\ddp [PATTERN] list default privileges\n")); + fprintf(output, _(" \\dE[S+] [PATTERN] list foreign tables\n")); fprintf(output, _(" \\det[+] [PATTERN] list foreign tables\n")); fprintf(output, _(" \\des[+] [PATTERN] list foreign servers\n")); fprintf(output, _(" \\deu[+] [PATTERN] list user mappings\n")); @@ -255,7 +256,6 @@ slashUsage(unsigned short int pager) fprintf(output, _(" \\dT[S+] [PATTERN] list data types\n")); fprintf(output, _(" \\du[S+] [PATTERN] list roles\n")); fprintf(output, _(" \\dv[S+] [PATTERN] list views\n")); - fprintf(output, _(" \\dE[S+] [PATTERN] list foreign tables\n")); fprintf(output, _(" \\dx[+] [PATTERN] list extensions\n")); fprintf(output, _(" \\dy [PATTERN] list event triggers\n")); fprintf(output, _(" \\l[+] [PATTERN] list databases\n")); @@ -289,9 +289,9 @@ slashUsage(unsigned short int pager) else fprintf(output, _(" \\c[onnect] {[DBNAME|- USER|- HOST|- PORT|-] | conninfo}\n" " connect to new database (currently no connection)\n")); + fprintf(output, _(" \\conninfo display information about current connection\n")); fprintf(output, _(" \\encoding [ENCODING] show or set client encoding\n")); fprintf(output, _(" \\password [USERNAME] securely change the password for a user\n")); - fprintf(output, _(" \\conninfo display information about current connection\n")); fprintf(output, "\n"); fprintf(output, _("Operating System\n")); @@ -413,10 +413,10 @@ helpVariables(unsigned short int pager) fprintf(output, _(" PGAPPNAME same as the application_name connection parameter\n")); fprintf(output, _(" PGDATABASE same as the dbname connection parameter\n")); fprintf(output, _(" PGHOST same as the host connection parameter\n")); - fprintf(output, _(" PGPORT same as the port connection parameter\n")); - fprintf(output, _(" PGUSER same as the user connection parameter\n")); fprintf(output, _(" PGPASSWORD connection password (not recommended)\n")); fprintf(output, _(" PGPASSFILE password file name\n")); + fprintf(output, _(" PGPORT same as the port connection parameter\n")); + fprintf(output, _(" PGUSER same as the user connection parameter\n")); fprintf(output, _(" PSQL_EDITOR, EDITOR, VISUAL\n" " editor used by the \\e, \\ef, and \\ev commands\n")); fprintf(output, _(" PSQL_EDITOR_LINENUMBER_ARG\n" -- cgit v1.2.3 From a12c09ad86e60a8acb269744b8ee86429dda2cd8 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Wed, 14 Jun 2017 11:44:34 -0400 Subject: Improve release note text about set-returning-function changes. Paul Ramsey griped about this awhile ago, but I'd been holding fire on changing it until we settled what to do about the CASE/COALESCE issue. Discussion: https://postgr.es/m/CACowWR0AMyUt5fwtvuDqWyYNdp-hQJj9XqSxJR6YM9sKWov=_w@mail.gmail.com --- doc/src/sgml/release-10.sgml | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml index b10086bdb0..75c48c0f59 100644 --- a/doc/src/sgml/release-10.sgml +++ b/doc/src/sgml/release-10.sgml @@ -129,18 +129,29 @@ - Allow COALESCE and CASE to return multiple - rows when evaluating set-returning functions (Andres Freund). + Change the implementation of set-returning functions appearing in + a query's SELECT list (Andres Freund) - This also prevents conditionals like CASE from - controlling the execution of set-returning functions because - set-returning functions are now executed earlier. + Set-returning functions are now evaluated before evaluation of scalar + expressions in the SELECT list, much as though they had + been placed in a LATERAL FROM-clause item. This allows + saner semantics for cases where multiple set-returning functions are + present. If they return different numbers of rows, the shorter results + are extended to match the longest result by adding nulls. Previously + the results were cycled until they all terminated at the same time, + producing a number of rows equal to the least common multiple of the + functions' periods. In addition, set-returning functions are now + disallowed within CASE and COALESCE constructs. + For more information + see . @@ -1701,20 +1712,6 @@ - - - - Fix check_srf_call_placement() to handle - VALUES cases correctly (Tom Lane) - - - - NEED TEXT. - - - -- cgit v1.2.3 From f0cfff9da2c577a19dd6a15ffc7b404693b700bc Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 14 Jun 2017 13:55:43 -0400 Subject: doc: Whitespace fixes in man pages --- doc/src/sgml/ref/alter_aggregate.sgml | 3 +-- doc/src/sgml/ref/alter_publication.sgml | 3 +-- doc/src/sgml/ref/alter_subscription.sgml | 3 +-- doc/src/sgml/ref/alter_system.sgml | 3 +-- doc/src/sgml/ref/alter_type.sgml | 3 +-- doc/src/sgml/ref/create_access_method.sgml | 3 +-- doc/src/sgml/ref/create_publication.sgml | 3 +-- doc/src/sgml/ref/create_subscription.sgml | 3 +-- doc/src/sgml/ref/drop_aggregate.sgml | 3 +-- doc/src/sgml/ref/drop_policy.sgml | 3 +-- doc/src/sgml/ref/drop_publication.sgml | 4 +--- doc/src/sgml/ref/drop_statistics.sgml | 4 +--- doc/src/sgml/ref/drop_subscription.sgml | 4 +--- doc/src/sgml/ref/import_foreign_schema.sgml | 4 +--- doc/src/sgml/ref/insert.sgml | 3 +-- 15 files changed, 15 insertions(+), 34 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/alter_aggregate.sgml b/doc/src/sgml/ref/alter_aggregate.sgml index 3aa7c259da..7b7616ca01 100644 --- a/doc/src/sgml/ref/alter_aggregate.sgml +++ b/doc/src/sgml/ref/alter_aggregate.sgml @@ -179,8 +179,7 @@ ALTER AGGREGATE mypercentile(float8 ORDER BY integer) SET SCHEMA myschema; This will work too: ALTER AGGREGATE mypercentile(float8, integer) SET SCHEMA myschema; - - + diff --git a/doc/src/sgml/ref/alter_publication.sgml b/doc/src/sgml/ref/alter_publication.sgml index cc5d92f041..f064ec5f32 100644 --- a/doc/src/sgml/ref/alter_publication.sgml +++ b/doc/src/sgml/ref/alter_publication.sgml @@ -140,8 +140,7 @@ ALTER PUBLICATION noinsert SET (publish = 'update, delete'); Add some tables to the publication: ALTER PUBLICATION mypublication ADD TABLE users, departments; - - + diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml index bead99622e..b1b7765d76 100644 --- a/doc/src/sgml/ref/alter_subscription.sgml +++ b/doc/src/sgml/ref/alter_subscription.sgml @@ -204,8 +204,7 @@ ALTER SUBSCRIPTION mysub SET PUBLICATION insert_only; Disable (stop) the subscription: ALTER SUBSCRIPTION mysub DISABLE; - - + diff --git a/doc/src/sgml/ref/alter_system.sgml b/doc/src/sgml/ref/alter_system.sgml index c1d27b6564..b234793f3e 100644 --- a/doc/src/sgml/ref/alter_system.sgml +++ b/doc/src/sgml/ref/alter_system.sgml @@ -119,8 +119,7 @@ ALTER SYSTEM SET wal_level = replica; in postgresql.conf: ALTER SYSTEM RESET wal_level; - - + diff --git a/doc/src/sgml/ref/alter_type.sgml b/doc/src/sgml/ref/alter_type.sgml index fdb4f3367d..d65f70f674 100644 --- a/doc/src/sgml/ref/alter_type.sgml +++ b/doc/src/sgml/ref/alter_type.sgml @@ -356,8 +356,7 @@ ALTER TYPE colors ADD VALUE 'orange' AFTER 'red'; To rename an enum value: ALTER TYPE colors RENAME VALUE 'purple' TO 'mauve'; - - + diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml index 0a30e6ea3c..891926dba5 100644 --- a/doc/src/sgml/ref/create_access_method.sgml +++ b/doc/src/sgml/ref/create_access_method.sgml @@ -93,8 +93,7 @@ CREATE ACCESS METHOD name handler function heptree_handler: CREATE ACCESS METHOD heptree TYPE INDEX HANDLER heptree_handler; - - + diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml index 48be476374..c5299dd74e 100644 --- a/doc/src/sgml/ref/create_publication.sgml +++ b/doc/src/sgml/ref/create_publication.sgml @@ -191,8 +191,7 @@ CREATE PUBLICATION alltables FOR ALL TABLES; CREATE PUBLICATION insert_only FOR TABLE mydata WITH (publish = 'insert'); - - + diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml index dd5eda043a..77bf87681b 100644 --- a/doc/src/sgml/ref/create_subscription.sgml +++ b/doc/src/sgml/ref/create_subscription.sgml @@ -265,8 +265,7 @@ CREATE SUBSCRIPTION mysub CONNECTION 'host=192.168.1.50 port=5432 user=foo dbname=foodb' PUBLICATION insert_only WITH (enabled = false); - - + diff --git a/doc/src/sgml/ref/drop_aggregate.sgml b/doc/src/sgml/ref/drop_aggregate.sgml index 631b578df7..dde1ea2444 100644 --- a/doc/src/sgml/ref/drop_aggregate.sgml +++ b/doc/src/sgml/ref/drop_aggregate.sgml @@ -160,8 +160,7 @@ DROP AGGREGATE myrank(VARIADIC "any" ORDER BY VARIADIC "any"); To remove multiple aggregate functions in one command: DROP AGGREGATE myavg(integer), myavg(bigint); - - + diff --git a/doc/src/sgml/ref/drop_policy.sgml b/doc/src/sgml/ref/drop_policy.sgml index 69c87c0ade..f474692105 100644 --- a/doc/src/sgml/ref/drop_policy.sgml +++ b/doc/src/sgml/ref/drop_policy.sgml @@ -96,8 +96,7 @@ DROP POLICY [ IF EXISTS ] name ON < DROP POLICY p1 ON my_table; - - + diff --git a/doc/src/sgml/ref/drop_publication.sgml b/doc/src/sgml/ref/drop_publication.sgml index 1a1be579ad..517d142251 100644 --- a/doc/src/sgml/ref/drop_publication.sgml +++ b/doc/src/sgml/ref/drop_publication.sgml @@ -82,9 +82,7 @@ DROP PUBLICATION [ IF EXISTS ] name Drop a publication: DROP PUBLICATION mypublication; - - - + diff --git a/doc/src/sgml/ref/drop_statistics.sgml b/doc/src/sgml/ref/drop_statistics.sgml index ef659fca61..37fc402589 100644 --- a/doc/src/sgml/ref/drop_statistics.sgml +++ b/doc/src/sgml/ref/drop_statistics.sgml @@ -73,9 +73,7 @@ DROP STATISTICS [ IF EXISTS ] name DROP STATISTICS IF EXISTS accounting.users_uid_creation, public.grants_user_role; - - - + diff --git a/doc/src/sgml/ref/drop_subscription.sgml b/doc/src/sgml/ref/drop_subscription.sgml index 42068d617b..f535c000c4 100644 --- a/doc/src/sgml/ref/drop_subscription.sgml +++ b/doc/src/sgml/ref/drop_subscription.sgml @@ -102,9 +102,7 @@ DROP SUBSCRIPTION [ IF EXISTS ] name DROP SUBSCRIPTION mysub; - - - + diff --git a/doc/src/sgml/ref/import_foreign_schema.sgml b/doc/src/sgml/ref/import_foreign_schema.sgml index 331b362db7..b73dee9439 100644 --- a/doc/src/sgml/ref/import_foreign_schema.sgml +++ b/doc/src/sgml/ref/import_foreign_schema.sgml @@ -141,9 +141,7 @@ IMPORT FOREIGN SCHEMA foreign_films IMPORT FOREIGN SCHEMA foreign_films LIMIT TO (actors, directors) FROM SERVER film_server INTO films; - - - + diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml index 95aa77b907..94dad00870 100644 --- a/doc/src/sgml/ref/insert.sgml +++ b/doc/src/sgml/ref/insert.sgml @@ -725,8 +725,7 @@ INSERT INTO distributors (did, dname) VALUES (9, 'Antwerp Design') -- just use a regular unique constraint on "did" INSERT INTO distributors (did, dname) VALUES (10, 'Conrad International') ON CONFLICT (did) WHERE is_active DO NOTHING; - - + -- cgit v1.2.3 From 6c2003f8a1bbc7c192a2e83ec51581c018aa162f Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 14 Jun 2017 11:57:21 -0700 Subject: Don't force-assign transaction id when exporting a snapshot. Previously we required every exported transaction to have an xid assigned. That was used to check that the exporting transaction is still running, which in turn is needed to guarantee that that necessary rows haven't been removed in between exporting and importing the snapshot. The exported xid caused unnecessary problems with logical decoding, because slot creation has to wait for all concurrent xid to finish, which in turn serializes concurrent slot creation. It also prohibited snapshots to be exported on hot-standby replicas. Instead export the virtual transactionid, which avoids the unnecessary serialization and the inability to export snapshots on standbys. This changes the file name of the exported snapshot, but since we never documented what that one means, that seems ok. Author: Petr Jelinek, slightly editorialized by me Reviewed-By: Andres Freund Discussion: https://postgr.es/m/f598b4b8-8cd7-0d54-0939-adda763d8c34@2ndquadrant.com --- doc/src/sgml/ref/set_transaction.sgml | 6 +- src/backend/replication/logical/snapbuild.c | 14 ++-- src/backend/storage/ipc/procarray.c | 11 ++- src/backend/storage/lmgr/predicate.c | 26 +++--- src/backend/utils/time/snapmgr.c | 119 ++++++++++++++++++---------- src/include/storage/predicate.h | 4 +- src/include/storage/procarray.h | 2 +- 7 files changed, 115 insertions(+), 67 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/ref/set_transaction.sgml b/doc/src/sgml/ref/set_transaction.sgml index ca55a5b196..188d2ed92e 100644 --- a/doc/src/sgml/ref/set_transaction.sgml +++ b/doc/src/sgml/ref/set_transaction.sgml @@ -222,8 +222,8 @@ SET SESSION CHARACTERISTICS AS TRANSACTION transa BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ; SELECT pg_export_snapshot(); pg_export_snapshot --------------------- - 000003A1-1 +--------------------- + 00000003-0000001B-1 (1 row) @@ -233,7 +233,7 @@ SELECT pg_export_snapshot(); BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ; -SET TRANSACTION SNAPSHOT '000003A1-1'; +SET TRANSACTION SNAPSHOT '00000003-0000001B-1'; diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 8848f5b4ec..e06aa0992a 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -262,7 +262,7 @@ static bool ExportInProgress = false; static void SnapBuildPurgeCommittedTxn(SnapBuild *builder); /* snapshot building/manipulation/distribution functions */ -static Snapshot SnapBuildBuildSnapshot(SnapBuild *builder, TransactionId xid); +static Snapshot SnapBuildBuildSnapshot(SnapBuild *builder); static void SnapBuildFreeSnapshot(Snapshot snap); @@ -463,7 +463,7 @@ SnapBuildSnapDecRefcount(Snapshot snap) * and ->subxip/subxcnt values. */ static Snapshot -SnapBuildBuildSnapshot(SnapBuild *builder, TransactionId xid) +SnapBuildBuildSnapshot(SnapBuild *builder) { Snapshot snapshot; Size ssize; @@ -562,7 +562,7 @@ SnapBuildInitialSnapshot(SnapBuild *builder) if (TransactionIdIsValid(MyPgXact->xmin)) elog(ERROR, "cannot build an initial slot snapshot when MyPgXact->xmin already is valid"); - snap = SnapBuildBuildSnapshot(builder, GetTopTransactionId()); + snap = SnapBuildBuildSnapshot(builder); /* * We know that snap->xmin is alive, enforced by the logical xmin @@ -679,7 +679,7 @@ SnapBuildGetOrBuildSnapshot(SnapBuild *builder, TransactionId xid) /* only build a new snapshot if we don't have a prebuilt one */ if (builder->snapshot == NULL) { - builder->snapshot = SnapBuildBuildSnapshot(builder, xid); + builder->snapshot = SnapBuildBuildSnapshot(builder); /* increase refcount for the snapshot builder */ SnapBuildSnapIncRefcount(builder->snapshot); } @@ -743,7 +743,7 @@ SnapBuildProcessChange(SnapBuild *builder, TransactionId xid, XLogRecPtr lsn) /* only build a new snapshot if we don't have a prebuilt one */ if (builder->snapshot == NULL) { - builder->snapshot = SnapBuildBuildSnapshot(builder, xid); + builder->snapshot = SnapBuildBuildSnapshot(builder); /* increase refcount for the snapshot builder */ SnapBuildSnapIncRefcount(builder->snapshot); } @@ -1061,7 +1061,7 @@ SnapBuildCommitTxn(SnapBuild *builder, XLogRecPtr lsn, TransactionId xid, if (builder->snapshot) SnapBuildSnapDecRefcount(builder->snapshot); - builder->snapshot = SnapBuildBuildSnapshot(builder, xid); + builder->snapshot = SnapBuildBuildSnapshot(builder); /* we might need to execute invalidations, add snapshot */ if (!ReorderBufferXidHasBaseSnapshot(builder->reorder, xid)) @@ -1831,7 +1831,7 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn) { SnapBuildSnapDecRefcount(builder->snapshot); } - builder->snapshot = SnapBuildBuildSnapshot(builder, InvalidTransactionId); + builder->snapshot = SnapBuildBuildSnapshot(builder); SnapBuildSnapIncRefcount(builder->snapshot); ReorderBufferSetRestartPoint(builder->reorder, lsn); diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index 8a71536791..dfddfc4002 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -1793,14 +1793,15 @@ GetSnapshotData(Snapshot snapshot) * Returns TRUE if successful, FALSE if source xact is no longer running. */ bool -ProcArrayInstallImportedXmin(TransactionId xmin, TransactionId sourcexid) +ProcArrayInstallImportedXmin(TransactionId xmin, + VirtualTransactionId *sourcevxid) { bool result = false; ProcArrayStruct *arrayP = procArray; int index; Assert(TransactionIdIsNormal(xmin)); - if (!TransactionIdIsNormal(sourcexid)) + if (!sourcevxid) return false; /* Get lock so source xact can't end while we're doing this */ @@ -1817,8 +1818,10 @@ ProcArrayInstallImportedXmin(TransactionId xmin, TransactionId sourcexid) if (pgxact->vacuumFlags & PROC_IN_VACUUM) continue; - xid = pgxact->xid; /* fetch just once */ - if (xid != sourcexid) + /* We are only interested in the specific virtual transaction. */ + if (proc->backendId != sourcevxid->backendId) + continue; + if (proc->lxid != sourcevxid->localTransactionId) continue; /* diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c index 27c4af91cb..bce505a3fa 100644 --- a/src/backend/storage/lmgr/predicate.c +++ b/src/backend/storage/lmgr/predicate.c @@ -148,7 +148,7 @@ * predicate lock maintenance * GetSerializableTransactionSnapshot(Snapshot snapshot) * SetSerializableTransactionSnapshot(Snapshot snapshot, - * TransactionId sourcexid) + * VirtualTransactionId *sourcevxid) * RegisterPredicateLockingXid(void) * PredicateLockRelation(Relation relation, Snapshot snapshot) * PredicateLockPage(Relation relation, BlockNumber blkno, @@ -434,7 +434,8 @@ static uint32 predicatelock_hash(const void *key, Size keysize); static void SummarizeOldestCommittedSxact(void); static Snapshot GetSafeSnapshot(Snapshot snapshot); static Snapshot GetSerializableTransactionSnapshotInt(Snapshot snapshot, - TransactionId sourcexid); + VirtualTransactionId *sourcevxid, + int sourcepid); static bool PredicateLockExists(const PREDICATELOCKTARGETTAG *targettag); static bool GetParentPredicateLockTag(const PREDICATELOCKTARGETTAG *tag, PREDICATELOCKTARGETTAG *parent); @@ -1510,7 +1511,7 @@ GetSafeSnapshot(Snapshot origSnapshot) * one passed to it, but we avoid assuming that here. */ snapshot = GetSerializableTransactionSnapshotInt(origSnapshot, - InvalidTransactionId); + NULL, InvalidPid); if (MySerializableXact == InvalidSerializableXact) return snapshot; /* no concurrent r/w xacts; it's safe */ @@ -1643,7 +1644,7 @@ GetSerializableTransactionSnapshot(Snapshot snapshot) return GetSafeSnapshot(snapshot); return GetSerializableTransactionSnapshotInt(snapshot, - InvalidTransactionId); + NULL, InvalidPid); } /* @@ -1658,7 +1659,8 @@ GetSerializableTransactionSnapshot(Snapshot snapshot) */ void SetSerializableTransactionSnapshot(Snapshot snapshot, - TransactionId sourcexid) + VirtualTransactionId *sourcevxid, + int sourcepid) { Assert(IsolationIsSerializable()); @@ -1673,7 +1675,8 @@ SetSerializableTransactionSnapshot(Snapshot snapshot, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("a snapshot-importing transaction must not be READ ONLY DEFERRABLE"))); - (void) GetSerializableTransactionSnapshotInt(snapshot, sourcexid); + (void) GetSerializableTransactionSnapshotInt(snapshot, sourcevxid, + sourcepid); } /* @@ -1687,7 +1690,8 @@ SetSerializableTransactionSnapshot(Snapshot snapshot, */ static Snapshot GetSerializableTransactionSnapshotInt(Snapshot snapshot, - TransactionId sourcexid) + VirtualTransactionId *sourcevxid, + int sourcepid) { PGPROC *proc; VirtualTransactionId vxid; @@ -1741,17 +1745,17 @@ GetSerializableTransactionSnapshotInt(Snapshot snapshot, } while (!sxact); /* Get the snapshot, or check that it's safe to use */ - if (!TransactionIdIsValid(sourcexid)) + if (!sourcevxid) snapshot = GetSnapshotData(snapshot); - else if (!ProcArrayInstallImportedXmin(snapshot->xmin, sourcexid)) + else if (!ProcArrayInstallImportedXmin(snapshot->xmin, sourcevxid)) { ReleasePredXact(sxact); LWLockRelease(SerializableXactHashLock); ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("could not import the requested snapshot"), - errdetail("The source transaction %u is not running anymore.", - sourcexid))); + errdetail("The source process with pid %d is not running anymore.", + sourcepid))); } /* diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index b3d4fe3ae2..2b6fca9241 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -58,6 +58,7 @@ #include "storage/proc.h" #include "storage/procarray.h" #include "storage/sinval.h" +#include "storage/sinvaladt.h" #include "storage/spin.h" #include "utils/builtins.h" #include "utils/memutils.h" @@ -211,11 +212,15 @@ static Snapshot FirstXactSnapshot = NULL; /* Define pathname of exported-snapshot files */ #define SNAPSHOT_EXPORT_DIR "pg_snapshots" -#define XactExportFilePath(path, xid, num, suffix) \ - snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%d%s", \ - xid, num, suffix) -/* Current xact's exported snapshots (a list of Snapshot structs) */ +/* Structure holding info about exported snapshot. */ +typedef struct ExportedSnapshot +{ + char *snapfile; + Snapshot snapshot; +} ExportedSnapshot; + +/* Current xact's exported snapshots (a list of ExportedSnapshot structs) */ static List *exportedSnapshots = NIL; /* Prototypes for local functions */ @@ -558,8 +563,8 @@ SnapshotSetCommandId(CommandId curcid) * in GetTransactionSnapshot. */ static void -SetTransactionSnapshot(Snapshot sourcesnap, TransactionId sourcexid, - PGPROC *sourceproc) +SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid, + int sourcepid, PGPROC *sourceproc) { /* Caller should have checked this already */ Assert(!FirstSnapshotSet); @@ -617,12 +622,12 @@ SetTransactionSnapshot(Snapshot sourcesnap, TransactionId sourcexid, errmsg("could not import the requested snapshot"), errdetail("The source transaction is not running anymore."))); } - else if (!ProcArrayInstallImportedXmin(CurrentSnapshot->xmin, sourcexid)) + else if (!ProcArrayInstallImportedXmin(CurrentSnapshot->xmin, sourcevxid)) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("could not import the requested snapshot"), - errdetail("The source transaction %u is not running anymore.", - sourcexid))); + errdetail("The source process with pid %d is not running anymore.", + sourcepid))); /* * In transaction-snapshot mode, the first snapshot must live until end of @@ -632,7 +637,8 @@ SetTransactionSnapshot(Snapshot sourcesnap, TransactionId sourcexid, if (IsolationUsesXactSnapshot()) { if (IsolationIsSerializable()) - SetSerializableTransactionSnapshot(CurrentSnapshot, sourcexid); + SetSerializableTransactionSnapshot(CurrentSnapshot, sourcevxid, + sourcepid); /* Make a saved copy */ CurrentSnapshot = CopySnapshot(CurrentSnapshot); FirstXactSnapshot = CurrentSnapshot; @@ -1075,33 +1081,29 @@ AtEOXact_Snapshot(bool isCommit, bool resetXmin) */ if (exportedSnapshots != NIL) { - TransactionId myxid = GetTopTransactionId(); - int i; - char buf[MAXPGPATH]; ListCell *lc; /* * Get rid of the files. Unlink failure is only a WARNING because (1) * it's too late to abort the transaction, and (2) leaving a leaked * file around has little real consequence anyway. - */ - for (i = 1; i <= list_length(exportedSnapshots); i++) - { - XactExportFilePath(buf, myxid, i, ""); - if (unlink(buf)) - elog(WARNING, "could not unlink file \"%s\": %m", buf); - } - - /* - * As with the FirstXactSnapshot, we needn't spend any effort on - * cleaning up the per-snapshot data structures, but we do need to - * remove them from RegisteredSnapshots to prevent a warning below. + * + * We also also need to remove the snapshots from RegisteredSnapshots + * to prevent a warning below. + * + * As with the FirstXactSnapshot, we don't need to free resources of + * the snapshot iself as it will go away with the memory context. */ foreach(lc, exportedSnapshots) { - Snapshot snap = (Snapshot) lfirst(lc); + ExportedSnapshot *esnap = (ExportedSnapshot *) lfirst(lc); - pairingheap_remove(&RegisteredSnapshots, &snap->ph_node); + if (unlink(esnap->snapfile)) + elog(WARNING, "could not unlink file \"%s\": %m", + esnap->snapfile); + + pairingheap_remove(&RegisteredSnapshots, + &esnap->snapshot->ph_node); } exportedSnapshots = NIL; @@ -1159,6 +1161,7 @@ ExportSnapshot(Snapshot snapshot) { TransactionId topXid; TransactionId *children; + ExportedSnapshot *esnap; int nchildren; int addTopXid; StringInfoData buf; @@ -1183,9 +1186,9 @@ ExportSnapshot(Snapshot snapshot) */ /* - * This will assign a transaction ID if we do not yet have one. + * Get our transaction ID if there is one, to include in the snapshot. */ - topXid = GetTopTransactionId(); + topXid = GetTopTransactionIdIfAny(); /* * We cannot export a snapshot from a subtransaction because there's no @@ -1204,6 +1207,13 @@ ExportSnapshot(Snapshot snapshot) */ nchildren = xactGetCommittedChildren(&children); + /* + * Generate file path for the snapshot. We start numbering of snapshots + * inside the transaction from 1. + */ + snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d", + MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1); + /* * Copy the snapshot into TopTransactionContext, add it to the * exportedSnapshots list, and mark it pseudo-registered. We do this to @@ -1213,7 +1223,10 @@ ExportSnapshot(Snapshot snapshot) snapshot = CopySnapshot(snapshot); oldcxt = MemoryContextSwitchTo(TopTransactionContext); - exportedSnapshots = lappend(exportedSnapshots, snapshot); + esnap = (ExportedSnapshot *) palloc(sizeof(ExportedSnapshot)); + esnap->snapfile = pstrdup(path); + esnap->snapshot = snapshot; + exportedSnapshots = lappend(exportedSnapshots, esnap); MemoryContextSwitchTo(oldcxt); snapshot->regd_count++; @@ -1226,7 +1239,8 @@ ExportSnapshot(Snapshot snapshot) */ initStringInfo(&buf); - appendStringInfo(&buf, "xid:%u\n", topXid); + appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->backendId, MyProc->lxid); + appendStringInfo(&buf, "pid:%d\n", MyProcPid); appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId); appendStringInfo(&buf, "iso:%d\n", XactIsoLevel); appendStringInfo(&buf, "ro:%d\n", XactReadOnly); @@ -1245,7 +1259,8 @@ ExportSnapshot(Snapshot snapshot) * xmax. (We need not make the same check for subxip[] members, see * snapshot.h.) */ - addTopXid = TransactionIdPrecedes(topXid, snapshot->xmax) ? 1 : 0; + addTopXid = (TransactionIdIsValid(topXid) && + TransactionIdPrecedes(topXid, snapshot->xmax)) ? 1 : 0; appendStringInfo(&buf, "xcnt:%d\n", snapshot->xcnt + addTopXid); for (i = 0; i < snapshot->xcnt; i++) appendStringInfo(&buf, "xip:%u\n", snapshot->xip[i]); @@ -1276,7 +1291,7 @@ ExportSnapshot(Snapshot snapshot) * ensures that no other backend can read an incomplete file * (ImportSnapshot won't allow it because of its valid-characters check). */ - XactExportFilePath(pathtmp, topXid, list_length(exportedSnapshots), ".tmp"); + snprintf(pathtmp, sizeof(pathtmp), "%s.tmp", path); if (!(f = AllocateFile(pathtmp, PG_BINARY_W))) ereport(ERROR, (errcode_for_file_access(), @@ -1298,8 +1313,6 @@ ExportSnapshot(Snapshot snapshot) * Now that we have written everything into a .tmp file, rename the file * to remove the .tmp suffix. */ - XactExportFilePath(path, topXid, list_length(exportedSnapshots), ""); - if (rename(pathtmp, path) < 0) ereport(ERROR, (errcode_for_file_access(), @@ -1384,6 +1397,30 @@ parseXidFromText(const char *prefix, char **s, const char *filename) return val; } +static void +parseVxidFromText(const char *prefix, char **s, const char *filename, + VirtualTransactionId *vxid) +{ + char *ptr = *s; + int prefixlen = strlen(prefix); + + if (strncmp(ptr, prefix, prefixlen) != 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("invalid snapshot data in file \"%s\"", filename))); + ptr += prefixlen; + if (sscanf(ptr, "%d/%u", &vxid->backendId, &vxid->localTransactionId) != 2) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("invalid snapshot data in file \"%s\"", filename))); + ptr = strchr(ptr, '\n'); + if (!ptr) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("invalid snapshot data in file \"%s\"", filename))); + *s = ptr + 1; +} + /* * ImportSnapshot * Import a previously exported snapshot. The argument should be a @@ -1399,7 +1436,8 @@ ImportSnapshot(const char *idstr) char *filebuf; int xcnt; int i; - TransactionId src_xid; + VirtualTransactionId src_vxid; + int src_pid; Oid src_dbid; int src_isolevel; bool src_readonly; @@ -1463,7 +1501,8 @@ ImportSnapshot(const char *idstr) */ memset(&snapshot, 0, sizeof(snapshot)); - src_xid = parseXidFromText("xid:", &filebuf, path); + parseVxidFromText("vxid:", &filebuf, path, &src_vxid); + src_pid = parseIntFromText("pid:", &filebuf, path); /* we abuse parseXidFromText a bit here ... */ src_dbid = parseXidFromText("dbid:", &filebuf, path); src_isolevel = parseIntFromText("iso:", &filebuf, path); @@ -1513,7 +1552,7 @@ ImportSnapshot(const char *idstr) * don't trouble to check the array elements, just the most critical * fields. */ - if (!TransactionIdIsNormal(src_xid) || + if (!VirtualTransactionIdIsValid(src_vxid) || !OidIsValid(src_dbid) || !TransactionIdIsNormal(snapshot.xmin) || !TransactionIdIsNormal(snapshot.xmax)) @@ -1554,7 +1593,7 @@ ImportSnapshot(const char *idstr) errmsg("cannot import a snapshot from a different database"))); /* OK, install the snapshot */ - SetTransactionSnapshot(&snapshot, src_xid, NULL); + SetTransactionSnapshot(&snapshot, &src_vxid, src_pid, NULL); } /* @@ -2141,5 +2180,5 @@ RestoreSnapshot(char *start_address) void RestoreTransactionSnapshot(Snapshot snapshot, void *master_pgproc) { - SetTransactionSnapshot(snapshot, InvalidTransactionId, master_pgproc); + SetTransactionSnapshot(snapshot, NULL, InvalidPid, master_pgproc); } diff --git a/src/include/storage/predicate.h b/src/include/storage/predicate.h index 8f9ea29917..941ba7119e 100644 --- a/src/include/storage/predicate.h +++ b/src/include/storage/predicate.h @@ -14,6 +14,7 @@ #ifndef PREDICATE_H #define PREDICATE_H +#include "storage/lock.h" #include "utils/relcache.h" #include "utils/snapshot.h" @@ -46,7 +47,8 @@ extern bool PageIsPredicateLocked(Relation relation, BlockNumber blkno); /* predicate lock maintenance */ extern Snapshot GetSerializableTransactionSnapshot(Snapshot snapshot); extern void SetSerializableTransactionSnapshot(Snapshot snapshot, - TransactionId sourcexid); + VirtualTransactionId *sourcevxid, + int sourcepid); extern void RegisterPredicateLockingXid(TransactionId xid); extern void PredicateLockRelation(Relation relation, Snapshot snapshot); extern void PredicateLockPage(Relation relation, BlockNumber blkno, Snapshot snapshot); diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h index 22955a79dd..5cf8ff7538 100644 --- a/src/include/storage/procarray.h +++ b/src/include/storage/procarray.h @@ -82,7 +82,7 @@ extern int GetMaxSnapshotSubxidCount(void); extern Snapshot GetSnapshotData(Snapshot snapshot); extern bool ProcArrayInstallImportedXmin(TransactionId xmin, - TransactionId sourcexid); + VirtualTransactionId *sourcevxid); extern bool ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc); extern RunningTransactions GetRunningTransactionData(void); -- cgit v1.2.3 From 6108348c09df33773bed6e0ac762fe47bdcbb9e6 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii Date: Thu, 15 Jun 2017 10:01:39 +0900 Subject: Fix document bug regarding read only transactions. It was explained that read only transactions (not in standby) allow to update sequences. This had been wrong since the commit: 05d8a561ff85db1545f5768fe8d8dc9d99ad2ef7 Discussion: https://www.postgresql.org/message-id/20170614.110826.425627939780392324.t-ishii%40sraoss.co.jp --- doc/src/sgml/high-availability.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/src') diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml index ce67d4a5e3..72eb073621 100644 --- a/doc/src/sgml/high-availability.sgml +++ b/doc/src/sgml/high-availability.sgml @@ -1824,7 +1824,7 @@ if (!triggered) In normal operation, read-only transactions are allowed to - update sequences and to use LISTEN, UNLISTEN, and + use LISTEN, UNLISTEN, and NOTIFY, so Hot Standby sessions operate under slightly tighter restrictions than ordinary read-only sessions. It is possible that some of these restrictions might be loosened in a future release. -- cgit v1.2.3 From e800656d9a9b40b2f55afabe76354ab6d93353b3 Mon Sep 17 00:00:00 2001 From: Tatsuo Ishii Date: Thu, 15 Jun 2017 11:24:47 +0900 Subject: Fix typo in PostgreSQL 10.0 release note. Patch by Yugo Nagata. --- doc/src/sgml/release-10.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/src') diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml index 75c48c0f59..1918149333 100644 --- a/doc/src/sgml/release-10.sgml +++ b/doc/src/sgml/release-10.sgml @@ -536,7 +536,7 @@ - Specifically, a new CREATE + Specifically, a new CREATE INDEX option allows auto-summarizion of the previous BRIN page range when a new page range is created. -- cgit v1.2.3