| Age | Commit message (Collapse) | Author |
|---|
|
Accept expected output changes which were missed because we've an alternate
expected output file which is not merged automatically. The changes simply
include an additional Project node in the explain node.
|
|
Various things are done to ensure that the test case passes. Some of these
changes are not truly correct because we might not be testing what the original
test case was supposed to test, but given the limitations that we've imposed on
partition table column positions etc, this is inevitable
- accept some output changes because certain features (such as triggers) not
supported.
- make changes to column ordering so that partitions can be attached
- avoid dropping/adding columns which will change column ordering
- accept additional information displayed by \d+ command
|
|
|
|
This is the merge-base of PostgreSQL's master branch and REL_10_STABLE branch.
This should be the last merge from PG's master branch into XL 10 branch.
Subsequent merges must happen from REL_10_STABLE branch
|
|
|
|
Merge wait_slot_xmins() into get_slot_xmins(). At this point the only
place that wasn't doing a wait was the initial-state test, and a wait
there seems pretty harmless.
Michael Paquier
Discussion: https://postgr.es/m/CAB7nPqSp_SLQb2uU7am+sn4V3g1UKv8j3yZU385oAG1cG_BN9Q@mail.gmail.com
|
|
Perusal of the code coverage report shows that the existing regression
test cases for LIMIT/OFFSET don't exercise the nodeLimit code paths
involving backwards scan, empty results, or null values of LIMIT/OFFSET.
Improve the coverage.
|
|
Perusal of the code coverage report shows that the existing regression
test cases for INTERSECT and EXCEPT seemingly all prefer the SETOP_HASHED
implementation. Add some test cases in which we force use of the
SETOP_SORTED mode.
|
|
This affects mostly code comments, some documentation, and tests.
Official APIs already used "standby".
|
|
The previous message didn't mention the name of the table or the
bounds. Put the table name in the primary error message and the
bounds in the detail message.
Amit Langote, changed slightly by me. Suggestions on the exac
phrasing from Tom Lane, David G. Johnston, and Dean Rasheed.
Discussion: http://postgr.es/m/CA+Tgmoae6bpwVa-1BMaVcwvCCeOoJ5B9Q9-RHWo-1gJxfPBZ5Q@mail.gmail.com
|
|
find_composite_type_dependencies correctly found columns that are of
the specified type, and columns that are of arrays of that type, but
not columns that are domains or ranges over the given type, its array
type, etc. The most general way to handle this seems to be to assume
that any type that is directly dependent on the specified type can be
treated as a container type, and processed recursively (allowing us
to handle nested cases such as ranges over domains over arrays ...).
Since a type's array type already has such a dependency, we can drop
the existing special case for the array type.
The very similar logic in get_rels_with_domain was likewise a few
bricks shy of a load, as it supposed that a directly dependent type
could *only* be a sub-domain. This is already wrong for ranges over
domains, and it'll someday be wrong for arrays over domains.
Add test cases illustrating the problems, and back-patch to all
supported branches.
Discussion: https://postgr.es/m/[email protected]
|
|
In commit 5c77690f6, we added polling in front of most of the
get_slot_xmins calls in 001_stream_rep.pl, but today's results from
buildfarm member nightjar show that at least one more poll loop
is needed.
Proactively add a poll loop before the next-to-last get_slot_xmins call
as well. It may be that there is no race condition there because the
standby_2 server is shut down at that point, but I'm quite tired of
fighting with this test script. The empirical evidence that it's safe,
from the buildfarm, is no stronger than the evidence for the other
call that nightjar just proved unsafe.
The only remaining get_slot_xmins calls without wait_slot_xmins
protection are the first two, which should be OK since nothing has
happened at that point. It's tempting to ignore that special case
and merge get_slot_xmins and wait_slot_xmins into a single function.
I didn't go that far though.
Discussion: https://postgr.es/m/[email protected]
|
|
When adding the top-level remote subquery, the code used query_pathkeys,
but that seems to be incorrect as those may be group_pathkeys, as set
by standard_qp_callback().
Consider this query from xc_groupby tests:
select count(*) from xc_groupby_def where a is not null
group by a order by 1;
planned like this
QUERY PLAN
------------------------------------------------------------
Remote Subquery Scan on all
Output: count(*), a
Sort Key: xc_groupby_def.a
-> Sort
Output: (count(*)), a
Sort Key: (count(*))
-> HashAggregate
Output: count(*), a
Group Key: xc_groupby_def.a
-> Seq Scan on public.xc_groupby_def
Output: a, b
Filter: (xc_groupby_def.a IS NOT NULL)
(12 rows)
That's clearly incorrect, because the final sort key should be count(*)
and not xc_groupby_def.a (which is, in fact the group key).
For some reason this did not cause issues on XL 9.5, but apparently the
upper-planner pathification changed the code in a way that affected the
top-level remote subquery.
To fix this, simply use sort_pathkeys instead of query_pathkeys. That
fixes the plans, and also identifies a number of additional plans in
regression tests that were in fact incorrect (but no one noticed).
Several plans stopped producing results with stable ordering, so fix that
by adding an explicit ORDER BY clause.
|
|
- Accept expected output difference because FirstNormalTransactionId's status
is reported as 'committed' in XL. This happens because the oldestXmin is
advanced lazily in XL and hence clog truncation happens lazily too.
- Accept error message because of lack of SAVEPOINT support. But we added a new
test case to test the functionality
|
|
We now also do checks during CREATE TABLE. Also amend alter_table test case so
that a few tables are distributed using round robin method so that the new
checks/limitations don't come in their way. Also new test cases added to ensure
that the other checks for inheritance are exercised too.
|
|
This would lead to failures if local and remote tables have a different
column order. The tests previously didn't catch that because they only
tested the initial data copy. So add another test that exercises the
apply worker.
Author: Petr Jelinek <[email protected]>
|
|
lo_put() surely should require UPDATE permission, the same as lowrite(),
but it failed to check for that, as reported by Chapman Flack. Oversight
in commit c50b7c09d; backpatch to 9.4 where that was introduced.
Tom Lane and Michael Paquier
Security: CVE-2017-7548
|
|
Commit 3eefc51053f250837c3115c12f8119d16881a2d7 claimed to make
pg_user_mappings enforce the qualifications user_mapping_options had
been enforcing, but its removal of a longstanding restriction left them
distinct when the current user is the subject of a mapping yet has no
server privileges. user_mapping_options emits no rows for such a
mapping, but pg_user_mappings includes full umoptions. Change
pg_user_mappings to show null for umoptions. Back-patch to 9.2, like
the above commit.
Reviewed by Tom Lane. Reported by Jeff Janes.
Security: CVE-2017-7547
|
|
Some authentication methods allowed it, others did not. In the client-side,
libpq does not even try to authenticate with an empty password, which makes
using empty passwords hazardous: an administrator might think that an
account with an empty password cannot be used to log in, because psql
doesn't allow it, and not realize that a different client would in fact
allow it. To clear that confusion and to be be consistent, disallow empty
passwords in all authentication methods.
All the authentication methods that used plaintext authentication over the
wire, except for BSD authentication, already checked that the password
received from the user was not empty. To avoid forgetting it in the future
again, move the check to the recv_password_packet function. That only
forbids using an empty password with plaintext authentication, however.
MD5 and SCRAM need a different fix:
* In stable branches, check that the MD5 hash stored for the user does not
not correspond to an empty string. This adds some overhead to MD5
authentication, because the server needs to compute an extra MD5 hash, but
it is not noticeable in practice.
* In HEAD, modify CREATE and ALTER ROLE to clear the password if an empty
string, or a password hash that corresponds to an empty string, is
specified. The user-visible behavior is the same as in the stable branches,
the user cannot log in, but it seems better to stop the empty password from
entering the system in the first place. Secondly, it is fairly expensive to
check that a SCRAM hash doesn't correspond to an empty string, because
computing a SCRAM hash is much more expensive than an MD5 hash by design,
so better avoid doing that on every authentication.
We could clear the password on CREATE/ALTER ROLE also in stable branches,
but we would still need to check at authentication time, because even if we
prevent empty passwords from being stored in pg_authid, there might be
existing ones there already.
Reported by Jeroen van der Ham, Ben de Graaff and Jelte Fennema.
Security: CVE-2017-7546
|
|
If the table being attached had different attribute numbers than the
parent, the old code could incorrectly decide it needed to be scanned.
Amit Langote, reviewed by Ashutosh Bapat
Discussion: http://postgr.es/m/CA+TgmobexgbBr2+Utw-pOMw9uxaBRKRjMW_-mmzKKx9PejPLMg@mail.gmail.com
|
|
If you do ALTER COLUMN SET NOT NULL against an inheritance parent table,
it will recurse to mark all the child columns as NOT NULL as well. This
is necessary for consistency: if the column is labeled NOT NULL then
reading it should never produce nulls.
However, that didn't happen in the case where ALTER ... ADD PRIMARY KEY
marks a target column NOT NULL that wasn't before. That was questionable
from the beginning, and now Tushar Ahuja points out that it can lead to
dump/restore failures in some cases. So let's make that case recurse too.
Although this is meant to fix a bug, it's enough of a behavioral change
that I'm pretty hesitant to back-patch, especially in view of the lack
of similar field complaints. It doesn't seem to be too late to put it
into v10 though.
Michael Paquier, editorialized on slightly by me
Discussion: https://postgr.es/m/[email protected]
|
|
ALTER USER ... SET did not support all the syntax variants of ALTER ROLE
... SET. Fix that, and to avoid further deviations of this kind, unify
many the grammar rules for ROLE/USER/GROUP commands.
Reported-by: Pavel Golub <[email protected]>
|
|
Otherwise, partitioned tables with RETURNING expressions or subject
to a WITH CHECK OPTION do not work properly.
Amit Langote, reviewed by Amit Khandekar and Etsuro Fujita. A few
comment changes by me.
Discussion: http://postgr.es/m/[email protected]
|
|
The buildfarm is still showing at least three distinct behaviors for
a bad locale name in CREATE COLLATION. Although this test was helpful
for getting the error reporting code into some usable shape, it doesn't
seem worth carrying multiple expected-files in order to support the
test in perpetuity. So pull it back out.
Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
|
|
DROP CASCADE doesn't currently promise to visit dependent objects in
a fixed order, so when the regression tests use it, we typically need
to suppress the details of which objects get dropped in order to have
predictable test output. Traditionally we've done that by setting
client_min_messages higher than NOTICE, but there's a better way:
we can "\set VERBOSITY terse" in psql. That suppresses the DETAIL
message with the object list, but we still get the basic notice telling
how many objects were dropped. So at least the test case can verify
that the expected number of objects were dropped.
The VERBOSITY method was already in use in a few places, but run
around and use it wherever it makes sense.
Discussion: https://postgr.es/m/[email protected]
|
|
We were just printing errno, which is certainly not gonna work on
Windows. Now, it's not entirely clear from Microsoft's documentation
whether _create_locale() adheres to standard Windows error reporting
conventions, but let's assume it does and try to map the GetLastError
result to an errno. If this turns out not to work, probably the best
thing to do will be to assume the error is always ENOENT on Windows.
This is a longstanding bug, but given the lack of previous field
complaints, I'm not excited about back-patching it.
Per report from Murtuza Zabuawala.
Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
|
|
Most of our collations code has special handling for the locale names
"C" and "POSIX", allowing those collations to be used whether or not
the system libraries think those locale names are valid, or indeed
whether said libraries even have any locale support. But we missed
handling things that way in CREATE COLLATION. This meant you couldn't
clone the C/POSIX collations, nor explicitly define a new collation
using those locale names, unless the libraries allow it. That's pretty
pointless, as well as being a violation of pg_newlocale_from_collation's
API specification.
The practical effect of this change is quite limited: it allows creating
such collations even on platforms that don't HAVE_LOCALE_T, and it allows
making "POSIX" collation objects on Windows, which before this would only
let you make "C" collation objects. Hence, even though this is a bug fix
IMO, it doesn't seem worth the trouble to back-patch.
In passing, suppress the DROP CASCADE detail messages at the end of the
collation regression test. I'm surprised we've never been bit by
message ordering issues there.
Per report from Murtuza Zabuawala.
Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
|
|
Upstream commit 215b43cdc8d6b4a1700886a39df1ee735cb0274d significantly
reworked planning of leaky functions. In practice that change means we
no longer have to push leaky functions into a subquery. Which greatly
simplifies some plans, including the two in this patch.
This commit accepts the plans only partially, though. It uses the plans
from upstream, and adds a Remote Subquery Scan node at the top, so we
accept the general plan shape change.
But there are a few additional differences that need futher evaluation,
particularly in target lists (Postgres-XL generating more entries than
upstream) and SubPlans (Postgres-XL only generating one subplan, while
upstream generates two alternative ones).
|
|
The plans changed mainly due to abandoning the custom implementation
two-phase aggregation code, and using the upstream parallel aggregation.
That means we have stopped showing schema name in target lists, so
instead of
Output: pg_catalog.avg((avg(xcrem_employee.salary)))
the EXPLAIN now shows
Output: avg(xcrem_employee.salary)
and we also do projection at the scan nodes, so the target list only
shows the necessary subset of columns.
A somewhat surprising change is that the plans switch from distributed
aggregate plans like this one
-> Aggregate
-> Remote Subquery Scan
-> Aggregate
-> Seq Scan
to always performing simple (non-distributed) aggregate like this
-> Aggregate
-> Remote Subquery Scan
-> Seq Scan
This happens due to create_grouping_paths() relying on consider_parallel
flag when setting try_distributed_aggregate, disabling distributed
aggregation when consider_parallel=false. Both affected plans are however
for UPDATE queries, and PostgreSQL disables parallelism for queries that
do writes, so we end up with try_distributed_aggregate=false.
We should probably enable distributed aggregates in these cases, but we
can't ignore consider_parallel entirely, as we likely need some of the
checks. We will probably end up with consider_distributed flag, set in
a similar way to consider_parallel, but that's more an enhancement than
a bug fix.
|
|
Some of the queries in inet test are meant to exercise Index Only Scans.
Postgres-XL was not however picking those plans due to stale stats on
the coordinator (reltuples and relpages in pg_class).
On plain PostgreSQL the tests work fine, as CREATE INDEX also updates
statistics stored in the pg_class catalog. For example this
CREATE TABLE t (a INT);
INSERT INTO t SELECT i FROM generate_series(1,1000) s(i);
SELECT relpages, reltuples FROM pg_class
WHERE relname = 't';
CREATE INDEX ON t(a);
SELECT relpages, reltuples FROM pg_class
WHERE relname = 't';
will show zeroes before the CREATE INDEX command, and accurate values
after it completes.
On Postgres-XL that is not the case, and we will return zeroes even after
the CREATE INDEX command. To actually update the statistics we need to
fetch information from the datanodes the way VACUUM does it.
Fixed by adding an explicit VACUUM call right after the CREATE INDEX, to
fetch the stats from the datanodes and update the coordinator catalogs.
|
|
The test expects the plan to use Index Scan, but with 1000 rows the
differences are very small. With two data nodes, we however compute
the estimates as if the tables had 500 rows, making the cost difference
even smaller.
Fixed by increasing the total number of rows to 2000, which means each
datanode has about 1000 and uses the same cost estimates as upstream.
|
|
The plan changed in two ways. Firstly, the targetlists changed due to
abandoning the custom distributed aggregation and reusing the upstream
partial aggregation code. That means we're not prefixing the aggregate
with schema name, etc.
The plan also switches from distributed aggregation to plain aggregation
with all the work done on top of a remote query. This happens simply due
to costing, as the tables are tiny and two-phase aggregation has some
overhead. The original implementation (as in XL 9.5) distributed the
aggregate unconditionally, ignoring the costing.
Parf of the problem is that the query groups by two columns from two
different tables, resulting in overestimation of the number of groups.
That means the optimizer thinks distributing the aggregation would not
reduce the number of rows, which increases the cost estimate as each
row requires network transfer and the finalize aggregate also depends
on the number of input rows.
We could make the tables larger and the optimizer would eventually
switch to distributed aggregate. For example this seems to do the
trick:
insert into xc_groupby_tab1 select 1, mod(i,1000)
from generate_series(1,20000) s(i);
insert into xc_groupby_tab2 select 1, mod(i,1000)
from generate_series(1,20000) s(i);
But it does not seem worth it, considering it's just a workaround
for the estimation issue and the increased duration. And we already
have other regression tests testing plausible queries benefiting from
distributed aggregation. So just accept the plan change.
|
|
Without this, \dRp prints information that \dRp+ does not, which
seems pretty odd.
Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
|
|
When output of IPC::Run::run () is redirected to scalar references, in
certain circumstances the Msys perl does not correctly detect that the
end of file has been seen, making the test hang indefinitely. One such
circumstance is when the command is 'pg_ctl start', and such a change
was made in commit f13ea95f9e. The workaround, which only applies on
MSys, is to redirect the output to temporary files and then read them in
when the process has finished.
Patch by me, reviewed and tweaked by Tom Lane.
|
|
Most of these differences arise from the the fact that rows are fetched from
multiple datanodes in XL and hence TIDs can be duplicated. There are some other
differences because of EXPLAIN output.
The test case does not yet pass because of other unaddressed failures.
|
|
We don't support different column ordering for partitions in XL. So enforce
that either by ensuring consistent column ordering or accepting errors. Also we
don't support triggers in XL, so some changes were necessary to take that into
account.
|
|
Since 93cbab90b0c6fc3fc4aa515b93057127c0ee8a1b we expect that the child table
has columns at the same position as the parent table. So fix the test case to
follow the rule and resolve expected output differences arising from that.
The test case should pass with this change.
|
|
This module becomes much more useful if we allow it to be used as base
class for external projects. To achieve this, change the exported
get_new_node function into a class method instead, and use the standard
Perl idiom of accepting the class as first argument. This method works
as expected for subclasses. The standalone function is kept for
backwards compatibility, though it could be removed in pg11.
Author: Chap Flackman, based on an earlier patch from Craig Ringer
Discussion: https://postgr.es/m/CAMsr+YF8kO+4+K-_U4PtN==2FndJ+5Bn6A19XHhMiBykEwv0wA@mail.gmail.com
|
|
In various places where we reverse-map a tuple before calling
ExecBuildSlotValueDescription, we neglected to ensure that the
slot descriptor matched the tuple stored in it.
Amit Langote and Amit Khandekar, reviewed by Etsuro Fujita
Discussion: http://postgr.es/m/CAJ3gD9cqpP=WvJj=dv1ONkPWjy8ZuUaOM4_x86i3uQPas=0_jg@mail.gmail.com
|
|
Various cases involving renaming of view columns are handled by having
make_viewdef pass down the view's current relation tupledesc to
get_query_def, which then takes care to use the column names from the
tupledesc for the output column names of the SELECT. For some reason
though, we'd missed teaching make_ruledef to do similarly when it is
printing an ON SELECT rule, even though this is exactly the same case.
The results from pg_get_ruledef would then be different and arguably wrong.
In particular, this breaks pre-v10 versions of pg_dump, which in some
situations would define views by means of emitting a CREATE RULE ... ON
SELECT command. Third-party tools might not be happy either.
In passing, clean up some crufty code in make_viewdef; we'd apparently
modernized the equivalent code in make_ruledef somewhere along the way,
and missed this copy.
Per report from Gilles Darold. Back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]
|
|
These are only placements of Remote FQS or Remote Subplan nodes in the newly
added explain plans in the test case. There are some remaining failures in the
test case which will need more scrutiny.
|
|
I believe this changed as a consequence of commit 54baa4813: trying to
clone the "C" collation now produces a true clone with collencoding -1,
hence the error message if it's duplicate no longer specifies an encoding.
Per buildfarm member crake, which apparently hadn't been running this
test for the last few weeks.
|
|
The order of partitions listed by \d+ is in general locale-dependent.
Rename the partitions in the test added by d363d42bb9 to force them to
be listed in a consistent order.
|
|
Previously, UNBOUNDED meant no lower bound when used in the FROM list,
and no upper bound when used in the TO list, which was OK for
single-column range partitioning, but problematic with multiple
columns. For example, an upper bound of (10.0, UNBOUNDED) would not be
collocated with a lower bound of (10.0, UNBOUNDED), thus making it
difficult or impossible to define contiguous multi-column range
partitions in some cases.
Fix this by using MINVALUE and MAXVALUE instead of UNBOUNDED to
represent a partition column that is unbounded below or above
respectively. This syntax removes any ambiguity, and ensures that if
one partition's lower bound equals another partition's upper bound,
then the partitions are contiguous.
Also drop the constraint prohibiting finite values after an unbounded
column, and just document the fact that any values after MINVALUE or
MAXVALUE are ignored. Previously it was necessary to repeat UNBOUNDED
multiple times, which was needlessly verbose.
Note: Forces a post-PG 10 beta2 initdb.
Report by Amul Sul, original patch by Amit Langote with some
additional hacking by me.
Discussion: https://postgr.es/m/CAAJ_b947mowpLdxL3jo3YLKngRjrq9+Ej4ymduQTfYR+8=YAYQ@mail.gmail.com
|
|
Normally, a JoinExpr would have empty "quals" only if it came from CROSS
JOIN syntax. However, it's possible to get to this state by specifying
NATURAL JOIN between two tables with no common column names, and there
might be other ways too. The code previously printed no ON clause if
"quals" was empty; that's right for CROSS JOIN but syntactically invalid
if it's some type of outer join. Fix by printing ON TRUE in that case.
This got broken by commit 2ffa740be, which stopped using NATURAL JOIN
syntax in ruleutils output due to its brittleness in the face of
column renamings. Back-patch to 9.3 where that commit appeared.
Per report from Tushar Ahuja.
Discussion: https://postgr.es/m/[email protected]
|
|
Just as we already do in ExecConstraints, and for the same reason:
to improve the quality of error messages.
Etsuro Fujita, reviewed by Amit Langote
Discussion: http://postgr.es/m/[email protected]
|
|
Before, we always used a dummy value of 1, but that's not right when
the partitioned table being modified is inside of a WITH clause
rather than part of the main query.
Amit Langote, reported and reviewd by Etsuro Fujita, with a comment
change by me.
Discussion: http://postgr.es/m/[email protected]
|
|
Postgres-XL does not support Foreign Data Wrappers, so accept failures
and output differences in the foreign_data regression test as expected.
|
|
Postgres-XL does not support triggers, naturally causing many failures
in the triggers regression test. So treat the error messages as expected
behavior and remove the NOTICEs produced by the triggers.
Another failure was caused by INSERT in a subquery, which is another
feature unsupported by Postgres-XL.
|
|
Both plans keep the same join algorithm as on PostgreSQL, and the plans
match those produced by Postgres-XL 9.5.
|
