summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2025-04-27Disallow removing placeholders during Self-Join Elimination.Alexander Korotkov
fc069a3a6319 implements Self-Join Elimination (SJE), which can remove base relations when appropriate. However, regressions tests for SJE only cover the case when placeholder variables (PHVs) are evaluated and needed only in a single base rel. If this baserel is removed due to SJE, its clauses, including PHVs, will be transferred to the keeping relation. Removing these PHVs may trigger an error on plan creation -- thanks to the b3ff6c742f6c for detecting that. This commit skips removal of PHVs during SJE. This might also happen that we skip the removal of some PHVs that could be removed. However, the overhead of extra PHVs is small compared to the complexity of analysis needed to remove them. Reported-by: Alexander Lakhin <[email protected]> Author: Alena Rybakina <[email protected]> Author: Andrei Lepikhov <[email protected]> Reviewed-by: Alexander Korotkov <[email protected]> Reviewed-by: Richard Guo <[email protected]>
2025-04-27Remove inappropriate inclusions of c.h and postgres_fe.h.Tom Lane
Per our usual policy, Postgres header files should not include these; the decision as to which one to use is to be made in the calling .c file instead. These errors aren't particularly new, but I'm not feeling a need to back-patch these changes; it's mostly just neatnik-ism.
2025-04-27Don't use double-quotes in #include's of system headers, redux.Tom Lane
This cleans up some loose ends left by commit e8ca9ed1d. I hadn't looked closely enough at these places before, but now I have. The use of double-quoted #includes for Perl headers in plperl_system.h seems to be simply a mistake introduced in 6c944bf3c and faithfully copied forward since then. (I had thought possibly it was required by some weird Windows build setup, but there's no evidence of that in our history.) The occurrences in SectionMemoryManager.h and SectionMemoryManager.cpp evidently stem from those files' origin as LLVM code. It's understandable that LLVM would treat their own files as needing double-quoted #includes; but they're still system headers to us. I also applied the same check to *.c files, and found a few other random incorrect usages in both directions. Our ECPG headers and test files routinely use angle brackets to refer to ECPG headers. I left those usages alone, since it seems reasonable for an ECPG user to regard those headers as system headers.
2025-04-27Remove circular #include's between plpython.h and plpy_util.h.Tom Lane
plpython.h included plpy_util.h, simply on the grounds that "it's easier to just include it everywhere". However, plpy_util.h must include plpython.h, or it won't pass headerscheck. While the resulting circularity doesn't have any immediate bad effect, it's poor design. We have seen serious messes arise in the past from overly-broad inclusion footprints created by such circularities, so let's establish a project policy against it. To fix, just replace *.c files' inclusions of plpython.h with plpy_util.h. They'll pull in plpython.h indirectly; indeed, almost all have already done so via inclusions of other plpy_xxx.h headers. (Any extensions using plpython.h can do likewise without breaking the compatibility of their code with prior Postgres versions.) Reported-by: Bertrand Drouvot <[email protected]> Author: Tom Lane <[email protected]> Reviewed-by: Bertrand Drouvot <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-27Don't use double-quotes in #include's of system headers.Tom Lane
While few if any C compilers will complain about this, it's inconsistent with our other #include's of the same headers. There are some other questionable usages in src/include/jit/SectionMemoryManager.h and src/pl/plperl/plperl_system.h, but perhaps those have a reason to be like that. I can't see that these do. Noticed while fooling around with a script to do analysis of our header cross-inclusions.
2025-04-26Eliminate divide in new fast-path locking codeDavid Rowley
c4d5cb71d2 adjusted the fast-path locking code to allow some configuration of the number of fast-path locking slots via the max_locks_per_transaction GUC. In that commit the FAST_PATH_REL_GROUP() macro used integer division to determine the fast-path locking group slot to use for the lock. The divisor in this case is always a power-of-two value. Here we swap out the divide by a bitwise-AND, which is a significantly faster operation to perform. In passing, adjust the code that's setting FastPathLockGroupsPerBackend so that it's more clear that the value being set is a power-of-two. Also, adjust some comments in the area which contained some magic numbers. It seems better to justify the 1024 upper limit in the location where the #define is made instead of where it is used. Author: David Rowley <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Discussion: https://postgr.es/m/CAApHDvodr3bcnpxcs7+k-3cFwYR0tP-BYhyd2PpDhe-bCx9i=g@mail.gmail.com
2025-04-26Match parameter in new function to earlier equivalentsJohn Naylor
Oversight in commit 3c6e8c123.
2025-04-25aio: Improve debug logging around waiting for IOsAndres Freund
Trying to investigate a bug report by Alexander Lakhin made it apparent that the debug logging around waiting for IO completion is insufficient. Fix that. Discussion: https://postgr.es/m/h4in2db37vepagmi2oz5vvqymjasc5gyb4lpqkunj4eusu274i@37jpd3c2spd3
2025-04-25Fix bug allowing io_combine_limit > io_max_combine_combine limitAndres Freund
10f66468475 intended to limit the value of io_combine_limit to the minimum of io_combine_limit and io_max_combine_limit. To avoid issues with interdependent GUCs, it introduced io_combine_limit_guc and set io_combine_limit in assign hooks. That plan was thwarted by guc_tables.c accidentally still referencing io_combine_limit, instead of io_combine_limit_guc. That lead to the GUC machinery overriding the work done in the assign hooks, potentially leaving io_combine_limit with a too high value. The consequence of this bug was that when running with io_combine_limit > io_combine_limit_guc the AIO machinery would not have reserved large enough iovec and IO data arrays, with one IO's arrays overlapping with another IO's, leading to total confusion. To make such a problem easier to detect in the future, add assertions to pgaio_io_set_handle_data_* checking the length is smaller than io_max_combine_limit (not just PG_IOV_MAX). It'd be nice to have a few tests for this, but it's not entirely obvious how to do so portably. As remarked upon by Tom, the GUC assignment hooks really shouldn't set the underlying variable, that's the job of the GUC machinery. Change that as well. Discussion: https://postgr.es/m/c5jyqnuwrpigd35qe7xdypxsisdjrdba5iw63mhcse4mzjogxo@qdjpv22z763f
2025-04-25aio: Fix crash potential for pg_aios views due to late state updateAndres Freund
pgaio_io_reclaim() reset the fields in PgAioHandle before updating the state to IDLE or incrementing the generation. For most things that's OK, but for pg_get_aios() it is not - if it copied the PgAioHandle while fields were being reset, we wouldn't detect that and could call pgaio_io_get_target_description() with ioh->target == PGAIO_TID_INVALID, leading to a crash. Fix this issue by incrementing the generation and state earlier, before resetting. Also add an assertion to pgaio_io_get_target_description() for the target to be valid - that'd have made this case a bit easier to debug. While at it, add/update a few related assertions. Author: Alexander Lakhin <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-25Fix incorrect format placeholdersPeter Eisentraut
Before commit a0ed19e0a9e there was a cast around these, but the cast inadvertently changed the signedness, but that made the format placeholder correct. Commit a0ed19e0a9e removed the casts, so now the format placeholders had the wrong signedness.
2025-04-25Fix terminology in comment and messagePeter Eisentraut
Should be "bracket" not "brace" for [].
2025-04-25Small code consistency improvementPeter Eisentraut
Adjust the way the increment operators are placed to be consistent throughout the function. Fixup for commit commit c1da7281060.
2025-04-24psql: Fix assertion failures with pipeline modeMichael Paquier
A correct cocktail of COPY FROM, SELECT and/or DML queries and \syncpipeline was able to break the logic in charge of discarding results of a pipeline, done in discardAbortedPipelineResults(). Such sequence make the backend generate a FATAL, due to a protocol synchronization loss. This problem comes down to the fact that we did not consider the case of libpq returning a PGRES_FATAL_ERROR when discarding the results of an aborted pipeline. The discarding code is changed so as this result status is handled as a special case, with the caller of discardAbortedPipelineResults() being responsible for consuming the result. A couple of tests are added to cover the problems reported, bringing an interesting gain in coverage as there were no tests in the tree covering the case of protocol synchronization loss. Issue introduced by 41625ab8ea3d. Reported-by: Alexander Kozhemyakin <[email protected]> Author: Anthonin Bonnefoy <[email protected]> Co-authored-by: Michael Paquier <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-24Add sanity check for dshash entries when reading pgstats fileMichael Paquier
Not having this check would produce a core dump at startup when running pgstat_read_statsfile(), in the case where the information of a stats kind for an entry in the dshash could not be found. The same check already happens for fixed-numbered stats and entries that are stored with their names. This issue can be seen with custom stats kinds. Note that this problem can be reproduced what what is in the core code: - Tweak the test module injection_points to not load the fixed-numbered stats part, leaving only the variable-numbered stats. - Create an instance with injection_points defined in shared_preload_libraries. - Create a pgstats entry by attaching and running a point. - Restart the server without shared_preload_libraries. The startup process detects that something is wrong and reports a WARNING. Author: Bertrand Drouvot <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-23Avoid possibly-theoretical OOM crash hazard in hash_create().Tom Lane
One place in hash_create() used DynaHashAlloc() as a convenient shorthand for MemoryContextAlloc(). That was fine when it was written, but it stopped being fine when 9c911ec06 changed DynaHashAlloc() to use MCXT_ALLOC_NO_OOM (mea culpa). Change the code to call plain MemoryContextAlloc() as intended. I think that this bug may be unreachable in practice, since we now always create AllocSets with some space already allocated, so that an OOM failure here for a non-shared hash table should be impossible (with a hash table name of reasonable length anyway). And there aren't enough shared hash tables to make a crash for one of those probable. Nonetheless it's clearly not operating as designed, so back-patch to v16 where 9c911ec06 came in. Reported-by: Maksim Korotkov <[email protected]> Author: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 16
2025-04-23oauth: Support Python 3.6 in testsJacob Champion
RHEL8 ships a patched 3.6.8 as its base Python version, and I accidentally let some newer Python-isms creep into oauth_server.py during development. Reported-by: Tom Lane <[email protected]> Reviewed-by: Jelte Fennema-Nio <[email protected]> Tested-by: Renan Alves Fonseca <[email protected]> Tested-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/16098.1745079444%40sss.pgh.pa.us
2025-04-23Maintain RelIdToTypeIdCacheHash in TypeCacheOpcCallback()Alexander Korotkov
b85a9d046efd introduced a new RelIdToTypeIdCacheHash, whose entries should exist for typecache entries with TCFLAGS_HAVE_PG_TYPE_DATA flag set or any of TCFLAGS_OPERATOR_FLAGS set or tupDesc set. However, TypeCacheOpcCallback(), which resets TCFLAGS_OPERATOR_FLAGS, was forgotten to update RelIdToTypeIdCacheHash. This commit adds a delete_rel_type_cache_if_needed() call to the TypeCacheOpcCallback() function to maintain RelIdToTypeIdCacheHash after resetting TCFLAGS_OPERATOR_FLAGS. Also, this commit fixes the name of the delete_rel_type_cache_if_needed() function in its mentions in the comments. Reported-by: Noah Misch Discussion: https://postgr.es/m/20250411203241.e9.nmisch%40google.com
2025-04-23Properly prepare varinfos in estimate_multivariate_bucketsize()Alexander Korotkov
To estimate with extended statistics, we need to clear the varnullingrels field in the expression, and duplicates are not allowed in the GroupVarInfo list. We might re-use add_unique_group_var(), but we don't do so for two reasons. 1) We must keep the origin_rinfos list ordered exactly the same way as varinfos. 2) add_unique_group_var() is designed for estimate_num_groups(), where a larger number of groups is worse. While estimating the number of hash buckets, we have the opposite: a lesser number of groups is worse. Therefore, we don't have to remove "known equal" vars: the removed var may valuably contribute to the multivariate statistics to grow the number of groups. This commit adds custom code to estimate_multivariate_bucketsize() to initialize varinfos properly. Reported-by: Robins Tharakan <[email protected]> Discussion: https://postgr.es/m/18885-da51324078588253%40postgresql.org Author: Andrei Lepikhov <[email protected]> Reviewed-by: David Rowley <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Alexander Korotkov <[email protected]>
2025-04-23Change the names generated for child foreign key constraints.Tom Lane
When a foreign key constraint is placed on a partitioned table, we actually make two pg_constraint entries associated with that table. (I have my doubts about the wisdom of that, but it's been like that since v12 and post-feature-freeze is no time to be messing with such entrenched decisions.) The second "child" entry always had a name generated according to the default rule, "table_column(s)_fkey[nnn]", even if the primary entry had an unrelated user-specified name. The trouble with doing that is that the default name could collide with the user-specified name of some other constraint on the same table. While we were willing to adjust the generated name to avoid collisions, that only helps if it's made second; if it's made first then creation of the other constraint would fail, potentially causing dump/reload or pg_upgrade failures. The core of the problem here is that we're infringing on user namespace, so I doubt that there's any 100% solution other than to find a way to not need the "child" entry. In the meantime, it seems like it'd be an improvement to make the child's name be the name of the parent constraint with an underscore and digit(s) appended as necessary to make it unique. This rule can in theory fail in the same way, but it seems much less probable; for one thing, this rule is guaranteed not to match primary entries having auto-generated names. (While an auto-generated primary name isn't user-specified to begin with, it acts like that during dump/reload, so collisions against such names are definitely possible.) An additional bonus, visible in some of the regression test cases that change here, arises from the fact that some error messages cite the child constraint's name not the parent's. In the previous approach the two names could be completely unrelated, leading to user confusion --- the more so since psql's \d command hides child constraints. With this approach it's hopefully much clearer which constraint-the-user-knows-about is failing. However, that does mean that there's user-visible behavior change occurring here, making it seem like not something to back-patch. I feel it's not too late for v18, though. Reported-by: Kirill Reshke <[email protected]> Author: Tom Lane <[email protected]> Reviewed-by: Alvaro Herrera <[email protected]> Discussion: https://postgr.es/m/CALdSSPhGitjpTfzEMJN-Y2x+Q-5QChSxAsmSJ1-E8mQJLkHOqQ@mail.gmail.com
2025-04-23Allocate JsonLexContexts on the heap to avoid warningsDaniel Gustafsson
The stack allocated JsonLexContexts, in combination with codepaths using goto, were causing warnings when compiling with LTO enabled as the optimizer is unable to figure out that is safe. Rather than contort the code with workarounds for this simply heap allocate the structs instead as these are not in any performance critical paths. Author: Daniel Gustafsson <[email protected]> Reported-by: Tom Lane <[email protected]> Reviewed-by: Jacob Champion <[email protected]> Reviewed-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-23psql: Rework TAP routine psql_fails_like() to define WAL sender contextMichael Paquier
The routine was coded so as a WAL sender was always used, state required only for one failure test related to START_REPLICATION. This test is changed so as a WAL sender is used by passing a replication option to psql_fails_like(), instead of forcing the use of a WAL sender for all the tests. This has come up as useful in the context of a separate bug fix where we are looking at extending tests for some failure scenarios. These tests need to happen in the context of a normal backend, and not a WAL sender where the extended query protocol cannot be used. Discussion: https://postgr.es/m/[email protected]
2025-04-23Fix an oversight in 3f28b2fcac.Amit Kapila
Commit 3f28b2fcac tried to ensure that the replication origin shouldn't be advanced in case of an ERROR in the apply worker, so that it can request the same data again after restart. However, it is possible that an ERROR was caught and handled by a (say PL/pgSQL) function, and the apply worker continues to apply further changes, in which case, we shouldn't reset the replication origin. Ensure to reset the origin only when the apply worker exits after an ERROR. Commit 3f28b2fcac added new function geterrlevel, which we removed in HEAD as part of this commit, but kept it in backbranches to avoid breaking any applications. A separate case can be made to have such a function even for HEAD. Reported-by: Shawn McCoy <[email protected]> Author: Hayato Kuroda <[email protected]> Reviewed-by: Masahiko Sawada <[email protected]> Reviewed-by: vignesh C <[email protected]> Reviewed-by: Amit Kapila <[email protected]> Backpatch-through: 16, where it was introduced Discussion: https://postgr.es/m/CALsgZNCGARa2mcYNVTSj9uoPcJo-tPuWUGECReKpNgTpo31_Pw@mail.gmail.com
2025-04-23Remove assertion based on pending_since in pgstat_report_stat()Michael Paquier
This assertion, based on pending_since (timestamp used to prevent stats reports to be too frequent or should a partial flush happen), is reached when it is found that no data can be flushed but a previous call of pgstat_report_stat() determined that some stats data has been found as in need of a flush. So pending_since is set when some stats data is pending (in non-force mode) or if report attempts are too frequent, and reset to 0 once all stats have been flushed. Since 5cbbe70a9cc6, WAL senders have begun to report their stats on a periodic basis for IO stats in v16~ and backend stats on HEAD, creating some friction with the concurrent pgstat_report_stat() calls that can happen in the context of a WAL sender (shutdown callback doing a final report or backend-related code paths). This problem is the cause of spurious failures in the TAP tests. In theory, this assertion can be also reached in v15, even if that's very unlikely. For example, a process, say a background worker, could do periodic and direct stats flushes with concurrent calls of pgstat_report_stat() that could cause conflicting values of pending_since. This can be done with WAL or SLRU stats flushes using pgstat_flush_wal() or pgstat_slru_flush(). HEAD makes this situation easier to happen with custom cumulative stats. This commit removes the assertion altogether, per discussion, as it is more useful to keep the state of things as they are for the WAL sender. The assertion could use a special state based on for example am_walsender, but I doubt that this would be meaningful in the long run based on the other arguments raised while discussing this issue. Reported-by: Tom Lane <[email protected]> Reported-by: Andres Freund <[email protected]> Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/dwrkeszz6czvtkxzr5mqlciy652zau5qqnm3cp5f3p2po74ppk@omg4g3cc6dgq Backpatch-through: 15
2025-04-22Re-enable SSL connect_fails tests, and fix related race conditions.Tom Lane
Cluster.pm's connect_fails routine has long had the ability to sniff the postmaster log file for expected messages after a connection failure. However, that's always had a race condition: on some platforms it's possible for psql to exit and the test script to slurp up the postmaster log before the backend process has been able to write out its final log messages. Back in commit 55828a6b6 we disabled a bunch of tests after discovering that, and the aim of this patch is to re-enable them. (The sibling function connect_ok doesn't seem to have a similar problem, mainly because the messages we look for come out during the authentication handshake, so that if psql reports successful connection they should certainly have been emitted already.) The solution used here is borrowed from 002_connection_limits.pl's connect_fails_wait routine: set the server's log_min_messages setting to DEBUG2 so that the postmaster will log child-process exit, and then wait till we see that log entry before checking for the messages we are actually interested in. If a TAP test uses connect_fails' log_like or log_unlike options, and forgets to set log_min_messages, those connect_fails calls will now hang until timeout. Fixing up the existing callers shows that we had several other TAP tests that were in theory vulnerable to the same problem. It's unclear whether the lack of failures is just luck, or lack of buildfarm coverage, or perhaps there is some obscure timing effect that only manifests in SSL connections. In any case, this change should in principle make those other call sites more robust. I'm not inclined to back-patch though, unless sometime we observe an actual failure in one of them. Reported-by: Andrew Dunstan <[email protected]> Author: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-22Avoid depending on post-UPDATE row order in float4/float8 tests.Tom Lane
While heapam reproduces the insertion order of rows well, updates can move rows to varying places depending on autovacuum activity. In most regression tests we've guarded against getting variable results due to that, but float4.sql and float8.sql had escaped notice so far because they update tables that are too small for autovacuum to pay attention to. With increasing interest in non-heap table AMs, it seems worth allowing for update behaviors that are not like heapam's. Hence, add ORDER BY to stabilize the results in case the updates put the rows in a different order. (We'll continue to assume that a seqscan will reproduce original insertion order, though. Removing that assumption would require vastly-more-invasive test changes.) Author: Pavel Borisov <[email protected]> Reviewed-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/CALT9ZEExHAnBoBVQzQuWPMKUbapF5-FBO3fdeYG3s2tuWQz1NQ@mail.gmail.com
2025-04-22gen_node_support.pl: improve error message for unclosed struct.Tom Lane
This error message was 'runaway "struct_name"', which isn't all that clear; I think 'could not find closing brace for "struct_name"' is better. Also, provide the location of the struct start using the script's usual '$file:$lineno' style. Bug: #18901 Reported-by: Clemens Ruck <[email protected]> Author: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-22Rename injection point for invalidation messages at end of transactionMichael Paquier
This injection point was named "AtEOXact_Inval-with-transInvalInfo", not respecting the implied naming convention that injection points should use lower-case characters, with terms separated by dashes. All the other points defined in the tree follow this style, so let's be more consistent. Author: Hayato Kuroda <[email protected]> Reviewed-by: Aleksander Alekseev <[email protected]> Discussion: https://postgr.es/m/OSCPR01MB14966E14C1378DEE51FB7B7C5F5B32@OSCPR01MB14966.jpnprd01.prod.outlook.com Backpatch-through: 17
2025-04-21Fix INITCAP() word boundaries for PG_UNICODE_FAST.Jeff Davis
Word boundaries are based on whether a character is alphanumeric or not. For the PG_UNICODE_FAST collation, alphanumeric includes non-ASCII digits; whereas for the PG_C_UTF8 collation, it only includes digits 0-9. Pass down the right information from the pg_locale_t into initcap_wbnext to differentiate the behavior. Reported-by: Noah Misch <[email protected]> Reviewed-by: Noah Misch <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-21Use the same cmd_context throughout a walsender's lifetime.Tom Lane
exec_replication_command created a cmd_context to work in and then deleted it on exit. This is pretty dangerous because some replication commands start/finish transactions. In the wake of commit 1afe31f03, that could lead to re-selecting a CurrentMemoryContext that's already been deleted, leading to hilarity such as a memory context that is its own parent. To fix, let's make the cmd_context persist across exec_replication_command calls; instead of deleting it, we'll just reset it each time. In this way it retains the same identity and there's no problem if transaction abort restores it as the working context. It probably even saves a few microseconds to do this. This fix also ensures that exec_replication_command returns to the caller (PostgresMain) with the same context active that had been when it was called (probably MessageContext). The previous coding could get that wrong too. Reported-by: Anthonin Bonnefoy <[email protected]> Author: Anthonin Bonnefoy <[email protected]> Reviewed-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/CAO6_XqoJA7-_G6t7Uqe5nWF3nj+QBGn4F6Ptp=rUGDr0zo+KvA@mail.gmail.com
2025-04-21MemoryContextCreate: assert parent is valid and different from node.Tom Lane
The case of "node == parent" might seem impossible, since we just allocated the new node. But it's possible if parent is a dangling reference to a recently-deleted context. In fact, given aset.c's habit of recycling contexts, it's actually rather likely if that's so. If we'd had this assertion before, it would have simplified debugging a recently-identified walsender issue. Reported-by: Anthonin Bonnefoy <[email protected]> Author: Tom Lane <[email protected]> Discussion: https://postgr.es/m/CAO6_XqoJA7-_G6t7Uqe5nWF3nj+QBGn4F6Ptp=rUGDr0zo+KvA@mail.gmail.com
2025-04-21Fix a few more duplicate words in commentsDavid Rowley
Similar to 84fd3bc14 but these ones were found using a regex that can span multiple lines. Author: David Rowley <[email protected]> Discussion: https://postgr.es/m/CAApHDvrMcr8XD107H3NV=WHgyBcu=sx5+7=WArr-n_cWUqdFXQ@mail.gmail.com
2025-04-20Fix a few duplicate words in commentsDavid Rowley
These are all new to v18 Author: David Rowley <[email protected]> Discussion: https://postgr.es/m/CAApHDvrMcr8XD107H3NV=WHgyBcu=sx5+7=WArr-n_cWUqdFXQ@mail.gmail.com
2025-04-20Comment on need to MarkBufferDirty() if omitting DELAY_CHKPT_START.Noah Misch
Blocking checkpoint phase 2 requires MarkBufferDirty() and BUFFER_LOCK_EXCLUSIVE; neither suffices by itself. transam/README documents this, citing SyncOneBuffer(). Update the DELAY_CHKPT_START documentation to say this. Expand the heap_inplace_update_and_unlock() comment that cites XLogSaveBufferForHint() as precedent, since heap_inplace_update_and_unlock() could have opted not to use DELAY_CHKPT_START. Commit 8e7e672cdaa6bfec85d4d5dd9be84159df23bb41 added DELAY_CHKPT_START to heap_inplace_update_and_unlock(). Since commit bc6bad88572501aecaa2ac5d4bc900ac0fd457d5 reverted it in non-master branches, no back-patch. Discussion: https://postgr.es/m/[email protected]
2025-04-20Test restartpoints in archive recovery.Noah Misch
v14 commit 1f95181b44c843729caaa688f74babe9403b5850 and its v13 equivalent caused timing-dependent failures in archive recovery, at restartpoints. The symptom was "invalid magic number 0000 in log segment X, offset 0", "unexpected pageaddr X in log segment Y, offset 0" [X < Y], or an assertion failure. Commit 3635a0a35aafd3bfa80b7a809bc6e91ccd36606a and predecessors back-patched v15 changes to fix that. This test reproduces the problem probabilistically, typically in less than 1000 iterations of the test. Hence, buildfarm and CI runs would have surfaced enough failures to get attention within a day. Reported-by: Arun Thirupathi <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 13
2025-04-20Avoid ERROR at ON COMMIT DELETE ROWS after relhassubclass=f.Noah Misch
Commit 7102070329d8147246d2791321f9915c3b5abf31 fixed a similar bug, but it missed the case of database-wide ANALYZE ("use_own_xacts" mode). Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 changed consequences from silent discard of a pg_class stats (relpages et al.) update to ERROR "tuple to be updated was already modified". Losing a relpages update of an ON COMMIT DELETE ROWS table was negligible, but a COMMIT-time error isn't negligible. Back-patch to v13 (all supported versions). Reported-by: Richard Guo <[email protected] Reported-by: Robins Tharakan <[email protected]> Discussion: https://postgr.es/m/CAMbWs4-XwMKMKJ_GT=p3_-_=j9rQSEs1FbDFUnW9zHuKPsPNEQ@mail.gmail.com Backpatch-through: 13
2025-04-20Fix issue with ORDER BY / DISTINCT aggregates and FILTERDavid Rowley
1349d2790 added support so that aggregate functions with an ORDER BY or DISTINCT clause could make use of presorted inputs to avoid an implicit sort within nodeAgg.c. That commit failed to consider that a FILTER clause may exist that filters rows before the aggregate function arguments are evaluated. That can be problematic if an aggregate argument contains an expression which could error out during evaluation. It's perfectly valid to want to have a FILTER clause which eliminates such values, and with the pre-sorted path added in 1349d2790, it was possible that the planner would produce a plan with a Sort node above the Aggregate to perform the sort on the aggregate's arguments long before the Aggregate node would filter out the non-matching values. Here we fix this by inspecting ORDER BY / DISTINCT aggregate functions which have a FILTER clause to see if the aggregate's arguments are anything more complex than a Var or a Const. Evaluating these isn't going to cause an error. If we find any non-Var, non-Const parameters then the planner will now opt to perform the sort in the Aggregate node for these aggregates, i.e. disable the presorted aggregate optimization. An alternative fix would have been to completely disallow the presorted optimization for Aggrefs with any FILTER clause, but that wasn't done as that could cause large performance regressions for queries that see significant gains from 1349d2790 due to presorted results coming in from an Index Scan. Backpatch to 16, where 1349d2790 was introduced Author: David Rowley <[email protected]> Reported-by: Kaimeh <[email protected]> Diagnosed-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/CAK-%2BJz9J%3DQ06-M7cDJoPNeYbz5EZDqkjQbJnmRyQyzkbRGsYkA%40mail.gmail.com Backpatch-through: 16
2025-04-19psql: Split extended query protocol meta-commands in --help=commandsMichael Paquier
Compared to v17 with only \bind able to do extended query protocol work, v18 has now a total of 11 meta-commands related to the extended query protocol. These were all listed under the "General" section of the --help=commands output and are specialized, bloating the output generated. All these meta-commands are moved into a new section called "Extended Query Protocol", listed at the end of --help=commands. This split has been suggested by Noah Misch. Discussion: https://postgr.es/m/[email protected]
2025-04-19psql: Improve descriptions of \\flush[request] in --helpMichael Paquier
Noah has reported that the current wording was confusing compared to the description of the underlying libpq routine. The new wording is from me. Reported-by: Noah Misch <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-19psql: Fix incorrect status code returned by \getresultsMichael Paquier
When an invalid number of results is requested for \getresults, the status code returned by exec_command_getresults() was PSQL_CMD_SKIP_LINE and not PSQL_CMD_ERROR. This led to incorrect behaviors, with ON_ERROR_STOP for example. Reported-by: Noah Misch <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-19Fix typos and grammar in the codeMichael Paquier
The large majority of these have been introduced by recent commits done in the v18 development cycle. Author: Alexander Lakhin <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-19Rename injection points used in AIO testsMichael Paquier
The format of the injection point names used by the AIO code does not match the existing naming convention used everywhere else in the code, so let's be consistent. These points are used in test_aio. Reviewed-by: Hayato Kuroda <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-18Make pg_upgrade log message with control file path translatable.Fujii Masao
Commit 173c97812ff replaced the hardcoded "global/pg_control" in pg_upgrade log message with a string literal concatenation of XLOG_CONTROL_FILE macro. However, this change made the message untranslatable. This commit fixes the issue by using %s with XLOG_CONTROL_FILE instead of that literal concatenation, allowing the message to be translated properly. It also wraps the file path in double quotes for consistency with similar log messages. Author: Kyotaro Horiguchi <[email protected]> Reviewed-by: Masao Fujii <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-18Fixup various older misuses of appendPQExpBufferDavid Rowley
Use appendPQExpBufferStr when there are no parameters and appendPQExpBufferChar when the string length is 1. Unlike 3fae25cbb, which fixed this issue for code that was new to v18, this one fixes up instances which exist in the backbranches. We've historically tried to maintain this standard and if we're going to continue doing that, then we won't be doing that selectively based on when the code was introduced. Now seems like a good time to flush out the existing misuses. Waiting until v19 just prolongs their existence in terms of released versions that the misuses exist in. Author: David Rowley <[email protected]> Discussion: https://postgr.es/m/CAApHDvoARMvPeXTTC0HnpARBHn-WgVstc8XFCyMGOzvgu_1HvQ@mail.gmail.com
2025-04-17Make levels 1-based in pg_log_backend_memory_contexts()David Rowley
Both pg_get_process_memory_contexts() and pg_backend_memory_contexts have 1-based levels, whereas pg_log_backend_memory_contexts() was using 0-based levels. Align these. This results in slightly saner behavior from MemoryContextStatsDetail() in regards to the max_level. Previously it would stop at 1 level before the maximum requested level rather than at that level. Reported-by: Atsushi Torikoshi <[email protected]> Author: Atsushi Torikoshi <[email protected]> Author: David Rowley <[email protected] Reviewed-by: Melih Mutlu <[email protected]> Reviewed-by: Rahila Syed <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-04-17Suppress "may be used uninitialized" warnings from older compilers.Tom Lane
The "children" list won't be used until "got_children" has been set true, but older compilers don't get that; about half a dozen buildfarm animals are warning about this. Issue added by 11ff192b5. While here, improve slightly-shaky grammar in comment. Discussion: https://postgr.es/m/[email protected]
2025-04-17Portability fix: isdigit() must be passed an unsigned char.Tom Lane
Oversight in commit 40b9c2701, per buildfarm member mamba.
2025-04-17Cache typlens of a SQL function's input arguments.Tom Lane
This gets rid of repetitive get_typlen calls in postquel_sub_params, which show up as costing a few percent of the runtime in simple test cases (more with more parameters). In combination with the preceding patches, this gets us most of the way back down to the amount of per-call overhead that functions.c had before commit 0dca5d68d. There are some more things that could be done, but this seems like an okay place to stop for v18.
2025-04-17Make SQLFunctionCache long-lived again.Tom Lane
At this point, the only data structures we allocate directly in fcontext are the SQLFunctionCache struct itself, the ParamListInfo struct, and the execution_state array, all of which are small and perfectly capable of being re-used across executions of the same FmgrInfo. Hence, let's give them the same lifespan as the FmgrInfo. This step gets rid of the separate SQLFunctionLink struct and makes fn_extra point to SQLFunctionCache again. We also get rid of the separate fcontext memory context and allocate these items directly in fn_mcxt. For notational simplicity, SQLFunctionCache still has an fcontext field, but it's just a copy of fn_mcxt. The motivation for this is to allow these structures to live as long as the FmgrInfo and be re-used across calls, restoring the original design without its propensity for memory leaks. This gets rid of some per-call overhead that we added in 0dca5d68d. We also make an effort to re-use the JunkFilter and result slot. Those might need to change if the function definition changes, so we compromise by rebuilding them if the cached plan changes. This also moves the tuplestore into fn_mcxt so that it can be re-used across calls, again undoing a change made in 0dca5d68d.
2025-04-17Split some storage out to separate subcontexts of fcontext.Tom Lane
Put the JunkFilter and its result slot (and thence also some subsidiary data such as the result tupledesc) into a separate subcontext "jfcontext". This doesn't accomplish a lot at this point, because we make a new JunkFilter each time through the SQL function. However, the plan is to make the fcontext long-lived, and that raises the possibility that we'll need a new JunkFilter because the plan for the result-generating query changes. A separate context makes it easy to free the obsoleted data when that happens. Also, instead of always running the sub-executor in fcontext, make a separate context for it if we're doing lazy eval of a SRF, and otherwise just run it inside CurrentMemoryContext.