summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2025-06-24Make query jumbling also squash PARAM_EXTERN paramsÁlvaro Herrera
Commit 62d712ecfd94 made query jumbling squash lists of Consts as a single element, but there's no reason not to treat PARAM_EXTERN parameters the same. For these purposes, these values are indeed constants for any particular execution of a query. In particular, this should make list squashing more useful for applications using extended query protocol, which would use parameters extensively. A complication arises: if a query has both external parameters and squashable lists, then the parameter number used as placeholder for the squashed list might be inconsistent with regards to the parameter numbers used by the query literal. To reduce the surprise factor, all parameters are renumbered starting from 1 in that case. Author: Sami Imseih <[email protected]> Author: Dmitry Dolgov <[email protected]> Reviewed-by: Michael Paquier <[email protected]> Discussion: https://postgr.es/m/CAA5RZ0tRXoPG2y6bMgBCWNDt0Tn=unRerbzYM=oW0syi1=C1OA@mail.gmail.com
2025-06-24Improve jumble squashing through CoerceViaIO and RelabelTypeÁlvaro Herrera
There's no principled reason for query jumbling to only remove the first layer of RelabelType and CoerceViaIO. Change it to see through as many layers as there are.
2025-06-24Test that vacuum removes tuples older than OldestXminMelanie Plageman
If vacuum fails to prune a tuple killed before OldestXmin, it will decide to freeze its xmax and later error out in pre-freeze checks. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This test is coverage for a case fixed in 83c39a1f7f3. The test was originally committed to master in aa607980aee but later reverted in efcbb76efe4 due to test instability. The test requires multiple index passes. In Postgres 17+, vacuum uses a TID store for the dead TIDs that is very space efficient. With the old minimum maintenance_work_mem of 1 MB, it required a large number of dead rows to generate enough dead TIDs to force multiple index vacuuming passes. Once the source code changes were made to allow a minimum maintenance_work_mem value of 64kB, the test could be made much faster and more stable. Author: Melanie Plageman <[email protected]> Reviewed-by: John Naylor <[email protected]> Reviewed-by: Peter Geoghegan <[email protected]> Discussion: https://postgr.es/m/CAAKRu_ZJBkidusDut6i%3DbDCiXzJEp93GC1%2BNFaZt4eqanYF3Kw%40mail.gmail.com Backpatch-through: 17
2025-06-24Fix virtual generated column type checking for ALTER TABLEPeter Eisentraut
Virtual generated columns have some special checks in CheckAttributeType(), mainly to check that domains are not used. But this check was only applied during CREATE TABLE, not during ALTER TABLE. This fixes that. Reported-by: jian he <[email protected]> Discussion: https://www.postgresql.org/message-id/CACJufxE0KHR__-h=zHXbhSNZXMMs4LYo4-dbj8H3YoStYBok1Q@mail.gmail.com
2025-06-24Fix missing comment update in 1462aad2e4.Amit Kapila
Remove the part of comment that says we don't allow toggling two_phase option as that is supported in commit 1462aad2e4. Author: Hayato Kuroda <[email protected]> Author: Amit Kapila <[email protected]> Discussion: https://postgr.es/m/OSCPR01MB1496656725F3951AEE8749EBDF579A@OSCPR01MB14966.jpnprd01.prod.outlook.com
2025-06-24psql: Rename meta-command \close to \close_preparedMichael Paquier
\close has been introduced in d55322b0da60 to be able to close a prepared statement using the extended protocol in psql. Per discussion, the name "close" is ambiguous. At the SQL level, CLOSE is used to close a cursor. At protocol level, the close message can be used to either close a statement or a portal. This patch renames \close to \close_prepared to avoid any ambiguity and make it clear that this is used to close a prepared statement. This new name has been chosen based on the feedback from the author and the reviewers. Author: Anthonin Bonnefoy <[email protected]> Reviewed-by: Peter Eisentraut <[email protected]> Reviewed-by: Jelte Fennema-Nio <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-23Temporarily remove 046_checkpoint_logical_slot.plAlexander Korotkov
This new test was intended to check the handling of the replication slot's restart lsn fixed in ca307d5cec90. However, it also reveals another issue related to logical decoding. This commit temporarily removes this test to keep the buildfarm and CFbot green and avoid distorting others' work. This test will be restored once we investigate and fix the issue. Discussion: https://postgr.es/m/CAAKRu_ZCOzQpEumLFgG_%2Biw3FTa%2BhJ4SRpxzaQBYxxM_ZAzWcA%40mail.gmail.com
2025-06-23Remove excess assert from InvalidatePossiblyObsoleteSlot()Alexander Korotkov
ca307d5cec90 introduced keeping WAL segments by slot's last saved restart LSN. It also added an assertion that the slot's restart LSN never goes backward. However, situations when the restart LSN goes backward have been spotted by buildfarm animals and investigated in the thread. When pg_receivewal starts the replication, it sets the last replayed LSN to the beginning of the segment, which is older than what ReplicationSlotReserveWal() set for the slot. A similar situation can happen to pg_basebackup. When standby reconnects to the primary, it sends the last replayed LSN, which might be older than the last confirmed flush LSN. In both these situations, a concurrent checkpoint may trigger an assert trap. Based on ideas from Vitaly Davydov <[email protected]>, Hayato Kuroda (Fujitsu) <[email protected]>, Vignesh C <[email protected]>, Amit Kapila <[email protected]>. Reported-by: Vignesh C <[email protected]> Reported-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/CALDaNm3s-jpQTe1MshsvQ8GO%3DTLj233JCdkQ7uZ6pwqRVpxAdw%40mail.gmail.com Reviewed-by: Vignesh C <[email protected]> Reviewed-by: Amit Kapila <[email protected]>
2025-06-23Properly fix AVX-512 CRC calculation bugJohn Naylor
The problem that led to the workaround in f83f14881c7 was not in fact a compiler bug, but a failure to zero the upper bits of the vector register containing the initial scalar CRC value. Fix that and revert the workaround. Diagnosed-by: Nathan Bossart <[email protected]> Diagnosed-by: Raghuveer Devulapalli <[email protected]> Tested-by: Andy Fan <[email protected]> Tested-by: Soumyadeep Chakraborty <[email protected]> Reviewed-by: Nathan Bossart <[email protected]> Reviewed-by: Raghuveer Devulapalli <[email protected]> Discussion: https://postgr.es/m/PH8PR11MB82866B07AA6758D12F699C00FB70A@PH8PR11MB8286.namprd11.prod.outlook.com
2025-06-21Doc: improve documentation about width_bucket().Tom Lane
Specify whether the bucket bounds are inclusive or exclusive, and improve some other vague language. Explain the behavior that occurs when the "low" bound is greater than the "high" bound. Make width_bucket_numeric's comment more like that for width_bucket_float8, in particular noting that infinite bounds are rejected (since they became possible in v14). Reported-by: Ben Peachey Higdon <[email protected]> Author: Robert Treat <[email protected]> Co-authored-by: Tom Lane <[email protected]> Reviewed-by: Dean Rasheed <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 13
2025-06-20Remove planner's have_dangerous_phv() join-order restriction.Tom Lane
Commit 85e5e222b, which added (a forerunner of) this logic, argued that Adding the necessary complexity to make this work doesn't seem like it would be repaid in significantly better plans, because in cases where such a PHV exists, there is probably a corresponding join order constraint that would allow a good plan to be found without using the star-schema exception. The flaw in this claim is that there may be other join-order restrictions that prevent us from finding a join order that doesn't involve a "dangerous" PHV. In particular we now recognize that small join_collapse_limit or from_collapse_limit could prevent it. Therefore, let's bite the bullet and make the case work. We don't have to extend the executor's support for nestloop parameters as I thought at the time, because we can instead push the evaluation of the placeholder's expression into the left-hand input of the NestLoop node. So there's not really a lot of downside to this solution, and giving the planner more join-order flexibility should have value beyond just avoiding failure. Having said that, there surely is a nonzero risk of introducing new bugs. Since this failure mode escaped detection for ten years, such cases don't seem common enough to justify a lot of risk. Therefore, let's put this fix into master but leave the back branches alone (for now anyway). Bug: #18953 Reported-by: Alexander Lakhin <[email protected]> Diagnosed-by: Richard Guo <[email protected]> Author: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-20Use SnapshotDirty when checking for conflicting index names.Tom Lane
While choosing an autogenerated name for an index, look for pre-existing relations using a SnapshotDirty snapshot, instead of the previous behavior that considered only committed-good pg_class rows. This allows us to detect and avoid conflicts against indexes that are still being built. It's still possible to fail due to a race condition, but the window is now just the amount of time that it takes DefineIndex to validate all its parameters, call smgrcreate(), and enter the index's pg_class row. Formerly the race window covered the entire time needed to create and fill an index, which could be very long if the table is large. Worse, if the conflicting index creation is part of a larger transaction, it wouldn't be visible till COMMIT. So this isn't a complete solution, but it should greatly ameliorate the problem, and the patch is simple enough to be back-patchable. It might at some point be useful to do the same for pg_constraint entries (cf. ChooseConstraintName, ConstraintNameExists, and related functions). However, in the absence of field complaints, I'll leave that alone for now. The relation-name test should be good enough for index-based constraints, while foreign-key constraints seem to be okay since they require exclusive locks to create. Bug: #18959 Reported-by: Maximilian Chrzan <[email protected]> Author: Tom Lane <[email protected]> Reviewed-by: Dilip Kumar <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 13
2025-06-20pgxs.mk: remove unreachable rule for deleting regress.def.Tom Lane
We never create regress.def, and if we did this code would fail to delete it, because "win" is not the correct PORTNAME for Windows. This thinko seems to have originated in commit 7a6b562fd from 1999, although it got moved around multiple times since then. Author: Christoph Berg <[email protected]> Reviewed-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-19Improve runtime and output of tests for replication slots checkpointing.Alexander Korotkov
The TAP tests that verify logical and physical replication slot behavior during checkpoints (046_checkpoint_logical_slot.pl and 047_checkpoint_physical_slot.pl) inserted two batches of 2 million rows each, generating approximately 520 MB of WAL. On slow machines, or when compiled with '-DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE', this caused the tests to run for 8-9 minutes and occasionally time out, as seen on the buildfarm animal prion. This commit modifies the mentioned tests to utilize the $node->advance_wal() function, thereby reducing runtime. Once we do not use the generated data, the proposed function is a good alternative, which cuts the total wall-clock run time. While here, remove superfluous '\n' characters from several note() calls; these appeared literally in the build-farm logs and looked odd. Also, remove excessive 'shared_preload_libraries' GUC from the config and add a check for 'injection_points' extension availability. Reported-by: Alexander Lakhin <[email protected]> Reported-by: Tom Lane <[email protected]> Author: Alexander Korotkov <[email protected]> Author: Vitaly Davydov <[email protected]> Reviewed-by: Hayato Kuroda <[email protected]> Discussion: https://postgr.es/m/fbc5d94e-6fbd-4a64-85d4-c9e284a58eb2%40gmail.com Backpatch-through: 17
2025-06-19Improve pg_dump/pg_dumpall help synopses and terminologyPeter Eisentraut
Increase consistency of --help and man page synopses between pg_dump and pg_dumpall. These should now be very similar, as pg_dumpall can now also produce non-text dump output. But actually, they had drifted further apart. - Use verb "export" consistently, instead of "dump" or "extract". - Use "SQL script" instead of just "script" or "text file". - Maintain consistent distinction between SQL script and other formats/archives (which is relevant for pg_restore). Reviewed-by: Robert Treat <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/3f71d8a7-095b-4829-9b0b-fce09e9866b3%40eisentraut.org
2025-06-19Improve log messages and docs for slot synchronization.Amit Kapila
Improve the clarity of LOG messages when a failover logical slot synchronization fails, making the reasons more explicit for easier debugging. Update the documentation to outline scenarios where slot synchronization can fail, especially during the initial sync, and emphasize that pg_sync_replication_slot() is primarily intended for testing and debugging purposes. We also discussed improving the functionality of pg_sync_replication_slot() so that it can be used reliably, but we would take up that work for next version after some more discussion and review. Reported-by: Suraj Kharage <[email protected]> Author: shveta malik <[email protected]> Reviewed-by: Zhijie Hou <[email protected]> Reviewed-by: Peter Smith <[email protected]> Reviewed-by: Amit Kapila <[email protected]> Backpatch-through: 17, where it was introduced Discussion: https://postgr.es/m/CAF1DzPWTcg+m+x+oVVB=y4q9=PYYsL_mujVp7uJr-_oUtWNGbA@mail.gmail.com
2025-06-19doc: Mention GIN indexes support parallel builds.Fujii Masao
Commit 8492feb98f6 added support for parallel CREATE INDEX on GIN indexes. However, previously two places in the documentation and two in the source code comments still stated that only B-tree and BRIN indexes support parallel builds. This commit updates those references to correctly include GIN indexes. Author: Fujii Masao <[email protected]> Reviewed-by: Robert Treat <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-18pg_dump: Allow pg_dump to dump the statistics for foreign tables.Fujii Masao
Commit 1fd1bd87101 introduced support for dumping statistics with pg_dump and pg_dumpall, covering tables, materialized views, and indexes. However, it overlooked foreign tables, even though functions like pg_restore_relation_stats() support them. This commit fixes that oversight by allowing pg_dump and pg_dumpall to include statistics for foreign tables. Author: Fujii Masao <[email protected]> Reviewed-by: Corey Huinker <[email protected]> Reviewed-by: Nathan Bossart <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-18Document "relrewrite" at the top of heap_create_with_catalog()Michael Paquier
This parameter has been introduced in 325f2ec5557f, and it was not documented contrary to all the other arguments of heap_create_with_catalog(). Reviewed-by: Yugo Nagata <[email protected]> Reviewed-by: Steven Niu <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-17Fix allocation check to test the right variableDaniel Gustafsson
The memory allocation for cancelConn->be_cancel_key was accidentally checking the be_cancel_key member in the conn object instead of the one in cancelConn. Author: Ranier Vilela <[email protected]> Reviewed-by: Daniel Gustafsson <[email protected]> Discussion: https://postgr.es/m/CAEudQAq4ySDR6dsg9xwurBXwud02hX7XCOZZAcZx-JMn6A06nA@mail.gmail.com
2025-06-17Fix re-distributing previously distributed invalidation messages during ↵Masahiko Sawada
logical decoding. Commit 4909b38af0 introduced logic to distribute invalidation messages from catalog-modifying transactions to all concurrent in-progress transactions. However, since each transaction distributes not only its original invalidation messages but also previously distributed messages to other transactions, this leads to an exponential increase in allocation request size for invalidation messages, ultimately causing memory allocation failure. This commit fixes this issue by tracking distributed invalidation messages separately per decoded transaction and not redistributing these messages to other in-progress transactions. The maximum size of distributed invalidation messages that one transaction can store is limited to MAX_DISTR_INVAL_MSG_PER_TXN (8MB). Once the size of the distributed invalidation messages exceeds this threshold, we invalidate all caches in locations where distributed invalidation messages need to be executed. Back-patch to all supported versions where we introduced the fix by commit 4909b38af0. Note that this commit adds two new fields to ReorderBufferTXN to store the distributed transactions. This change breaks ABI compatibility in back branches, affecting third-party extensions that depend on the size of the ReorderBufferTXN struct, though this scenario seems unlikely. Additionally, it adds a new flag to the txn_flags field of ReorderBufferTXN to indicate distributed invalidation message overflow. This should not affect existing implementations, as it is unlikely that third-party extensions use unused bits in the txn_flags field. Bug: #18938 #18942 Author: vignesh C <[email protected]> Reported-by: Duncan Sands <[email protected]> Reported-by: John Hutchins <[email protected]> Reported-by: Laurence Parry <[email protected]> Reported-by: Max Madden <[email protected]> Reported-by: Braulio Fdo Gonzalez <[email protected]> Reviewed-by: Masahiko Sawada <[email protected]> Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Hayato Kuroda <[email protected]> Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/CAD1FGCT2sYrP_70RTuo56QTizyc+J3wJdtn2gtO3VttQFpdMZg@mail.gmail.com Discussion: https://postgr.es/m/CANO2=B=2BT1hSYCE=nuuTnVTnjidMg0+-FfnRnqM6kd23qoygg@mail.gmail.com Backpatch-through: 13
2025-06-16Fix possible Assert failure in verify_compact_attribute()David Rowley
Sometimes the TupleDesc used in verify_compact_attribute() is shared among backends, and since CompactAttribute.attcacheoff gets updated during tuple deformation, it was possible that another backend would set attcacheoff on a given CompactAttribute in the small window of time from when the attcacheoff from the live CompactAttribute was being set in the 'tmp' CompactAttribute and before the Assert verifying that the live and tmp CompactAttributes matched. Here we adjust the code to make a copy of the live CompactAttribute so that we're not trying to Assert against a shared copy of it. Author: David Rowley <[email protected]> Reported-by: Alexander Lakhin <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-16aio: Add missing memory barrier when waiting for IO handleAndres Freund
Previously there was no memory barrier enforcing correct memory ordering when waiting for a free IO handle. However, in the much more common case of waiting for IO to complete, memory barriers already were present. On strongly ordered architectures like x86 this had no negative consequences, but on some armv8 hardware (observed on Apple hardware), it was possible for the update, in the IO worker, to PgAioHandle->state to become visible before ->distilled_result becoming visible, leading to rather confusing assertion failures. The failures were rare enough that the bug sometimes took days to reproduce when running 027_stream_regress in a loop. Once finally debugged, it was easy enough to come up with a much quicker repro: Trigger a lot of very fast IO by limiting io_combine_limit to 1 and ensure that we always have to wait for a free handle by setting io_max_concurrency to 1. Triggering lots of concurrent seqscans in that setup triggers the issue within seconds. One reason this was hard to debug was that the assertion failure most commonly happened in WaitReadBuffers(), rather than in the AIO subsystem itself. The assertions added in this commit make problems like this easier to understand. Also add a comment to the IO worker explaining that we rely on the lwlock acquisition for correct memory ordering. I think it'd be good to add a tap test that stress tests buffer IO, but that's material for a separate patch. Thanks a lot to Alexander and Konstantin for all the debugging help. Reported-by: Tom Lane <[email protected]> Reported-by: Alexander Lakhin <[email protected]> Investigated-by: Andres Freund <[email protected]> Investigated-by: Alexander Lakhin <[email protected]> Investigated-by: Konstantin Knizhnik <[email protected]> Discussion: https://postgr.es/m/2dkz7azclpeiqcmouamdixyn5xhlzy4rvikxrbovyzvi6rnv5c@pz7o7osv2ahf
2025-06-16libpq-oauth: Add exports.list to .gitignorePeter Eisentraut
2025-06-16Message style improvementsPeter Eisentraut
Some message style improvements in new code, and some small refactorings to make translations easier.
2025-06-16Workaround code generation bug in clangJohn Naylor
At optimization level -O0, builds on recent clang fail to produce the correct CRC32C with our AVX-512 implementation. For now, just disable the runtime check for clang at -O0. When this is fixed upstream and we know the extent of the breakage, we can adjust to be version-specific. Reported-by: Soumyadeep Chakraborty <[email protected]> Reported-by: Andy Fan <[email protected]> Tested-by: Andy Fan <[email protected]> Discussion: https://postgr.es/m/CAE-ML%2B-OV6p9uvCFBcSQjZUEh__y0h-KjN%2BBseyGJHt7u8EP%2Bw%40mail.gmail.com Discussion: https://postgr.es/m/87o6uqd3iv.fsf%40163.com
2025-06-15Sync typedefs.list with the buildfarm.Tom Lane
Our maintenance of typedefs.list has been a little haphazard (and apparently we can't alphabetize worth a darn). Replace the file with the authoritative list from our buildfarm, and run pgindent using that. I also updated the additions/exclusions lists in pgindent where necessary to keep pgindent from messing things up significantly. Notably, now that regex_t and some related names are macros not real typedefs, we have to whitelist them explicitly. The exclusions list has also drifted noticeably, presumably due to changes of system headers on the buildfarm animals that contribute to the list. Unlike in prior years, I've not manually added typedef names that are missing from the buildfarm's list because they are not used to declare any variables or fields. So there are a few places where the typedef declaration itself is formatted worse than before, e.g. typedef enum IoMethod. I could preserve the names that were manually added to the list previously, but I'd really prefer to find a less manual way of dealing with these cases. A quick grep finds about 75 such symbols, most of which have never gotten any special treatment. Per discussion among pgsql-release, doing this now seems appropriate even though we're still a week or two away from making the v18 branch.
2025-06-15psql: Change new \conninfo to use SSL instead of TLSPeter Eisentraut
Commit bba2fbc6238 introduced a new implementation of the \conninfo command in psql. That new code uses the term "TLS" while the rest of PostgreSQL, including the rest of psql, consistently uses "SSL". This is uselessly confusing. This changes the new code to use "SSL" as well. Reviewed-by: Alvaro Herrera <[email protected]> Discussion: https://www.postgresql.org/message-id/[email protected]
2025-06-14Improve comments for TidRangeEvalDavid Rowley
Here we provide a bit more detail on why TidRangeEval() does return false when trss_mintid is greater than trss_maxtid. Reported-by: Junwang Zhao <[email protected]> Author: David Rowley <[email protected]> Reviewed-by: Junwang Zhao <[email protected]> Discussion: https://postgr.es/m/CAEG8a3KUbUUqQgfK5X8Sj-%2BppPtGNTU%2BZiep0Rxr7SLjoR%2BB6w%40mail.gmail.com
2025-06-14psql: Report full protocol version in \conninfo output.Fujii Masao
Commit bba2fbc6238 modified \conninfo to display the protocol version used by the current connection, but it only showed the major version (e.g., 3). This commit updates \conninfo to display the full protocol version (e.g., 3.2). Since support for new version 3.2 was added in v18, and the server supports both 3.0 and 3.2, showing the complete version helps users understand exactly which protocol version the current session is using. Although this is a minor behavior change, it's considered a fix for an oversight in the original patch and is included in v18. Author: Fujii Masao <[email protected]> Reviewed-by: David G. Johnston <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-14Add TAP tests to check replication slot advance during the checkpointAlexander Korotkov
The new tests verify that logical and physical replication slots are still valid after an immediate restart on checkpoint completion when the slot was advanced during the checkpoint. This commit introduces two new injection points to make these tests possible: * checkpoint-before-old-wal-removal - triggered in the checkpointer process just before old WAL segments cleanup; * logical-replication-slot-advance-segment - triggered in LogicalConfirmReceivedLocation() when restart_lsn was changed enough to point to the next WAL segment. Discussion: https://postgr.es/m/flat/1d12d2-67235980-35-19a406a0%4063439497 Author: Vitaly Davydov <[email protected]> Author: Tomas Vondra <[email protected]> Reviewed-by: Alexander Korotkov <[email protected]> Reviewed-by: Amit Kapila <[email protected]> Backpatch-through: 17
2025-06-14Keep WAL segments by slot's last saved restart LSNAlexander Korotkov
The patch fixes the issue with the unexpected removal of old WAL segments after checkpoint, followed by an immediate restart. The issue occurs when a slot is advanced after the start of the checkpoint and before old WAL segments are removed at the end of the checkpoint. The patch introduces a new in-memory state for slots: last_saved_restart_lsn, which is used to calculate the oldest LSN for removing WAL segments. This state is updated every time with the current restart_lsn at the moment when the slot is saved to disk. This fix changes the shared memory layout. It's applied to HEAD only because we don't have to preserve ABI compatibility during the beta stage. Another fix that doesn't affect the ABI is committed to back branches. Discussion: https://postgr.es/m/1d12d2-67235980-35-19a406a0%4063439497 Author: Vitaly Davydov <[email protected]> Author: Alexander Korotkov <[email protected]> Reviewed-by: Amit Kapila <[email protected]>
2025-06-13nbtree: _bt_readnextpage doesn't affect markPos.Peter Geoghegan
_bt_readnextpage expects so->currPos.buf to be InvalidBuffer (and for the position's page to be unlocked) when called. However, it does not expect there to be no pins held on any page. In particular, so->markPos might hold a separate pin, both before and after the call. Fix some comments that seemed to suggest otherwise. Follow-up commit to commit 7c319f54, which made _bt_killitems drop pins it acquired itself.
2025-06-13Comment fixups from 626df47ad9.Jeff Davis
Reported-by: Peter Smith <[email protected]> Discussion: https://postgr.es/m/CAHut+PspbHQmRCBL1c-opoJeTUKUaFFfUQJd2rhDZqwUrWCi7w@mail.gmail.com
2025-06-13psql: Reword help message and docs for WATCH_INTERVALDaniel Gustafsson
Reword the documentation around the default value to make interaction between WATCH_INTERVAL and the \watch command clearer. While there, also remove a stray parenthesis left over from a previous version of the patch. Reported-by: Peter Eisentraut <[email protected]> Reviewed-by: David G. Johnston <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-13psql: Forbid use of COPY and \copy while in a pipelineMichael Paquier
Running COPY within a pipeline can break protocol synchronization in multiple ways. psql is limited in terms of result processing if mixing COPY commands with normal queries while controlling a pipeline with the new meta-commands, as an effect of the following reasons: - In COPY mode, the backend ignores additional Sync messages and will not send a matching ReadyForQuery expected by the frontend. Doing a \syncpipeline just after COPY will leave the frontend waiting for a ReadyForQuery message that won't be sent, leaving psql out-of-sync. - libpq automatically sends a Sync with the Copy message which is not tracked in the command queue, creating an unexpected synchronisation point that psql cannot really know about. While it is possible to track such activity for a \copy, this cannot really be done sanely with plain COPY queries. Backend failures during a COPY would leave the pipeline in an aborted state while the backend would be in a clean state, ready to process commands. At the end, fixing those issues would require modifications in how libpq handles pipeline and COPY. So, rather than implementing workarounds in psql to shortcut the libpq internals (with command queue handling for one), and because meta-commands for pipelines in psql are a new feature with COPY in a pipeline having a limited impact compared to other queries, this commit forbids the use of COPY within a pipeline to avoid possible break of protocol synchronisation within psql. If there is a use-case for COPY support within pipelines in libpq, this could always be added in the future, if necessary. Most of the changes of this commit impacts the tests for psql pipelines, removing the tests related to COPY. Some TAP tests still exist for COPY TO/FROM and \copy to/from, to check that that connections are aborted when this operation is attempted. Reported-by: Nikita Kalinin <[email protected]> Author: Anthonin Bonnefoy <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-12Replace %llu by PRIu64 in AIO io_uring codeMichael Paquier
This is a continuation of 15a79c73111f, cleaning up the AIO io_uring code that has been committed after that while still using %llu. The code changed here is new in v18, so cleaning things now means less conflicts if this area of the code changes on backpatch once the 18 stable branch is created. Reviewed-by: Nathan Bossart <[email protected]> Reviewed-by: Peter Eisentraut <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-12pg_restore: Fix wrong descriptions of --with-{schema,data,statistics} options.Fujii Masao
Commit bde2fb797aa added the --with-schema, --with-data, and --with-statistics options to pg_restore. These options control whether to restore schema, data, or statistics if present in the archive. However, the help message and documentation incorrectly described them as affecting what gets dumped. This commit corrects those descriptions to clarify that the options control restoration, not dumping. Bug: #18952 Reported-by: TAKATSUKA Haruka <[email protected]> Author: Fujii Masao <[email protected]> Reviewed-by: TAKATSUKA Haruka <[email protected]> Reviewed-by: Daniel Gustafsson <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-12Fix squashing algorithm for query textsÁlvaro Herrera
The algorithm to squash lists of constants added by commit 62d712ecfd94 was a bit too simplistic; we wanted to avoid adding unnecessary complexity, but cases like direct function calls of typecasting functions (and others) were missed, and bogus SQL syntax was being shown in pg_stat_statements normalized query text field. To fix normalization for those cases, we need the parser to transmit information about were each list of constant values starts and ends, so add that to a couple of nodes. Also add a few more test cases to make sure we're doing the right thing. The patch initially submitted by Sami added a new private struct in gram.y to carry the start/end information for A_Expr, but I (Álvaro) decided that a better fix was to remove the parser indirection via the in_expr production, and instead create separate components in the a_expr rule. I'm surprised that this works and doesn't require more changes, but I assume (without checking) that the grammar used to be more complex and got simplified at some point. Bump catversion. Author: Sami Imseih <[email protected]> Author: Dmitry Dolgov <[email protected]> Reviewed-by: Michael Paquier <[email protected]> Discussion: https://postgr.es/m/CAA5RZ0tRXoPG2y6bMgBCWNDt0Tn=unRerbzYM=oW0syi1=C1OA@mail.gmail.com
2025-06-12Revert support for improved tracking of nested queriesMichael Paquier
This commit reverts the two following commits: - 499edb09741b, track more precisely query locations for nested statements. - 06450c7b8c70, a follow-up fix of 499edb09741b with query locations. The test introduced in this commit is not reverted. This is proving useful to track a problem that only pgaudit was able to detect. These prove to have issues with the tracking of SELECT statements, when these use multiple parenthesis which is something supported by the grammar. Incorrect location and lengths are causing pg_stat_statements to become confused, failing its job in query normalization with potential out-of-bound writes because the location and the length may not match with what can be handled. A lot of the query patterns discussed when this issue was reported have no test coverage in the main regression test suite, or the recovery test 027_stream_regress.pl would have caught the problems as pg_stat_statements is loaded by the node running the regression tests. A first step would be to improve the test coverage to stress more the query normalization logic. A different portion of this work was done in 45e0ba30fc40, with the addition of tests for nested queries. These can be left in the tree. They are useful to track the way inner queries are currently tracked by PGSS with non-top-level entries, and will be useful when reconsidering in the future the work reverted here. Reported-by: Alexander Kozhemyakin <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-11Revert "nbtree: Remove useless row compare arg."Peter Geoghegan
This reverts commit 54c6ea8c81db718508eeea50991d3c1c5dff54a5. Further analysis has shown that the forcenonrequired row compare behavior is in fact necessary, despite the new restrictions on RowCompares imposed by _bt_set_startikey following commit 5f4d98d4. Discussion: https://postgr.es/m/CAH2-Wzm3bKcz3TbHGem3_+SinEyG=VZVPbApQghp7YiZj+MM3g@mail.gmail.com
2025-06-11Revert a few small patches that were intended for version 19.Jeff Davis
- 4c787a24e7e220a60022e47c1776f22f72902899 - 78bd364ee39ca70a8f9cb8719282389866a08e14 - 7a6880fadc177873d5663961ec3a02d67e34dcbe - 8898082a5d3e94eef073f0e08124137e096e78ef Suggested-by: Robert Haas <[email protected]> Discussion: https://postgr.es/m/CA+TgmoZ=J=PVNZUNKaxULu+KUVSt3Y-aJ1DZ9Y3Co6mu0z62jA@mail.gmail.com Discussion: https://postgr.es/m/[email protected]
2025-06-11Add tab completion for REJECT_LIMIT option.Masahiko Sawada
This addresses an oversight in commit 4ac2a9bec, which introduced the REJECT_LIMIT option to the COPY command. Author: Atsushi Torikoshi <[email protected]> Reviewed-by: Yugo Nagata <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-06-11Make _bt_killitems drop pins it acquired itself.Peter Geoghegan
Teach nbtree's _bt_killitems to leave the so->currPos page that it sets LP_DEAD items on in whatever state it was in when _bt_killitems was called. In particular, make sure that so->dropPin scans don't acquire a pin whose reference is saved in so->currPos.buf. Allowing _bt_killitems to change so->currPos.buf like this is wrong. The immediate consequence of allowing it is that code in _bt_steppage (that copies so->currPos into so->markPos) will behave as if the scan is a !so->dropPin scan. so->markPos will therefore retain the buffer pin indefinitely, even though _bt_killitems only needs to acquire a pin (along with a lock) for long enough to mark known-dead items LP_DEAD. This issue came to light following a report of a failure of an assertion from recent commit e6eed40e. The test case in question involves the use of mark and restore. An initial call to _bt_killitems takes place that leaves so->currPos.buf in a state that is inconsistent with the scan being so->dropPin. A subsequent call to _bt_killitems for the same position (following so->currPos being saved in so->markPos, and then restored as so->currPos) resulted in the failure of an assertion that tests that so->currPos.buf is InvalidBuffer when the scan is so->dropPin (non-assert builds got a "resource was not closed" WARNING instead). The same problem exists on earlier releases, though the issue is far more subtle there. Recent commit e6eed40e introduced the so->dropPin field as a partial replacement for testing so->currPos.buf directly. Earlier releases won't get an assertion failure (or buffer pin leak), but they will allow the second _bt_killitems call from the test case to behave as if a buffer pin was consistently held since the original call to _bt_readpage. This is wrong; there will have been an initial window during which no pin was held on the so->currPos page, and yet the second _bt_killitems call will neglect to check if so->currPos.lsn continues to match the page's now-current LSN. As a result of all this, it's just about possible that _bt_killitems will set the wrong items LP_DEAD (on release branches). This could only happen with merge joins (the sole user of nbtree mark/restore support), when a concurrently inserted index tuple used a recently-recycled TID (and only when the new tuple was inserted onto the same page as a distinct concurrently-removed tuple with the same TID). This is exactly the scenario that _bt_killitems' check of the page's now-current LSN against the LSN stashed in currPos was supposed to prevent. A follow-up commit will make nbtree completely stop conditioning whether or not a position's pin needs to be dropped on whether the 'buf' field is set. All call sites that might need to drop a still-held pin will be taught to rely on the scan-level so->dropPin field recently introduced by commit e6eed40e. That will make bugs of the same general nature as this one impossible (or make them much easier to detect, at least). Author: Peter Geoghegan <[email protected]> Reported-By: Alexander Lakhin <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 13
2025-06-11psql: Remove PARTITION BY clause in tab completion for unlogged tablesMichael Paquier
CREATE UNLOGGED TABLE was still being recommended by psql's tab completion as a possible pattern, but the backend is rejecting this option since e2bab2d79204. Reported-by: Shinya Kato <[email protected]> Reviewed-by: Nathan Bossart <[email protected]> Reviewed-by: Shinya Kato <[email protected]> Discussion: https://postgr.es/m/CAOzEurQZ1a+6d1K8b=+Ww1NFQVwAt9KSCQsBWXYBaPnYCenK3g@mail.gmail.com
2025-06-10Don't reduce output request size on non-Unix-socket connections.Tom Lane
Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <[email protected]> Author: Tom Lane <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 13
2025-06-10inet_net_pton.c: use pg_ascii_tolower() rather than tolower().Jeff Davis
Avoid dependence on setlocale(). No behavior change. Discussion: https://postgr.es/m/[email protected] Reviewed-by: Peter Eisentraut <[email protected]>
2025-06-10copyfromparse.c: use pg_ascii_tolower() rather than tolower().Jeff Davis
Avoid dependence on setlocale(). No behavior change. Discussion: https://postgr.es/m/[email protected] Reviewed-by: Peter Eisentraut <[email protected]>
2025-06-10Use exported symbols list on macOS for loadable modules as wellPeter Eisentraut
On macOS, when building with the make system, the exported symbols list $(SHLIB_EXPORTS) was ignored. This was probably not intentional, it was probably just forgotten, since that combination has never actually been used until now (for libpq-oauth). The meson build system handles this correctly. Also, other platforms have been doing this correctly. This fixes it. It also does a bit of refactoring to make the code match the layout for other platforms. Reviewed-by: Jacob Champion <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/c70ca32e-b109-460d-9810-6e23ebb4473f%40eisentraut.org
2025-06-08pg_restore: fix incompatibility with old directory-format dumps.Tom Lane
pg_restore failed to restore large objects (blobs) out of directory-format dumps made by versions before PG v12. That's because, due to a bug fixed in commit 548e50976, those old versions put the wrong filename into the BLOBS TOC entry. Said bug was harmless before v17, because we ignored the incorrect filename field --- but commit a45c78e32 assumed it would be correct. Reported-by: Pavel Stehule <[email protected]> Author: Pavel Stehule <[email protected]> Reviewed-by: Tom Lane <[email protected]> Discussion: https://postgr.es/m/CAFj8pRCrZ=_e1Rv1N+6vDaH+6gf=9A2mE2J4RvnvKA1bLiXvXA@mail.gmail.com Backpatch-through: 17