Permalink
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also or
learn more about diff comparisons.
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
Learn more about diff comparisons here.
...
- 13 commits
- 29 files changed
- 2 contributors
Commits on Apr 1, 2025
-
Add very basic test for kill_prior_tuples
Previously our tests did not exercise kill_prior_tuples for hash and gist. The latter only because it does not work if all the dead tuples are on one page (or something like that). Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
heapam: Move logic to handle HEAP_MOVED into a helper function
Before we dealt with this in 6 near identical and one very similar copy. The helper function errors out when encountering a HEAP_MOVED_IN/HEAP_MOVED_OUT tuple with xvac considered current or in-progress. It'd be preferrable to do that change separately, but otherwise it'd not be possible to deduplicate the handling in HeapTupleSatisfiesVacuum(). Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
bufmgr: Add BufferLockHeldByMe()
Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
heapam: Use exclusive lock on old page in CLUSTER
To be able to guarantee that we can set the hint bit, acquire an exclusive lock on the old buffer. We need the hint bits to be set as otherwise reform_and_rewrite_tuple() -> rewrite_heap_tuple() -> heap_freeze_tuple() will get confused. It'd be better if we somehow coulda void setting hint bits on the old page. One reason to use VACUUM FULL are very bloated tables - rewriting most of the old table before during VACUUM FULL doesn't. Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
bufmgr: Separate slow/fast path of LockBufHdr
This mainly is beneficial for some upcoming patches that use LockBufHdr() in reasonably common paths. But I do see a very small improvement when scanning a large table fully resident in the kernel page cache. Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
heapam: Add batch mode mvcc check and use it in page mode
There are two reasons for doing so: 1) It is generally faster to perform checks in a batched fashion and making sequential scans faster is nice. 2) We would like to stop setting hint bits while pages are being written out. The necessary locking becomes visible for page mode scans if done for every tuple. With batching the overhead can be amortized to only happen once per page. There are substantial further optimization opportunities along these lines: - Right now HeapTupleSatisfiesMVCCBatch() simply uses the single-tuple HeapTupleSatisfiesMVCC(), relying on the compiler to inline it. We could instead write an explicitly optimized version that avoids repeated xid tests. - Introduce batched version of the serializability test - Introduce batched version of HeapTupleSatisfiesVacuum Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
bufmgr: Add interface to acquire right to set hint bits
At the moment hint bits can be set with just a share lock on a page (and in one place even without any lock). Because of this we need to copy pages while writing them out, as otherwise the checksum could be corrupted. The need to copy the page is problematic for the AIO patchset: 1) Instead of just needing a single buffer for a copied page we need one for each page that's potentially undergoing IO 2) To be able to use the "worker" AIO implementation the copied page needs to reside in shared memory. Even without AIO copying the page isn't free... This commit starts to address that by adding BufferPrepareToSetHintBits(), which needs to be called before setting hint bits on a buffer. BufferPrepareToSetHintBits() only allows hint bit writes if there's no ongoing IO and while hint bits are being set no IO is allowed to be started. To know that a buffer is undergoing IO a new BufferDesc state flag is used, BM_SETTING_HINTS. Theoretically it'd be possible to reuse BM_IO_IN_PROGRESS, but that'd make it harder to debug the system. The new interface is not yet used, that will happen in subsequent commits, to make review a bit easier. Therefore we cannot yet rely on not needing a copy of the buffer during IO. Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
heapam: Acquire right to set hint bits
This commit starts to use BufferPrepareToSetHintBits(), introduced in a previous commit, for the reasons explained in said commit. To amortize the cost of BufferPrepareToSetHintBits() for cases where hint bits are set at a high frequency, HeapTupleSatisfiesMVCCBatch() uses the new SetHintBitsExt() which defers BufferFinishSetHintBits() until all hint bits on a page have been set. It's likely worth introducing additional batch visibility routines, e.g. for vacuuming, but I did not find a regression with the state as of this commit. So that's left for later. Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
Acquire right to set hint bits in the remaining places
Use BufferPrepareToSetHintBits() in the remaining places that modify buffers without an exclusive lock. The remaining places are indexes with support for kill_prior_tuples and the freespace code. After this we do not need to copy buffers to write them out anymore. That change is done separately however. Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
bufmgr: Don't copy pages while writing out
After the series of preceding commits introducing and using BufferPrepareToSetHintBits() hint bits cannot be set while IO is going on. Therefore we do not need to copy pages while they are being written out anymore. Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
bufmgr: Detect some missing BufferPrepareToSetHintBits() calls
Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
-
WIP: bufmgr: Detect some bad buffer accesses
I wrote this mainly to ensure that I did not miss converting any hint bit sets to BufferPrepareToSetHintBits(), but it seems like it might be more generally useful. If we do want to include it, it needs a bit more polishing. On my workstation, the performance effect of this test infrastructure is as follows: base: real 1m4.613s user 4m31.409s sys 3m20.445s ENFORCE_BUFFER_PROT real 1m11.912s user 4m27.332s sys 3m28.063s ENFORCE_BUFFER_PROT + ENFORCE_BUFFER_PROT_READ real 1m33.455s user 4m32.188s sys 3m41.275s Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch:
Loading
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff cf/5483~1...cf/5483