CYBERTEC PostgreSQL Logo

btree_gist improvements in PostgreSQL 18

08.2025
Category: 
Tags: 

Motivation

The extension btree_gist lives in PostgreSQL contrib and implements btree-like operators and behavior on top of the GiST framework. GiST stands for Generalized Search Tree and is a framework to implement advanced indexes for many use cases. Fulltext indexing or spatial index types are an example for such specialized index implementations.

btree_gist can serve as an example on how to implement such specialized indexes, but also serves very important use cases nowadays in PostgreSQL:

  1. Nearest-Neighbour-Search
  2. Exclusion Constraints

Especially the latter is a very important use case, since this helps implement advanced constraints in PostgreSQL databases. For example, if you want to combine standard scalar types like bigint or uuid with a range type, you need btree_gist to be able to create the constraint, because otherwise the necessary combination of those types is not able to be used together with a GiST index, which is required in this case. Though there was one important problem until now:

  • To build GiST indexes, two methods are available: buffered and sorted.
  • sorted is used if operator classes for such a GiST index implement sortsupport
  • If not, buffered is the selected method

The problem with buffered is that this follows a one-by-one method: Each value is inserted individually when the index is built, which is considerably slower than using a sorted input. Since each insertion into the building index results in an individual lookup, this is a very slow procedure and also results in a suboptimal organized index afterwards. In contrast, sorted input makes it easier to build an index and keep an optimal structure. Including PostgreSQL 17, btree_gist operator classes didn't provide sortsupport, thus supporting the slow buffered method only.

The patch

With the upcoming PostgreSQL 18 release, btree_gist was extended and now uses sortsupport per default when building such indexes. The patch itself has its history though. It started first with the work by Andrey Borodin in 2020. Unfortunately this was reverted due to some unresolved problems with the handling of specific datatypes. Christoph Heiss and me then picked up this idea again, and after some extensive review finally all issues are solved.

Some benchmarks

The following benchmark is based on a dataset provided by a customer, who is fighting with int4range range values and their time required to rebuild the indexes when importing the data. The simplified schema for this benchmark is defined as below:

We are reusing this table layout for all the other benchmarks below. The sample data looks like the following:

I did this benchmark with just a slightly tuned instance of PostgreSQL 17.5 and 18beta2, the adjusted settings are:

  • shared_buffers = 4GB
  • maintenance_work_mem = 1GB
  • max_wal_size = 6GB
  • effective_cache_size is already set to 4GB per default, so this setting is untouched

The performance improvements are shown in the following graph, which compares the CREATE INDEX timings for 1, 5 and 10 million rows to index with PostgreSQL 17.5 and PostgreSQL 18beta2.

The timings for the specific sizes of the test dataset are significantly lower on PostgreSQL 18 with sortsupport when compared to PostgreSQL 17. So this promises much faster timings when building indexes using btree_gist in PostgreSQL 18.

The next benchmark illustrates the performance improvement we get using btree_gist behind the scenes when creating an exclusion constraint. For that, we create an exclusion constraint on the test table above that ensures that an UUID doesn't have any overlapping integer values within its stored ranges. We can achieve this by using the && overlaps operator, supported by int4range. The constraint will be added with the following DDL statement:

We benchmark this statement again for 1 million, 5 million and 10 million tuples from the test dataset above. The resulting runtimes draw the following graph:

Again, the improvements in PG18 with sortsupport are impressive. Especially with workloads that need to import and index large amount of data and want to use btree_gist because of the previously outlined features are benefitting.

The numbers above shows the improvement when creating indexes with btree_gist index access methods, but what about the general performance when these indexes get utilized in queries? During his work on the patch, Christoph Heiss created a python script to measure the performance of btree_gist indexes compared with and without sortsupport. While the patch primarily is targeted to decrease the required time to build such indexes, they should also affect the query performance, since with sortsupport this should result in physically better organized index structure.

To test if querying the new index yields increases throughput because of probably better index quality, we've done the following benchmark utilizing pgbench and its scripting feature. Based on the test dataset with 10 million tokens we have the following initialization script:

This creates a new table test_dataset after dropping it (if it already exists). Then the table is filled with 10000 random keys selected from the token table with 10 million rows. The pgbench script using this structure looks as follows:

We then run the benchmark three times and calculate the average transactions per seconds measured with these runs. Each run is 60 seconds, we measure exactly just one connection and also the latency during the benchmark. pgbench.tokens is the file with the pgbench script shown above.

On my machine, this yields the following figure (higher numbers are better):

The throughput by just selecting tokens that are overlapping with the && operator shows a clear performance improvement on PostgreSQL 18beta2. This means that by having a better index quality in this case yields three times higher transaction throughput per second than the same workload in PostgreSQL 17.

Christoph Heiss created a benchmark tool to track down why these lookup queries are much faster than before. This python script can be found in the PostgreSQL archives here. It performs the same workload as the random access benchmark above with pgbench, but examines the runtime statistics of each query with EXPLAIN. This allows to gather some runtime metrics like the number of pages the query needs to access. The benchmark follows the same pattern as the pgbench script above: First, a number of tokens (default 10000) are selected randomly from the tokens table, then the selected token is read from the table again. This is done within a loop, which is configurable and repeated 10 times per default. This sums up to 100000 queries issued. For each query, the execution plan is analyzed and execution time, shared page hits, shared page reads and I/O time are collected.

When successfully executed, metrics are stored in a name file in CSV format and a summary is printed to console. I ran the benchmark against current PostgreSQL 17.5 and 18beta2 with the tokens table filled with 10 million rows again. Repeating the queries multiple times causes the index to be fully cached. Looking at the average page hits within the shared buffer pool when comparing both versions, we get the following graph:

This benchmark causes PostgreSQL 17 with this table and btree_gist index to read ten more pages average than compared with PostgreSQL 18beta2. The benchmark was run several times, but the figures stay stable. This backups our hope that sortsupport yields a much better index quality and also improves performance for such index lookups.

Conclusion

The new sortsupport for the btree_gist extension in the upcoming PostgreSQL 18 release provides a much better performance when compared to former PostgreSQL major versions. While just being an extension, btree_gist serves important use cases like having specialized composite indexes with data types that don't support GiST, for example to implement exclusion constraints on tables.

Users that rely on these features should try out PostgreSQL 18 as soon as possible.

Leave a Reply

Your email address will not be published. Required fields are marked *

CYBERTEC Logo white
Get the newest PostgreSQL Info & Tools


    This site is protected by reCAPTCHA and the Google Privacy Policy & Terms of Service apply.

    ©
    2025
    CYBERTEC PostgreSQL International GmbH
    phone-handsetmagnifiercrosscross-circle
    linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram