Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: postgresql-cfbot/postgresql
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: cf/4337~1
Choose a base ref
...
head repository: postgresql-cfbot/postgresql
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: cf/4337
Choose a head ref
  • 12 commits
  • 40 files changed
  • 3 contributors

Commits on Jul 16, 2024

  1. Add a syntax to create Incrementally Maintainable Materialized Views 

    Allow to create Incrementally Maintainable Materialized View (IMMV)
by using INCREMENTAL option in CREATE MATERIALIZED VIEW command
as follow:

     CREATE [INCREMANTAL] MATERIALIZED VIEW xxxxx AS SELECT ....;
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    1b53f92 View commit details
    Browse the repository at this point in the history

  2. Add relisivm column to pg_class system catalog 

    If this boolean column is true, a relations is Incrementally Maintainable
Materialized View (IMMV). This is set when IMMV is created.

Also, isimmv columns is added to pg_matviews system view.

isimmv
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    46794f9 View commit details
    Browse the repository at this point in the history

  3. Allow to prolong life span of transition tables until transaction end 

    Originally, tuplestores of AFTER trigger's transition tables were
freed for each query depth. For our IVM implementation, we would like
to prolong life of the tuplestores because we have to preserve them
for a whole query assuming that some base tables might be changed
in some trigger functions.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    37b547a View commit details
    Browse the repository at this point in the history

  4. Add Incremental View Maintenance support to pg_dump 

    Support CREATE INCREMENTAL MATERIALIZED VIEW syntax.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    ef54610 View commit details
    Browse the repository at this point in the history

  5. Add Incremental View Maintenance support to psql 

    Add tab completion and meta-command output for IVM.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    d7617d5 View commit details
    Browse the repository at this point in the history

  6. Add Incremental View Maintenance support 

    In this implementation, AFTER triggers are used to collect
tuplestores containing transition table contents. When multiple tables
are changed, multiple AFTER triggers are invoked, then the final AFTER
trigger performs actual update of the matview. In addition, BEFORE
triggers are also used to handle global information for view
maintenance.

To calculate view deltas, we need both pre-state and post-state of base
tables. Post-update states are available in AFTER trigger, and pre-update
states can be calculated by removing inserted tuples and appending deleted
tuples. Insterted tuples are filtered using the snapshot taken before
table modiication, and deleted tuples are contained in the old transition
table.

Incrementally Maintainable Materialized Views (IMMV) can contain
duplicated tuples.

This patch also allows self-join, simultaneous updates of more than
one base table, and multiple updates of the same base table.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    79cd064 View commit details
    Browse the repository at this point in the history

  7. Add DISTINCT support for IVM 

    When IMMV is created with DISTINCT, multiplicity of tuples is
counted and stored in  "__ivm_count__" column, which is a hidden
column of IMMV.  The value in __ivm_count__ is updated when IMMV
is maintained incrementally. A tuple in IMMV can be removed if
and only if the count becomes zero.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    0934908 View commit details
    Browse the repository at this point in the history

  8. Add aggregates support in IVM 

    count, sum, adn avg are supported.

As a restriction, expressions specified in GROUP BY must appear in
the target list because tuples to be updated in IMMV are identified
by using this group key. However, in the case of aggregates without
GROUP BY, there is only one tuple in the view, so keys are not uses
to identify tuples.

When creating a IMMV, in addition to __ivm_count column, some hidden
columns for each aggregate are added to the target list. For example,
names of these hidden columns are ivm_count_avg and ivm_sum_avg for
the average function, and so on.

When a base table is modified, the aggregated values and related
hidden columns are also updated as well as __ivm_count__. The
way of update depends the kind of aggregate function.　Specifically,
sum and count are updated by simply adding or subtracting delta value
calculated from delta tables. avg is updated by using values of sum
and count stored in views as hidden columns and deltas calculated
from delta tables.

About aggregate functions except "count()" (sum and avg), NULLs in input
values are ignored, and the result of aggegate should be NULL when no
rows are selected.  To support this specification, the numbers of non-NULL
input values are counted and stored in hidden columns. In the case of
count(), count(x) returns zero when no rows are selected, but count(*)
doesn't ignore NULL input.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    d1604fd View commit details
    Browse the repository at this point in the history

  9. Add support for min/max aggregates for IVM 

    Supporting min and max is more complicated than count, sum, or avg.

For an example of min, when tuples are inserted, the current min value
in the view and the min value in the inseteted tuples are compared,
then the smaller one is used as the latest min value. On the other
hand, when tuples are deleted, if the current min value in the view
equals to the min in the deleted tuples, we need re-computation the
latest min value from base tables. Otherwise, the current value in
the view remains.
    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    9d3d792 View commit details
    Browse the repository at this point in the history

  10. Add regression tests for Incremental View Maintenance

    @thoshiai
    thoshiai authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    6581086 View commit details
    Browse the repository at this point in the history

  11. Add documentations about Incremental View Maintenance

    @yugo-n
    yugo-n authored and Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    39ef95f View commit details
    Browse the repository at this point in the history

  12. [CF 48/4337] v34 - Incremental View Maintenance (IVM), take 2 

    This commit was automatically generated by a robot at cfbot.cputube.org.
It is based on patches submitted to the PostgreSQL mailing lists and
registered in the PostgreSQL Commitfest application.

This branch will be overwritten each time a new patch version is posted to
the email thread, and also periodically to check for bitrot caused by changes
on the master branch.

Commitfest entry: https://commitfest.postgresql.org/48/4337
Patch(es): https://www.postgresql.org/message-id/[email protected]
Author(s): Yugo Nagata
    Commitfest Bot committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    8a4d02a View commit details
    Browse the repository at this point in the history
Loading