Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: postgresql-cfbot/postgresql
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: cf/5599~1
Choose a base ref
...
head repository: postgresql-cfbot/postgresql
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: cf/5599
Choose a head ref
  • 3 commits
  • 6 files changed
  • 2 contributors

Commits on Feb 26, 2025

  1. Get rid of O(N^2) script-parsing overhead in pgbench. 

    pgbench wants to record the starting line number of each command
in its scripts.  It was computing that by scanning from the script
start and counting newlines, so that O(N^2) work had to be done
for an N-command script.  In a script with 50K lines, this adds
up to about 10 seconds on my machine.

To add insult to injury, the results were subtly wrong, because
expr_scanner_offset() scanned to find the NUL that flex inserts
at the end of the current token --- and before the first yylex
call, no such NUL has been inserted.  So we ended by computing the
script's last line number not its first one.  This was visible only
in case of \gset at the start of a script, which perhaps accounts
for the lack of complaints.

To fix, steal an idea from plpgsql and track the current lexer
ending position and line count as we advance through the script.
(It's a bit simpler than plpgsql since we can't need to back up.)
Also adjust a couple of other places that were invoking scans
from script start when they didn't really need to.  I made a new
psqlscan function psql_scan_get_location() that replaces both
expr_scanner_offset() and expr_scanner_get_lineno(), since in
practice expr_scanner_get_lineno() was only being invoked to find
the line number of the current lexer end position.

Reported-by: Daniel Vérité <[email protected]>
Author: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
    @tglsfdc
    tglsfdc authored and Commitfest Bot committed Feb 26, 2025
    Configuration menu
    Copy the full SHA
    5b633fe View commit details
    Browse the repository at this point in the history

  2. Avoid unnecessary computation of pgbench's script line number. 

    ParseScript only needs the lineno for meta-commands, so let's not
bother computing it otherwise.  While this doesn't save much given
the previous patch, there's no point in doing unnecessary work.
While we're at it, avoid calling psql_scan_get_location() twice for
a meta-command.

One reason for making this change is that the line number computed
in ParseScript's main loop was actually wrong in most cases: it
would point just past the semicolon of the previous SQL command,
not at what the user thinks the current command's line number is.
We could add some code to skip whitespace before capturing the line
number, but it would be pretty pointless at present.  Just move the
call to avoid the temptation to rely on that value.  (Once we've
lexed the backslash, the line number will be right.)

This change also means that pgbench never inquires about the
location before it's lexed something, so that the care taken in
the previous patch to behave sanely in that case is unnecessary.
It seems best to keep that logic, though, as future callers
might depend on it.

Author: Daniel Vérité <[email protected]>
Discussion: https://postgr.es/m/[email protected]
    @tglsfdc
    tglsfdc authored and Commitfest Bot committed Feb 26, 2025
    Configuration menu
    Copy the full SHA
    1ecd8b3 View commit details
    Browse the repository at this point in the history

  3. [CF 5599] v2 - Avoid O(N^2) script parsing cost in pgbench 

    This branch was automatically generated by a robot using patches from an
email thread registered at:

https://commitfest.postgresql.org/patch/5599

The branch will be overwritten each time a new patch version is posted to
the thread, and also periodically to check for bitrot caused by changes
on the master branch.

Patch(es): https://www.postgresql.org/message-id/[email protected]
Author(s): Tom Lane
    Commitfest Bot committed Feb 26, 2025
    Configuration menu
    Copy the full SHA
    6839309 View commit details
    Browse the repository at this point in the history
Loading