summaryrefslogtreecommitdiff
path: root/src/bin/pg_rewind/t/RewindTest.pm
AgeCommit message (Collapse)Author
2023-02-23pg_rewind: Fix determining TLI when server was just promoted.Heikki Linnakangas
If the source server was just promoted, and it hasn't written the checkpoint record yet, pg_rewind considered the server to be still on the old timeline. Because of that, it would claim incorrectly that no rewind is required. Fix that by looking at minRecoveryPointTLI in the control file in addition to the ThisTimeLineID on the checkpoint. This has been a known issue since forever, and we had worked around it in the regression tests by issuing a checkpoint after each promotion, before running pg_rewind. But that was always quite hacky, so better to fix this properly. This doesn't add any new tests for this, but removes the previously-added workarounds from the existing tests, so that they should occasionally hit this codepath again. This is arguably a bug fix, but don't backpatch because we haven't really treated it as a bug so far. Also, the patch didn't apply cleanly to v13 and below. I'm sure sure it could be made to work on v13, but doesn't seem worth the risk and effort. Reviewed-by: Kyotaro Horiguchi, Ibrar Ahmed, Aleksander Alekseev Discussion: https://www.postgresql.org/message-id/9f568c97-87fe-a716-bd39-65299b8a60f4%40iki.fi
2023-01-02Update copyright for 2023Bruce Momjian
Backpatch-through: 11
2022-05-12Pre-beta mechanical code beautification.Tom Lane
Run pgindent, pgperltidy, and reformat-dat-files. I manually fixed a couple of comments that pgindent uglified.
2022-04-06Add option --config-file to pg_rewindMichael Paquier
This option is useful to do a rewind with the server configuration file (aka postgresql.conf) located outside the data directory, which is something that some Linux distributions and some HA tools like to rely on. As a result, this can simplify the logic around a rewind by avoiding the copy of such files before running pg_rewind. This option affects pg_rewind when it internally starts the target cluster with some "postgres" commands, adding -c config_file=FILE to the command strings generated, when: - retrieving a restore_command using a "postgres -C" command for -c/--restore-target-wal. - forcing crash recovery once to get the cluster into a clean shutdown state. Author: Gunnar "Nick" Bluth Reviewed-by: Michael Banck, Alexander Kukushkin, Michael Paquier, Alexander Alekseev Discussion: https://postgr.es/m/[email protected]
2022-03-27Remove more unused module imports from TAP testsDaniel Gustafsson
This is a follow-up to commit 7dac61402 which removed a set of unused modules from the TAP test. The Config references in the pg_ctl and pg_rewind tests were removed in commit 1c6d46293. Fcntl ':mode' and File::stat in the pg_ctl test were added in c37b3d08c which was probably a leftover from an earlier version of the patch, as the function using these was added to another module in that commit. The Config reference in the ldap test was added in ee56c3b21 which in turn use $^O instead of interrogating Config. Author: Dagfinn Ilmari Mannsåker <[email protected]> Discussion: https://postgr.es/m/[email protected]
2022-02-23Use test functions in pg_rewind test moduleDaniel Gustafsson
Commit 61081e75c introduced pg_rewind along with the test suite, which ensured that subroutines didn't incur more than one test to plan. Now that we no longer explicitly plan tests (since 549ec201d), we can use the usual Test::More functions. Reviewed-by: Andrew Dunstan <[email protected]> Discussion: https://postgr.es/m/[email protected]
2022-02-20Remove most msys special processing in TAP testsAndrew Dunstan
Following migration of Windows buildfarm members running TAP tests to use of ucrt64 perl for those tests, special processing for msys perl is no longer necessary and so is removed. Backpatch to release 10 Discussion: https://postgr.es/m/[email protected]
2022-01-08Update copyright for 2022Bruce Momjian
Backpatch-through: 10
2021-10-24Move Perl test modules to a better namespaceAndrew Dunstan
The five modules in our TAP test framework all had names in the top level namespace. This is unwise because, even though we're not exporting them to CPAN, the names can leak, for example if they are exported by the RPM build process. We therefore move the modules to the PostgreSQL::Test namespace. In the process PostgresNode is renamed to Cluster, and TestLib is renamed to Utils. PostgresVersion becomes simply PostgreSQL::Version, to avoid possible confusion about what it's the version of. Discussion: https://postgr.es/m/[email protected] Reviewed by Erik Rijkers and Michael Paquier
2021-07-29Unify PostgresNode's new() and get_new_node() methodsAndrew Dunstan
There is only one constructor now for PostgresNode, with the idiomatic name 'new'. The method is not exported by the class, and must be called as "PostgresNode->new('name',[args])". All the TAP tests that use PostgresNode are modified accordingly. Third party scripts will need adjusting, which is a fairly mechanical process (I just used a sed script).
2021-05-12Initial pgindent and pgperltidy run for v14.Tom Lane
Also "make reformat-dat-files". The only change worthy of note is that pgindent messed up the formatting of launcher.c's struct LogicalRepWorkerId, which led me to notice that that struct wasn't used at all anymore, so I just took it out.
2021-05-07Add a copyright notice to perl files lacking one.Andrew Dunstan
2020-07-20Rename wal_keep_segments to wal_keep_size.Fujii Masao
max_slot_wal_keep_size that was added in v13 and wal_keep_segments are the GUC parameters to specify how much WAL files to retain for the standby servers. While max_slot_wal_keep_size accepts the number of bytes of WAL files, wal_keep_segments accepts the number of WAL files. This difference of setting units between those similar parameters could be confusing to users. To alleviate this situation, this commit renames wal_keep_segments to wal_keep_size, and make users specify the WAL size in it instead of the number of WAL files. There was also the idea to rename max_slot_wal_keep_size to max_slot_wal_keep_segments, in the discussion. But we have been moving away from measuring in segments, for example, checkpoint_segments was replaced by max_wal_size. So we concluded to rename wal_keep_segments to wal_keep_size. Back-patch to v13 where max_slot_wal_keep_size was added. Author: Fujii Masao Reviewed-by: Álvaro Herrera, Kyotaro Horiguchi, David Steele Discussion: https://postgr.es/m/[email protected]
2020-07-09Tighten up Windows CRLF conversion in our TAP test scripts.Tom Lane
The previous approach was to search-and-destroy all \r occurrences no matter what. That seems more likely to hide bugs than anything else; indeed it seems to be hiding one now. Fix things so that we only transform \r\n to \n. Side effects: must do this before, not after, chomp'ing if we're going to chomp, else we'd fail to clean up a trailing \r\n. Also, remove safe_psql's redundant repetition of what psql already did; else it might reduce \r\r\n to \n, which is exactly the scenario I'm hoping to expose. Perhaps this should be back-patched, but for now I'm content to see what happens in HEAD. Discussion: https://postgr.es/m/[email protected]
2020-07-08tap tests: replace 'master' with 'primary'.Andres Freund
We've largely replaced master with primary in docs etc, but tap test still widely used master. Author: Andres Freund Reviewed-By: David Steele Discussion: https://postgr.es/m/[email protected]
2020-04-01Add -c/--restore-target-wal to pg_rewindMichael Paquier
pg_rewind needs to copy from the source cluster to the target cluster a set of relation blocks changed from the previous checkpoint where WAL forked up to the end of WAL on the target. Building this list of relation blocks requires a range of WAL segments that may not be present anymore on the target's pg_wal, causing pg_rewind to fail. It is possible to work around this issue by copying manually the WAL segments needed but this may lead to some extra and actually useless work. This commit introduces a new option allowing pg_rewind to use a restore_command while doing the rewind by grabbing the parameter value of restore_command from the target cluster configuration. This allows the rewind operation to be more reliable, so as only the WAL segments needed by the rewind are restored from the archives. In order to be able to do that, a new routine is added to src/common/ to allow frontend tools to restore files from archives using an already-built restore command. This version is more simple than the backend equivalent as there is no need to handle the non-recovery case. Author: Alexey Kondratov Reviewed-by: Andrey Borodin, Andres Freund, Alvaro Herrera, Alexander Korotkov, Michael Paquier Discussion: https://postgr.es/m/[email protected]
2019-10-07Improve handling and coverage of --no-ensure-shutdown in pg_rewindMichael Paquier
This includes a couple of changes around the new behavior of pg_rewind which enforces recovery to happen once on a cluster not shut down cleanly: - Some comments and documentation improvements. - Shutdown the cluster to rewind with immediate mode in all the tests, this allows to check after the forced recovery behavior which is wanted as new default. - Use -F for the forced recovery step, so as postgres does not use fsync. This was useless as a final sync is done once the tool is done. Author: Michael Paquier Reviewed-by: Alexey Kondratov Discussion: https://postgr.es/m/[email protected]
2019-09-30pg_rewind: test new --write-recovery-conf functionalityAlvaro Herrera
Author: Alexey Kondratov Reviewed-by: Paul Guo Discussion: https://postgr.es/m/[email protected]
2019-06-08Update stale comments, and fix comment typos.Noah Misch
2019-05-22Initial pgperltidy run for v12.Tom Lane
Make all the perl code look nice, too (for some value of "nice").
2019-04-14Switch TAP tests of pg_rewind to use non-superuser role, take twoMichael Paquier
Up to now the tests of pg_rewind have been using a superuser for all its tests (which is the default of many tests actually, and something that ought to be reviewed) when involving an online source server, still it is possible to use a non-superuser role to do that as long as this role is granted permissions to execute all the source-side functions used for the rewind. This is possible since v11, and was already documented as of bfc8068. PostgresNode::init is extended so as callers of this routine can add extra options to configure the authentication of a new node, which gets used by this commit, and allows the tests to work properly on Windows where SSPI is used. This will allow to catch up easily any change in pg_rewind if the tool begins to use more backend-side functions, so as the properties introduced by v11 are kept. Per suggestion from Peter Eisentraut. Author: Michael Paquier Reviewed-by: Magnus Hagander Discussion: https://postgr.es/m/[email protected]
2019-04-13Revert "Switch TAP tests of pg_rewind to use a role with minimal permissions"Michael Paquier
This reverts commit d4e2a84, which added a new user with limited permissions to run the TAP tests of pg_rewind. Buildfarm machine members on Windows jacana and bowerbird have been complaining about that, the new role not being able to run the rewind because SSPI is not configured to allow it. Fixing the test requires passing down directly the new user to pg_regress with --create-role so as SSPI can work properly. Reported-by: Andrew Dunstan Discussion: https://postgr.es/m/[email protected]
2019-04-12Switch TAP tests of pg_rewind to use a role with minimal permissionsMichael Paquier
Up to now the tests of pg_rewind have been using a superuser for all the tests (which is the default of many tests actually, and something that ought to be reviewed) when involving an online source server, still it is possible to use a non-superuser role to do that as long as this role is granted permissions to execute all the source-side functions used for the rewind. This is possible since v11, and was already documented as of bfc8068. This will allow to catch up easily any change in pg_rewind if the tool begins to use more backend-side functions, so as the properties introduced by v11 are kept. Per suggestion from Peter Eisentraut. Author: Michael Paquier Reviewed-by: Magnus Hagander Discussion: https://postgr.es/m/[email protected]
2019-03-18Fix pg_rewind when rewinding new database with tables includedMichael Paquier
This fixes an issue introduced by 266b6ac, which has added filters to exclude file patterns on the target and source data directories to reduce the number of files transferred. Filters get applied to both the target and source data files, and include pg_internal.init which is present for each database once relations are created on it. However, if the target differed from the source with at least one new database with relations, the rewind would fail due to the exclusion filters applied on the target files, causing pg_internal.init to still be present on the target database folder, while its contents should have been completely removed so as there is nothing remaining inside at the time of the folder deletion. Applying exclusion filters on the source files is fine, because this way the amount of data copied from the source to the target is reduced. And actually, not applying the filters on the target is what pg_rewind should do, because this causes such files to be automatically removed during the rewind on the target. Exclusion filters apply to paths which are removed or recreated automatically at startup, so removing all those files on the target during the rewind is a win. The existing set of TAP tests already stresses the rewind of databases, but it did not include any tables on those newly-created databases. Creating extra tables in this case is enough to reproduce the failure, so the existing tests are extended to close the gap. Reported-by: Mithun Cy Author: Michael Paquier Discussion: https://postgr.es/m/CADq3xVYt6_pO7ZzmjOqPgY9HWsL=kLd-_tNyMtdfjKqEALDyTA@mail.gmail.com Backpatch-through: 11
2019-03-16Don't propagate PGAPPNAME through pg_ctl in testsPeter Eisentraut
When libpq is loaded in the server (for instance, by libpqwalreceiver), it may use libpq environment variables set in the postmaster environment for connection parameter defaults. This has some confusing effects in our test suites. For example, the TAP test infrastructure sets PGAPPNAME to allow identifying clients in the server log. But this environment variable is also inherited by temporary servers started with pg_ctl and is then in turn used by libpqwalreceiver as the application_name for connecting to remote servers where it then shows up in pg_stat_replication and is relevant for things like synchronous_standby_names. Replication already has a suitable default for application_name, and overriding that accidentally then requires the individual test cases to re-override that, which is all very confusing and unnecessary. To fix, unset PGAPPNAME temporarily before running pg_ctl start or restart in the tests. More comprehensive approaches like unsetting all environment variables in pg_ctl were considered but might be too complicated to achieve portably. The now unnecessary re-overriding of application_name by test cases is also removed. Reviewed-by: Noah Misch <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/[email protected]
2019-02-07Fix searchpath and module location for pg_rewind and ssl TAP testsAndrew Dunstan
The modules RewindTest.pm and ServerSetup.pm are really only useful for TAP tests, so they really belong in the TAP test directories. In addition, ServerSetup.pm is renamed to SSLServer.pm. The test scripts have their own directories added to the search path so that the relocated modules will be found, regardless of where the tests are run from, even on modern perl where "." is no longer in the searchpath. Discussion: https://postgr.es/m/[email protected] Backpatch as appropriate to 9.5