Auditing and Monitoring PostgreSQL/EPAS

Auditing and Monitoring
EDB Advanced Server
database
Presented by:
Ashutosh Sharma
Principal Software Engineer, EDB

© Copyright EnterpriseDB Corporation, 2020. All rights reserved.
2
Agenda
Auditing
● What is database auditing?
● What is the need of database auditing?
● What is EDB Audit Logging feature?
● How to use EDB Audit Logging feature?
● What is Object level auditing in EDB?
● How to filter Audit Logs using error codes?
● How to redact passwords in Audit Logs?
Monitoring
● The PostgreSQL Statistics Collector
● Statistics Views and their use cases

3
Agenda (Continued)
Monitoring
● pg_stat_statements
● dbms_profiler
Debugging Tools
● pageinspect
● pgstattuple
Q&A

4
Auditing
What is database auditing?
● Database auditing is the process of observing the database activity so as to be aware of the actions of
database users.
● In short, it is a database facility that provides answers to the following questions:
✓ Who viewed and modified the sensitive data in the system?
✓ At what date and time was the data viewed?
✓ Which program or client application was used to view the data?
✓ What was the SQL statement used to view the data?
✓ What data was changed ? (both old and new data should be accessible)

5
Auditing
What is the need of database auditing?
● Preventing database users from taking inappropriate actions
● Investigating suspicious activity
● Identifying abuse of access rights
● Keeping track of changes and updates made to data
● Observing the overall database utilization

6
Auditing
What is EDB Audit Logging feature?
● EDB Audit Logging is a facility that allows auditors, database administrators or security administrators to
track and analyze the database activities.
● It basically generates the audit log files which contains all the relevant information required to track and
analyze the database activities.
● It can be used to record various informations such as,
✓ the date and time when a database user established a connection to the Advanced Server.
✓ the number of failed authentication attempts.
✓ the database objects that a user created, modified or viewed when connected to the Advanced Server.

7
Auditing
How to use EDB Audit Logging feature?
● By default, auditing is disabled in EDB Advanced Server. So, to use it, we must first enable it.
● To enable auditing, we must set the configuration parameter "edb_audit" to some non-default value (either
csv or xml).
1. csv: enable auditing and write the audit records to a csv file
2. xml: enable auditing and write the audit records to a xml file
● Once auditing is enabled, we must *think* about where to store the audit files?
● The configuration parameter “edb_audit_directory” can be used to set the location for the audit files.

8
Auditing
How to use EDB Audit Logging feature? (Continued)
● And... now the next step would be to decide what exactly we want to audit?
● There are several configuration parameters available which tells us about what information can be logged
into the audit files.
1. edb_audit_connect: enables auditing of all connections to the instance, either successful, failed or all.
2. edb_audit_disconnect: the opposite of edb_audit_connect, enables auditing of all disconnections.
3. edb_audit_statement: enables auditing of different categories of SQL statements such as insert,update, delete,
truncate etc.
4. edb_audit_tag: specify a string value that will be included in the audit log file for each entry as a tracking tag.

9
Auditing
● Demo
Step 1: Enable auditing and set location for audit files
ALTER SYSTEM SET edb_audit='csv';
ALTER SYSTEM SET edb_audit_directory = '/var/lib/edb/audit';
Step 2: Set auditing parameters as required
ALTER SYSTEM SET edb_audit_connect = 'all';
ALTER SYSTEM SET edb_audit_statement TO 'create, insert, update, delete, truncate, select, alter, error';
ALTER SYSTEM SET edb_audit_tag TO 'DEMO';
SELECT pg_reload_conf();

10
Auditing
● Demo
Step 3: Perform changes in the database
CREATE USER adminusr IDENTIFIED BY 'admin' superuser;
c edb adminusr
CREATE TABLE accounts(aid int primary key, aname text, abalance int);
INSERT INTO accounts VALUES (10, 'alex', 100000), (20, 'bob', 500000);
c edb localusr
DELETE FROM adminusr.accounts WHERE aid = 10;
UPDATE adminusr.accounts SET abalance = 400000 WHERE aid = 20;
Step 4: View information captured in the audit file
2020-12-23 17:19:21.572 IST,"ashu","edb",114450,"[local]",5fe32e5b.1bf12,5,"idle",2020-12-23 17:17:39 IST,3/34,0,AUDIT,00000,"statement:
create user adminusr identified by 'admin' superuser;",,,,,,,,,"psql","client backend",,"CREATE ROLE","DEMO","create"

11
Auditing
● Demo
2020-12-23 17:19:21.595 IST,"adminusr","edb",114605,"[local]",5fe32ec1.1bfad,1,"authentication",2020-12-23 17:19:21
IST,4/1,0,AUDIT,00000,"connection authorized: user=adminusr database=edb",,,,,,,,,"","client backend",,"","DEMO","connect"
2020-12-23 17:19:21.600 IST,"adminusr","edb",114605,"[local]",5fe32ec1.1bfad,2,"idle",2020-12-23 17:19:21
IST,4/3,0,AUDIT,00000,"statement: create table accounts(aid int primary key, aname text, abalance int);",,,,,,,,,"psql","client
backend",,"CREATE
2020-12-23 17:19:21.609 IST,"adminusr","edb",114605,"[local]",5fe32ec1.1bfad,3,"idle",2020-12-23 17:19:21
IST,4/4,0,AUDIT,00000,"statement: insert into accounts values (10, 'alex', 100000), (20, 'bob', 500000);",,,,,,,,,"psql","client
backend",,"INSERT","DEMO","insert"
2020-12-23 17:19:21.612 IST,"localusr","edb",114606,"[local]",5fe32ec1.1bfae,1,"authentication",2020-12-23 17:19:21
IST,3/36,0,AUDIT,00000,"connection authorized: user=localusr database=edb",,,,,,,,,"","client backend",,"","DEMO","connect"

12
Auditing
● Demo
2020-12-23 17:19:21.614 IST,"localusr","edb",114606,"[local]",5fe32ec1.1bfae,2,"idle",2020-12-23 17:19:21
IST,3/38,0,AUDIT,00000,"statement: delete from adminusr.accounts where aid = 10;",,,,,,,,,"psql","client backend",,"DELETE","DEMO","delete"
2020-12-23 17:19:22.643 IST,"localusr","edb",114606,"[local]",5fe32ec1.1bfae,3,"idle",2020-12-23 17:19:21
IST,3/39,0,AUDIT,00000,"statement: update adminusr.accounts set abalance = 400000 where aid = 20;",,,,,,,,,"psql","client
backend",,"UPDATE","DEMO","update"
✓ As expected, we can see an entry in the audit file for all the actions taken by database user

13
Auditing
What is Object Level Auditing in EDB?
● It is a new capability added to the EDB Audit Logging feature that allows users to perform auditing at the per-
object level.
● To use this new feature, you can add all objects to audit in a group and enable auditing for that group.
● Currently, this feature can only be used for tables.
● Demo
Step 1: Create some tables (objects)
CREATE TABLE t11(a int);

14
Auditing
What is Object Level Auditing in EDB? (Continued)
● Demo
Step 2: Add tables (objects) to different groups
ALTER TABLE t11 SET (edb_audit_group = 'group1');
Step 3: Enable auditing for a group (which may contain a single object)
ALTER SYSTEM SET edb_audit_statement = 'select@group1, insert@group1';
SELECT pg_reload_conf();
Step 4: Make some changes to the objects created in step 1
INSERT INTO t11 VALUES(10);
SELECT * FROM t11;

15
Auditing
What is Object Level Auditing in EDB? (Continued)
● Demo
Step 4: Make some changes to the objects created in step 1
UPDATE t11 SET a = 11;
DELETE t11;
Step 5: View information captured in audit file
2020-12-24 06:52:09.647 IST,"localusr","edb",114606,"[local]",5fe32ec1.1bfae,24,"INSERT",2020-12-23 17:19:21
IST,3/62,0,AUDIT,00000,"statement: INSERT INTO t11 VALUES(10);",,,,,,,,,"psql","client backend",,"INSERT","DEMO","insert"
2020-12-24 06:52:09.650 IST,"localusr","edb",114606,"[local]",5fe32ec1.1bfae,25,"SELECT",2020-12-23 17:19:21
IST,3/64,0,AUDIT,00000,"statement: SELECT * FROM t11;",,,,,,,,,"psql","client backend",,"SELECT","DEMO","select"
✓ As expected, there is no entry in the audit file for the changes made in table t21 as it belongs to a group that
is not being audited. Also, only SELECT and INSERT operation on table t11 of group1 got logged.

16
Auditing
How to filter Audit Logs using error codes?
● EDB Advanced Server includes an extension (named edb_filter_log) that can be used to filter a user-specified
error codes from audit files
● To filter audit log entries, you must first load the extension by adding the following value to
share_preload_libraries parameter in postgresql.conf.
shared_preload_libraries = $libdir/edb_filter_log
● Then, use the edb_filter_log.errcodes parameter to specify any error codes to be omitted from the log files.
edb_filter_log.errcode = 'error_code'
where error_code can be one or more error codes that you wish to omit from the log file.

17
Auditing
How to filter Audit Logs using error codes? (Continued)
● Demo
Step 1: Create a test table (containing primary key column) and insert some data into it
CREATE TABLE t1(a int primary key);
Step 2: Set edb_filter_log.errcode to some value (e.g. 23505 - for violating a unique constraint)
SET edb_filter_log.errcode TO ‘23505’;
Step 3: Insert some values in the test table to generate an error with code 23505 and some other errors
INSERT INTO t1 VALUES(null);

18
Auditing
How to filter Audit Logs using error codes? (Continued)
● Demo
2020-12-24 09:40:54.655 IST,"ashu","edb",1691,"[local]",5fe41274.69b,17,"idle",2020-12-24 09:30:52
IST,2/21,0,AUDIT,00000,"statement: insert into t1 values(1);",,,,,,,,,"psql","client backend",,"INSERT","DEMO","insert"
2020-12-24 09:41:00.143 IST,"ashu","edb",1691,"[local]",5fe41274.69b,18,"idle",2020-12-24 09:30:52
IST,2/22,0,AUDIT,00000,"statement: insert into t1 values(null);",,,,,,,,,"psql","client backend",,"INSERT","DEMO","insert"
2020-12-24 09:41:00.143 IST,"ashu","edb",1691,"[local]",5fe41274.69b,19,"INSERT",2020-12-24 09:30:52
IST,2/22,0,ERROR,23502,"null value in column ""a"" of relation ""t1"" violates not-null constraint","Failing row contains
(null).",,,,,"insert into t1 values(null);",,,"psql","client backend",,"INSERT","DEMO","error"
✓ As expected, we can see an entry in the audit file for the null-constraint violation, but here is no entry for
unique constraint violation (with error code 23505)

19
Auditing
How to redact passwords in Audit Logs?
● edb_filter_log extension in EDB Advanced Server has the capability to hide the passwords specified with
CREATE or ALTER USER command
● You may use edb_filter_log.redact_password_commands option to instruct the server to redact stored
passwords from the log file
● It only recognizes the following syntax:
{CREATE|ALTER} {USER|ROLE|GROUP} identifier { [WITH] [ENCRYPTED] PASSWORD
'nonempty_string_literal' | IDENTIFIED BY { 'nonempty_string_literal' | bareword } } [ REPLACE {
'nonempty_string_literal' | bareword } ]

20
Auditing
How to redact passwords in Audit Logs? (Continued)
● Demo
Step 1: Enable password redaction
SET edb_filter_log.redact_password_commands = TRUE;
Step 2: Create a new user or change the password of an existing user
CREATE USER u1 IDENTIFIED BY ‘xyz@123’;
ALTER USER u1 PASSWORD 'pqr@123' replace 'xyz@123';
Step 3: View information captured in audit file
2020-12-24 11:13:24.073 IST,"ashu","edb",1691,"[local]",5fe41274.69b,25,"idle",2020-12-24 09:30:52 IST,2/29,0,AUDIT,00000,"statement:
create user u1 identified by 'x';",,,,,,,,,"psql","client backend",,"CREATE ROLE","DEMO","create"
2020-12-24 11:14:17.591 IST,"ashu","edb",1691,"[local]",5fe41274.69b,26,"idle",2020-12-24 09:30:52 IST,2/30,0,AUDIT,00000,"statement:
alter user u1 password 'x' replace 'x';",,,,,,,,,"psql","client backend",,"ALTER ROLE","DEMO","alter"

21
Monitoring
The PostgreSQL Statistics Collector
● It is an optional process in postgres which is ON by default.
● It collects server-wide database statistics that can be used for monitoring the overall database activity.
● Each individual processes transmit new statistical counts to the collector process just before going to idle
state, the collector process then collects the stats sent by backend process and writes the data into some stats
file which can be read via number of views.
● The behaviour of this process is dependent on a set of track parameters, which tells the stats collector about
which metrics it needs to collect from the running instance.

22
Monitoring
The PostgreSQL Statistics Collector (Continued)
● Track parameters associated with the Statistics Collector
1. track_activities : enables monitoring of the current command being executed by any backend process , on by
default.
2. track_activity_query_size : decides the space in terms of bytes reserved to store user query, 1024 is the default
value.
3. track_counts : allows the stats collector process to collect all the base table and index table access related
statistics and store them into the pg_stat_tmp location in the form of db_<database_oid>.stat or globals.stat, on
by default.
4. track_io_timing : enables monitoring of disk blocks read and write time i.e. the time spent on disk blocks
read/write operations by each backend process, off by default.
5. track_functions : controls tracking of metrics about the user level functions, default value is none meaning that
it won't be tracking any type of user functions, can be set to pl, C, all..

23
Monitoring
Statistics Views
● Statistics Views aka "Catalog Views" are the set of predefined views used to view the database statistics and
various activities performed by the database server at the run-time.
● It uses data collected by the collector process to report various database activities at run-time.
● There are several statistics views present in PostgreSQL. In this presentation, we will just talk about some of
them which includes:
✓ pg_stat_database
✓ pg_stat_all_tables
✓ pg_stat_activity

24
Monitoring
Statistics Views (Continued)
pg_stat_database
● The database-level statistics are saved in the pg_stat_database view.
● It contains one row for each database, showing database-wide statistics.
● It shows the informations such as the number of backend processes currently connected to a database,
number of transactions committed or rollback in a particular database, number of data blocks read from disk
or the total time spent in disk read or write activities.
● For details on the layout of pg_stat_database statistics view, have a look at the documentation about
pg_stat_database

25
Monitoring
Use cases of pg_stat_database
● Getting statistics like the cache hit ratio, dml statistics, transaction statistics etc. for a particular database
● Example:
SELECT datname, round((blks_hit::float / (blks_read+blks_hit+1) * 100)::numeric, 2) as hitratio, xact_commit, xact_rollback,
tup_fetched, tup_inserted, tup_updated FROM pg_stat_database WHERE datname NOT IN ('template0', 'template1')ORDER BY hitratio
desc;
datname | hitratio | xact_commit | xact_rollback | tup_fetched | tup_inserted | tup_updated
----------+----------+-------------+---------------+-------------+--------------+-------------
edb | 98.67 | 98 | 3 | 8805 | 1000041 | 1
postgres | 91.97 | 36 | 0 | 5548 | 0 | 0
(2 rows)

26
Monitoring
Use cases of pg_stat_database
● Finding the total number of temp files generated in the database
● Example:
SELECT temp_files, temp_bytes FROM pg_stat_database WHERE datname = current_database();
SHOW work_mem;
● Monitoring database loads
● Example:
SELECT numbackends , xact_commit , xact_rollback, blks_read + blks_hit as total_buffer_read FROM pg_stat_database where datname
NOT IN ('template0', 'template1') order by xact_commit desc;
numbackends | xact_commit | xact_rollback | total_buffer_read
-------------+-------------+---------------+-------------------
1 | 99 | 3 | 1094106
0 | 36 | 0 | 11264

27
Monitoring
pg_stat_all_tables
● The pg_stat_all_tables view contains one row for each table (which includes system table or a user table or
may be TOAST table) in the current database, showing statistics about accesses to that specific table.
● The pg_stat_user_tables and pg_stat_sys_tables views contain the same information as pg_stat_all_tables,
but are restricted to only user and system tables respectively.
● For details on the layout of pg_stat_all_tables statistics view, have a look at the documentation about
pg_stat_all_table.

28
Monitoring
Use cases of pg_stat_all_tables
● Finding top 10 most read tables in the database
SELECT relname, idx_tup_fetch + seq_tup_read as TotalReads FROM pg_stat_all_tables WHERE idx_tup_fetch + seq_tup_read != 0
order by TotalReads desc LIMIT 10;
relname | totalreads
--------------+------------
pg_class | 4255
pg_attribute | 3651
pg_opclass | 1170
pg_proc | 862

29
Monitoring
Use cases of pg_stat_all_tables
● Autovacuum monitoring
SELECT schemaname, relname, last_autovacuum, last_autoanalyze FROM pg_stat_all_tables WHERE relname='tab1';
● Checking for the dead tuples count to see if a table needs to be manually VACUUMED or not..
SELECT relname, last_vacuum, n_dead_tup, last_analyze FROM pg_stat_all_tables where relname='tab1';
● Finding the ratio of index scan to seq scan on a table.
SELECT sum(idx_scan)/(sum(idx_scan) + sum(seq_scan)) as idx_scan_ratio FROM pg_stat_all_tables WHERE schemaname='public';

30
Monitoring
pg_stat_activity
● The pg_stat_activity view shows what activity is currently happening on your PostgreSQL database server.
● It contains one row per server process and shows some very useful informations like the current state of a
running backend process, the query that the client process is currently running, query start time or
transaction start time, the wait event on which the client is currently waiting and so on...
● In short, pg_stat_activity basically provides a way to get a snapshot of what every client on the server is
currently doing.
● For details on the layout of pg_stat_activity statistics view, have a look at the documentation about
pg_stat_activity

31
Monitoring
Use cases of pg_stat_activity
● Finding out the number of connections to your database
Example
CREATE VIEW get_active_sessions AS SELECT datname, count(*) AS open, count(*) FILTER (WHERE state= 'active') AS active, count(*)
FILTER (WHERE state = 'idle') AS idle, count(*) FILTER (WHERE state ='idle in transaction') AS idle_in_trans FROM pg_stat_activity
GROUP BY datname;
edb=# select * from get_active_sessions;
datname | open | active | idle | idle_in_trans
---------+------+--------+------+---------------
edb | 2 | 1 | 1 | 0

32
Monitoring
● Finding and killing long running idle database connections
Example
SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = 'postgres' AND pid <> pg_backend_pid() AND state in
('idle', 'idle in transaction', 'idle in transaction (aborted)', 'disabled') AND state_change < current_timestamp - INTERVAL '5' DAY;
● Detecting long running queries or transactions...
Example to find out a query running for very long time, say more than 2 hours on PostgreSQL, you can run the following command,
SELECT pid, datname, username, client_addr, now() - query_start as "runtime", query_start, wait_event_type, wait_event, state, query
FROM pg_stat_activity WHERE now() - query_start > '2 hours'::interval ORDER BY runtime DESC;

33
Monitoring
● Wait Event Monitoring for a long running queries
Example
SELECT pid, now() - query_start as "runtime", wait_event_type, wait_event, state, query FROM pg_stat_activity WHERE now()-
query_start > '5 hours'::interval ORDER BY runtime DESC;
● Finding blocked sessions
Example
SELECT datname, username, application_name, now()-backend_start AS "Session duration", pid, query FROM pg_stat_activity WHERE
state='active' AND wait_event IS NOT NULL;

34
Monitoring
pg_stat_statement
● pg_stat_statements is an extension module that tracks the execution statistics of all SQL statements
executed by a server and stores them in a pg_stat_statements table (which is basically a hash table).
● It's a module that needs to be loaded and is not available in the default configuration. It can be loaded by
adding pg_stat_statements to shared_preload_libraries in postgresql.conf.
● Whenever any SQL query is executed by a server, pg_stat_statements adds an entry for that query in the hash
table where all the statistics about the query execution are stored.
● When user queries pg_stat_statements view, it fetches the stats from the hash table.

35
Monitoring
pg_stat_statements (Continued)
● Track parameters associated with pg_stat_statements
1. pg_stat_statements.max : pg_stat_statements.max is the maximum number of statements tracked by
the pg_stat_statements module (i.e., the maximum number of rows in the pg_stat_statements table)
2. pg_stat_statements.track : pg_stat_statements.track specifies the statements that can be tracked by
pg_stat_statements module. It can be only top level statement or all the statements including the
nested statements or none.
3. pg_stat_statements.track_utility : pg_stat_statements.track_utility controls whether utility commands
(other than SELECT, INSERT, UPDATE, DELETE) are tracked by the module.
4. pg_stat_statements.save : pg_stat_statements.save specifies whether to save statement statistics
across server shutdowns. If it is off then statistics are not saved at shutdown nor reloaded at server
start. The default value is on.

36
Monitoring
● Finding statistics of the queries track by pg_stat_statements
SELECT queryid, query, calls, total_exec_time, min_exec_time, max_exec_time,shared_blks_hit, shared_blks_read FROM
pg_stat_statements WHERE calls > 3;
queryid | -6743957081790477185
query | insert into t1 values($1)
calls | 7
total_exec_time | 3.209689
min_exec_time | 0.032976
max_exec_time | 2.954937
shared_blks_hit | 6
shared_blks_read | 2

37
Monitoring
● Monitoring query performance using pg_stat_statements
SELECT substring(query, 1, 30) AS short_query, round(total_exec_time::numeric, 2) AS total_time, calls,
round(mean_exec_time::numeric, 2) AS mean,round((100 * total_exec_time / sum(total_exec_time::numeric) OVER ())::numeric, 2) AS
pct_cpu, shared_blks_hit AS blks_hit, shared_blks_read AS blks_read FROM pg_stat_statements ORDER BY total_time DESC LIMIT 5;
short_query | total_time | calls | mean | pct_cpu | blks_hit | blks_read
--------------------------------+------------+--------+----------+---------+----------+-----------
copy pgbench_accounts from std | 66724.81 | 1 | 66724.81 | 41.85 | 6 | 0
SELECT abalance FROM pgbench_a | 55293.41 | 350043 | 0.16 | 34.68 | 1371736 | 378479
alter table pgbench_accounts a | 20908.93 | 1 | 20908.93 | 13.12 | 2216 | 489724
vacuum analyze pgbench_account | 16353.95 | 1 | 16353.95 | 10.26 | 494126 | 519776
vacuum analyze pgbench_branche | 36.37 | 1 | 36.37 | 0.02 | 185 | 36
(5 rows)

38
Monitoring
dbms_profiler
● dbms_profiler is the package that provides a set of functions to collect the performance statistics about the
SPL and PLpgSQL statements.
● It gathers the performance information of each line inside the SPL or PLpgSQL block and store them into a
statistics table named PLPGSQL_PROFILER_RAWDATA
● Objects created by dbms_profiler
Functions
1. START_PROFILER
2. STOP_PROFILER
3. PAUSE_PROFILER
4. RESUME_PROFILER
5. FLUSH_DATA
Tables
1. PLSQL_PROFILER_RUNS
2. PLSQL_PROFILER_UNITS
3. PLSQL_PROFILER_RAWDATA

39
Monitoring
dbms_profiler (Continued)
● Demo
Step 1: Create SPL objects to profile
CREATE OR REPLACE FUNCTION spl_func_ins() RETURN VOID IS
BEGIN
END;
CREATE OR REPLACE FUNCTION spl_func_upd() RETURN VOID IS
BEGIN
UPDATE t1 SET a = 200;
END;

40
Monitoring
● Demo
Step 2: Start profiling
EXEC dbms_profiler.start_profiler('SPL_FUNC');
SELECT spl_func_ins();
SELECT spl_func_upd();
EXEC dbms_profiler.stop_profiler;
Step 3: Query stats table to view the performance data
SELECT runid, run_comment, run_total_time FROM plsql_profiler_runs;
runid | run_comment | run_total_time
-------+-------------+----------------
1 | SPL_FUNC | 1110

41
Monitoring
SELECT runid, sourcecode, line_number, time_total*1000000, exec_count FROM plsql_profiler_rawdata;
runid | sourcecode | line_number | ?column? | exec_count
-------+-------------------------------+-------------+----------+------------
1 | | 1 | 0 | 0
1 | BEGIN | 2 | 3 | 1
1 | INSERT INTO t1 VALUES(100); | 3 | 367 | 1
1 | END | 4 | 0 | 0
1 | | 1 | 0 | 0
1 | BEGIN | 2 | 3 | 1
1 | UPDATE t1 SET a = 200; | 3 | 737 | 1
1 | END | 4 | 0 | 0

42
Debugging Tools
pageinspect
● pageinspect is an extension module in postgres that provides functions to inspect the contents of database
pages at low level which can be used for debugging.
● It includes various user exposed functions that can be used to view the contents of heap and different index
pages.
● It is particularly useful in understanding the changes happening at page level when various actions are
performed on a relation.
● Inspecting heap and index pages using pageinspect
CREATE TABLE tab1(a int4 primary key);
SELECT txid_current();
INSERT INTO tab1 VALUES(10);
CREATE EXTENSION pageinspect;
SELECT lp, t_xmin, t_xmax, lp_off FROM heap_page_items(get_raw_page('tab1', 0));

43
Debugging Tools
pageinspect (Continued)
edb=# SELECT lp, lp_len, t_xmin, t_xmax, lp_off FROM heap_page_items(get_raw_page('tab1', 0));
lp | lp_len | t_xmin | t_xmax | lp_off
----+--------+--------+--------+--------
1 | 28 | 1306 | 0 | 8160
(1 row)
UPDATE tab1 SET a=20 WHERE a=10;
----+--------+--------+--------+--------
1 | 28 | 1306 | 1307 | 8160
2 | 28 | 1307 | 0 | 8128

44
Debugging Tools
pageinspect (Continued)
edb=# SELECT * FROM bt_page_items('tab1_pkey', 1);
itemoffset | ctid | itemlen | nulls | vars | data | dead | htid | tids
------------+-------+---------+-------+------+-------------------------+------+-------+------
1 | (0,1) | 16 | f | f | 0a 00 00 00 00 00 00 00 | f | (0,1) |
2 | (0,2) | 16 | f | f | 14 00 00 00 00 00 00 00 | f | (0,2) |
edb=# VACUUM;
----+--------+--------+--------+--------
1 | 0 | | | 0
2 | 28 | 1307 | 0 | 8160

45
Debugging Tools
pgstattuple
● pgstattuple is another extension module in postgres that provides table-level statistics.
● This contrib module is particularly useful in identifying the tables which have bloated and how much bloat is
there.
● Like pageinspect, this module also provides a set of functions that can be used to identify the bloated tables in
postgres.
● Identifying bloated tables using pgstattuple
pgbench -i -s10 edb
Session 1:
edb=# dt+ pgbench_accounts
Schema | Name | Type | Owner | Persistence | Access Method | Size | Description
--------+------------------+-------+-------+-------------+---------------+--------+-------------
public | pgbench_accounts | table | ashu | permanent | heap | 128 MB |

47
Debugging Tools
pgstattuple (Continued)
Session 1:
VACUUM ANALYZE pgbench_accounts;
CREATE EXTENSION pgstattuple;
SELECT table_len, scanned_percent, approx_free_space, approx_free_percent FROM pgstattuple_approx('pgbench_accounts');
table_len | scanned_percent | approx_free_space | approx_free_percent
-----------+-----------------+-------------------+---------------------
268607488 | 91 | 131164800 | 48.8314011558754
✓ The amount of free space available in the table clearly states that it is a bloated table.

48
• Auditing
https://www.enterprisedb.com/edb-docs/d/edb-postgres-advanced-server/user-guides/user-
guide/10/EDB_Postgres_Advanced_Server_Guide.1.39.html
• Monitoring
https://www.postgresql.org/docs/12/monitoring-stats.html
• Debugging Tools
https://www.postgresql.org/docs/12/pageinspect.html
https://www.postgresql.org/docs/12/pgstattuple.html
References

50 CONFIDENTIAL
The largest dedicated PostgreSQL company
• More customers: Than any dedicated PostgreSQL company
• More experts: Leading PostgreSQL contributors
• More innovation: Positioned to lead in enterprise PostgreSQL
and hybrid cloud
EDB acquires 2ndQuadrant in Sept 2020
+

51 CONFIDENTIAL
EDB supercharges PostgreSQL
Largest dedicated
PostgreSQL company
Major PostgreSQL
community leader
Over 5,000 customers -
1 in 4 of Fortune 500
Founded in
2004
Over 10 years of
consecutive quarterly
subscription growth
500+
employees
Global presence
Recognised leader in Relational
Database Management Systems
(RDBMS) by both Gartner and Forrester

Auditing and Monitoring PostgreSQL/EPAS

More Related Content

What's hot (20)

Similar to Auditing and Monitoring PostgreSQL/EPAS (20)

More from EDB (20)

Recently uploaded (20)

Auditing and Monitoring PostgreSQL/EPAS