EPAS Essentials

Course Agenda
• Introduction and Architectural • Creating and Managing Database
Overview Objects
• System Architecture • Database Security
• EDB Postgres Advanced Server • Monitoring and Admin Tools Overview
Installation • SQL Primer
• User Tools - Command Line Interfaces • Backup and Recovery
• Database Clusters • Routine Maintenance Tasks
• Database Configuration • Data Loading
• Data Dictionary • Data Replication and High Availability

2 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 1
Introduction
 Module Objectives

• EDB Portfolio
• Facts about PostgreSQL and EDB Postgres Advanced Server
• Major Features
• EDB Postgres Advanced Server Database Features
• General Database Limits
• Common Database Object Names

4 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB Supported Databases

Postgres Postgres Extended Postgres Advanced Server

Open source Postgres EDB proprietary distribution for EDB EDB proprietary distribution with
Postgres Distributed use cases Transparent Data Encryption
with Transparent Data Encryption

• EDB continues to be committed • SQL compatible with Postgres, • SQL compatible with Oracle,
to advancing features in extended for stringent availability reduces effort to migrate
collaboration with the broader and advanced replication needs applications and data to
community • Transparent Data Encryption Postgres
• Formerly known as • Transparent Data Encryption
2ndQPostgres • Additional value-add enterprise
features

5 © Copyright EnterpriseDB Corporation. All rights reserved.

 PostgreSQL
The open source database of choice

Performance Scalability Extensibility Community-driven

Handles enterprise workloads Multiple technical options for Supported by a wide array of Multiple companies and
with 50% improvement in the operating Postgres at scale extensions plus multiple SQL individuals contribute to
last 4 years and NoSQL data models the project and drive
innovation

6 © Copyright EnterpriseDB Corporation. All rights reserved.

 Facts about PostgreSQL

• The world’s most advanced open source database

• Designed for extensibility and customization
• ANSI/ISO compliant SQL support
• Actively developed for more than 20 years
• University Postgres (1986-1993)
• Postgres95 (1994-1995)
• PostgreSQL (1996-current)

7 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB Postgres Extended Server

Replication Enhancements
EDB Postgres Extended Server

PostgreSQL

● Replication Enhancements
○ Enables EDB Postgres Distributed functionality such as:
■ Group Commit, Commit at Most Once, and Eager all-node synchronous replication
■ Timestamp-based Snapshots
■ Estimates for Replication Catch-up times
■ Selective Backup of a Single Database
■ Hold back freezing to assist resolution of UPDATE/DELETE conflicts
■ Multi-node PITR
■ Application Assessment

● Only available for use with an additional subscription for Extreme HA

8 © Copyright EnterpriseDB Corporation. All rights reserved.
 EDB Postgres Advanced Server
EDB Postgres Advanced Server

Oracle Compatibility ● Oracle Compatibility - Compatibility for schemas, data types, indexes, users, roles,
partitioning, packages, views, PL/SQL triggers, stored procedures, functions, and utilities
● Additional Security - Password policy management, session tag auditing, data redaction, SQL
Additional Security injection protection, and procedural language code obfuscation
● Developer Productivity - Over 200 pre-packaged utility functions, user-defined object types,
Developer Productivity autonomous transactions, nested tables, synonyms, advanced queueing

● DBA Productivity - Throttle CPU and I/O at the process level, over 55 extended catalog views
DBA Productivity to profile all the objects and processing that occurs in the database

● Performance - Query optimizer hints, SQL session/system wait diagnostics

Performance
● Replication Enhancements - Enables EDB Postgres Distributed functionality such as Group
Commit, Commit at Most Once and Eager all-node synchronous replication, timestamp-based
Replication Enhancements
snapshots, estimates for replication catch-up times, selective backup of a single database,
hold back freezing to assist resolution of UPDATE/DELETE conflicts, multi-node PITR
PostgreSQL

9 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Servers - High Level Overview
EDB Postgres EDB Postgres Advanced EDB Postgres Advanced
Database Server PostgreSQL
Extended Server Server: Berkeley Server: Redwood

SQL Compatibility PostgreSQL PostgreSQL PostgreSQL + Oracle

Binary Compatibility Yes No No No

Advanced PGD Features ✔️ 14+ 14+

Transparent Data Encryption 15+ 15+ 15+

Advanced Security ✔️ ✔️

Advanced SQL ✔️ ✔️

Advanced Performance ✔️ ✔️

Resource Manager ✔️ ✔️

Bulk Data Loader ✔️ ✔️

Oracle Compatibility ✔️

10 © Copyright EnterpriseDB Corporation. All rights reserved.

 Capabilities And Tools

Management/Monitoring High Availability Backup and Recovery

Postgres Enterprise Manager EDB Postgres Distributed Barman
pgAdmin Failover Manager pgBackRest
Repmgr
Patroni

Migration Integration Kubernetes

Migration Portal Connectors EDB Postgres for Kubernetes
Migration Toolkit Foreign Data Wrappers CloudNativePG
Replication Server Connection Poolers
11 © Copyright EnterpriseDB Corporation. All rights reserved.
 Major Features
• Portable:
• Written in ANSI C
• Supports Windows, Linux, Mac OS/X and major UNIX platforms

• Reliable:
• ACID Compliant
• Supports Transactions and Savepoints
• Uses Write Ahead Logging (WAL)

• Scalable:
• Uses Multi-version Concurrency Control
• Table Partitioning and Tablespaces
• Parallel Sequential Scans, DDL(Table and Index Creation)

12 © Copyright EnterpriseDB Corporation. All rights reserved.

 Major Features(continued)
• Secure:
• Employs Host-Based Access Control
• Provides Object-Level Permissions and Row Level Security
• Supports SSL Connections and Logging
• Transparent Data Encryption - TDE

• Recovery and Availability:

• Streaming Replication, Logical Replication and Replication Slots
• Replication Slots, Sync or Async Options
• Supports Hot-Backup, pg_basebackup and Point-in-Time Recovery

• Advanced:
• Supports Triggers, Functions and Procedures
• Supports Custom Procedural Languages
• Upgrade using pg_upgrade
• Unlogged Tables and Materialized Views
• Just-in-Time (JIT) Compilation

13 © Copyright EnterpriseDB Corporation. All rights reserved.

 Postgres for Big Data

• Postgres enables you to support a wider range of workloads

with your relational database
• An Object-relational design and decades of proven reliability make
Postgres the most flexible, extensible and performant database
available
• Document store capabilities: XML, JSON, PLV8; HStore (key-value
store); non-durable storage; full text indexing
• Foreign Data Wrappers enable read/write integration with other
database technologies

14 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Database Limits
Limit Value
Maximum Database Size Unlimited

Maximum Table Size 32 TB

Maximum Row Size 1.6 TB

Maximum Field Size 1 GB

Maximum Rows per Table Unlimited

Maximum Columns per Table 250-1600 (Depending on Column types)

Maximum Indexes per Table Unlimited

15 © Copyright EnterpriseDB Corporation. All rights reserved.

 Common Database Object Names

Industry Term Postgres Term

Table or Index Relation

Row Tuple

Column Attribute

Data Block Page (when block is on disk)

Page Buffer (when block is in memory)

16 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Setup Guidelines

• All the instructor demos and labs are based on Linux

• CentOS 7 machine or virtual machine with at least 1 GB
RAM and 10 GB storage space is recommended
• Participants using Linux must follow instructor during the
installation module and install EDB Postgres Advanced
Server

17 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• EDB Portfolio
• Facts about PostgreSQL and EDB Postgres Advanced Server
• Major Features
• EDB Postgres Advanced Server Database Features
• General Database Limits
• Common Database Object Names

18 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 2
System Architecture
 Module Objectives

• Architectural Summary • Background Writer Cleaning

Scan
• Process and Memory
• Commit and Checkpoint
Architecture
• Statement Processing
• Utility Processes
• Physical Database Architecture
• Connection Request-Response
• Data Directory Layout
• Disk Read Buffering
• Installation Directory Layout
• Disk Write Buffering • Page Layout

20 © Copyright EnterpriseDB Corporation. All rights reserved.

 Architectural Summary

• EDB Postgres Advanced Server uses processes, not

threads
• The postmaster process acts as a supervisor
• Several utility processes perform background work
• postmaster starts them, restarts them if they die
• One backend process per user session
• postmaster listens for new connections

21 © Copyright EnterpriseDB Corporation. All rights reserved.

 Process and Memory Architecture
Postmaster

Shared Memory
Shared Buffers WAL Buffers Process Array

BGWRITER STATS COLLECTOR

WAL Archived
Data Files Segments
CHECKPOINTER ARCHIVER WAL

AUTOVACUUM WRITER

LOGICAL Error Log

WAL WRITER
REPLICATION
Files
DBMS_AQ

22 © Copyright EnterpriseDB Corporation. All rights reserved.

22
 Utility Processes
• Background writer
• Writes dirty data blocks to disk

• WAL writer
• Flushes write-ahead log to disk

• Checkpointer
• Automatically performs a checkpoint based on config parameters

• Autovacuum launcher
• Starts Autovacuum workers as needed

• Autovacuum workers
• Recover free space for reuse

23 © Copyright EnterpriseDB Corporation. All rights reserved.

 More Utility Process
• Logging collector
• Routes log messages to syslog, eventlog, or log files

• Stats collector
• Collects usage statistics by relation and block

• Archiver
• Archives write-ahead log files

• Logical replication launcher

• Starts logical replication apply process for logical replication

• Dbms_aq launcher
• Collects information for queueing functionality of advanced server

24 © Copyright EnterpriseDB Corporation. All rights reserved.

 Postmaster as Listener

• Postmaster is the main process

called postgres Client requests a

• Listens on 1, and only 1, tcp port connection

• Receives client connection

Postmaster
requests

Shared Memory

25
25 © Copyright EnterpriseDB Corporation. All rights reserved.
 User Backend Process

• Postmaster process spawns a new Postmaster

server process for each
connection request detected
• Communication is done using
semaphores and shared memory
work_mem
• Authentication - IP, user and Postgres
password
• Authorization - Verify permissions
Shared Memory

26
26 © Copyright EnterpriseDB Corporation. All rights reserved.
 Respond to Client

• User backend process Postmaster

called postgres
• Callback to client
work_mem

• Waits for SQL Postgres

• Query is transmitted Shared Memory

using plain text

27 © Copyright EnterpriseDB Corporation. All rights reserved.

 Disk Read Buffering

• EDB Postgres buffer Postgres Postgres Postgres

cache (shared_buffers)
reduces OS reads Shared (data) Buffers

Shared Memory
• Read the block once,
then examine it many
times in cache Stable Databases

28 © Copyright EnterpriseDB Corporation. All rights reserved.

 Disk Write Buffering

• Blocks are written to Postgres Postgres Postgres

disk only when needed:

Shared (data) Buffers
• To make room for new
blocks Shared Memory

• At checkpoint time
CHECKPOINT
Stable Databases

29 © Copyright EnterpriseDB Corporation. All rights reserved.

 Background Writer Cleaning Scan

Postgres Postgres Postgres

• Background writer scan
attempts to ensure an
adequate supply of Shared (data) Buffers

clean buffers Shared Memory

• Back end write dirty

BGWRITER
buffers as need
Stable Databases

30 © Copyright EnterpriseDB Corporation. All rights reserved.

 Write Ahead Logging (WAL)

• Back end write data to Postgres Postgres Postgres

WAL buffers Shared Memory

• Flush WAL buffers Shared(data) Buffers WAL

Buffers
periodically (WAL
writer), on commit, or
when buffers are full
Transaction Log
• Group commit Stable Databases

31 © Copyright EnterpriseDB Corporation. All rights reserved.

 Transaction Log Archiving
Postgres Postgres Postgres

• Archiver spawns a task

Shared Memory
to copy away pg_wal
WAL
log files when full Shared (data) Buffers Buffers

Transaction Log

Archive Command Stable Databases

32 © Copyright EnterpriseDB Corporation. All rights reserved.

 Commit and Checkpoint

• Before commit
• Uncommitted updates are in memory

• After commit
• WAL buffers are written to the disk (write-ahead log file) and shared
buffers are marked as committed

• After checkpoint
• Modified data pages are written from shared memory to the data files

33 © Copyright EnterpriseDB Corporation. All rights reserved.

 Statement Processing

• Check syntax Optimize

• Call traffic cop
• Identify query type • Execute query based
• Command processor if • Planner generates a plan on query plan
needed • Uses database statistics
• Break query into tokens • Apply Optimizer Hints
• Query cost calculation
Parse • Choose best plan
Execute

34 © Copyright EnterpriseDB Corporation. All rights reserved.

 Physical Database Architecture

• Database cluster is a collection of databases managed by

single server instance
• Each cluster has a separate
• Data directory
• TCP port
• Set of processes
• A cluster can contain multiple databases

35 © Copyright EnterpriseDB Corporation. All rights reserved.

 Installation Directory Layout

• Default Installation Directory Location:

• Linux - /usr/edb/as15

• Windows – C:\Program Files\edb\as15

• bin – Programs
• share – Shared data
• include – Header files
• lib or lib64 – Libraries
36 © Copyright EnterpriseDB Corporation. All rights reserved.
 Database Cluster Data Directory Layout

Data

log Status Configuration Postmaster

global base pg_tblsc pg_wal pg_log Info Files
Directories Files
Cluster wide pg_xact, pg_multiexact, postgresql.conf,
Contains Symbolic link to Write ahead
database Startup logs Error logs pg_snapshots, pg_stat, pg_hba.conf,
Databases tablespaces logs
objects pg_subtrans,pg_notify, pg_ident.conf,
pg_serial, pg_replslot, postgresql.auto.conf
pg_logical, pg_dynshmem

37 © Copyright EnterpriseDB Corporation. All rights reserved.

 Physical Database Architecture

• File-per-table, file-per-index
• A table-space is a directory
• Each database that uses that table-space gets a subdirectory
• Each relation using that table-space/database combination gets one
or more files, in 1GB chunks
• Additional files are used to hold auxiliary information (free space
map, visibility map)
• Each file name is a number (see pg_class.relfilenode)

38 © Copyright EnterpriseDB Corporation. All rights reserved.

 Sample - Data Directory Layout
14297

14300
14307
14405
Database OID
base
14498
14312
Data
pg_tblsc
16650
14299
Tablespace OID
/storage/pg_tab
14301

14307 16700

16701
16651

39 © Copyright EnterpriseDB Corporation. All rights reserved.

 Page Layout
• Page header
• General information about the page
• Pointers to free space
• 24 bytes long
• Row/index pointers
• Array of offset/length pairs pointing to the actual rows/index entries
• 4 bytes per item
• Free space
• Unallocated space
• New pointers allocated from the front, new rows/index entries from the rear
• Row/index entry
• The actual row or index entry data
• Special
• Index access method specific data
• Empty in ordinary tables

40 © Copyright EnterpriseDB Corporation. All rights reserved.

 Page Structure
Page
Item Item Item
Header

8K

Tuple

Tuple Tuple Special

41 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Architectural Summary • Background Writer Cleaning

• Shared Memory Scan
• Inter-processes • Commit and Checkpoint
Communication • Physical Database
• Statement Processing Architecture
• Utility Processes • Data Directory Layout
• Disk Read Buffering • Installation Directory Layout
• Disk Write Buffering • Page Layout

42 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 3
EDB Postgres Advanced Server Installation
 Module Objectives

• Deployment Options
• OS User and Permissions
• Package Installation
• Installation of EDB Postgres Advanced Server
• Setting Environmental Variables

44 © Copyright EnterpriseDB Corporation. All rights reserved.

 Deployment Options

• Deployment methods for EDB Postgres Advanced Server and

supported Tools:
• BigAnimal: Fully managed database-as-a-service with built-in Oracle
compatibility.
• EDB PostgreSQL for Kubernetes: Operator designed for managing
PostgreSQL workloads on Kubernetes clusters.
• Native packages or installers: EDB Repository can be used for YUM
and RPM based installation

45 © Copyright EnterpriseDB Corporation. All rights reserved.

 OS User and Permissions
• EDB Postgres Advanced Server runs as a daemon (Unix / Linux) or service
(Windows)
• The EDB Postgres Advanced Server Installation requires
superuser/admin access
• All processes and data files must be owned by a user in the OS
• During installation an enterprisedb locked user will be created on
Linux
• On Windows a password is required
• SELinux must be set to permissive mode on systems with SELinux

46 © Copyright EnterpriseDB Corporation. All rights reserved.

 The enterprisedb User Account
• It is advised to run EDB Postgres Advanced Server under a separate user
account
• This user account should only own the data directory that is managed by
the server
• The useradd or adduser Unix command can be used to add a user
• The user account named enterprisedb is used throughout this
training [root@Base ~]# useradd enterprisedb
[root@Base ~]# passwd enterprisedb
Changing password for user enterprisedb.
New password:
Retype new password:
passwd:
all authentication tokens updated successfully.
47 © Copyright EnterpriseDB Corporation. All rights reserved.
 Package Installation Options

Wizard Installer RPM Installer YUM Installer

• Interactive Method • Preferred Installation • Attempt to Install required
• Graphical or Command Method on Linux package dependencies
Line Mode,
Line Mode, available
available for
for •• Access
Access to
to EnterpriseDB’s
EnterpriseDB’s •• Can
Can be used to
be used to install
install EDB
EDB
Windows
Windows rpm Repository is required
rpm Repository is required Postgres in Isolated
Postgres in Isolated
•• Easy
Easy Download
Download from
from •• Dependencies
Dependencies are
are resolved
resolved Environments
Environments
www.enterprisedb.com
www.enterprisedb.com manually
manually

48 © Copyright EnterpriseDB Corporation. All rights reserved.

YUM Installation
 Configure EDB Repositories
• An EDB account is required to access our software repositories and
downloads
• Setup EDB Repositories: https://www.enterprisedb.com/repos-
downloads
• Ensure EPEL repositories are setup:
sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-
release-latest-7.noarch.rpm

• Check Instructions for various operating systems:

https://www.enterprisedb.com/docs/epas/latest/installing/

50 © Copyright EnterpriseDB Corporation. All rights reserved.

 YUM Installation Overview

• YUM command can be used to install EDB Postgres Advanced Server:

# yum install epel-release edb-as15-server
• Configure a Package Installation using service configuration file
# /usr/lib/systemd/system/edb-as-15.service
• Create a database cluster and start the cluster using services:
# /usr/edb/as15/bin/edb-as-15-setup initdb
# systemctl start edb-as-15

51 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – EDB Postgres Server Installation

• Step 1 – Login as root user and add user enterprisedb

using adduser or useradd command and set its password:
# useradd enterprisedb
# passwd enterprisedb

• Step 2 – Install EPEL using yum:

# yum install epel-release

• Step 3 – Configure EDB Repos 2.0 using your secure token:

# curl -1sLf 'https://downloads.enterprisedb.com/<secure
token>/enterprise/setup.rpm.sh' | sudo -E bash

52 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – EDB Postgres Advanced Server
Installation (continued)
• Step 4 – Install EDB Postgres Advanced server using yum
command
# yum install edb-as15-server

• Step 5 - Create a database cluster and start the cluster using

services:
# /usr/edb/as15/bin/edb-as-15-setup initdb
# systemctl start edb-as-15
# systemctl enable edb-as-15

53 © Copyright EnterpriseDB Corporation. All rights reserved.

After Installation
 Example – Environmental Variables setup
[enterprisedb@pgsrv1 ~]$ vi .bash_profile
Edit User Profile
PATH=/usr/edb/as15/bin/:$PATH:$HOME/.local/bin:$HOME/bin

export PATH
export PGDATA=/var/lib/edb/as15/data/
Logoff and Login
export PGUSER=enterprisedb
export PGPORT=5444
export PGDATABASE=enterprisedb

[enterprisedb@pgsrv1 ~]$ exit

logout
[root@pgsrv1 ~]# su - enterprisedb Verify
Environmental
[enterprisedb@pgsrv1 ~]$ which psql
Settings
/usr/edb/as15/bin/psql

[enterprisedb@pgsrv1 ~]$ pg_ctl status

pg_ctl: server is running (PID: 66406)
/usr/edb/as15/bin/edb-postgres "-D" "/var/lib/edb/as15/data"

55 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• OS User and Permissions

• Installation Options
• Installation of EDB Postgres Advanced Server
• Setting Environmental Variables

56 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

• Choose the platform on which you want to install EDB Postgres

Advanced Server
• Download the EDB Postgres Advanced Server installer from the
EnterpriseDB website for the chosen platform
• Prepare the platform for installation
• Install EDB Postgres Advanced Server
• Connect to EDB Postgres Advanced Server using psql

57 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 4
User Tools - Command Line Interfaces
 Module Objectives
• Introduction to psql
• Connecting to Database
• psql Command Line Parameters
• psql Meta-Commands
• Conditional and Information Commands
• EDB*Plus
• Installing and Starting EDB*Plus
• EDB*Plus Commands

59 © Copyright EnterpriseDB Corporation. All rights reserved.

psql(edb-psql)
 Introduction to psql

• psql is a command line interface (CLI) to Postgres

• Can be used to execute SQL queries and psql meta
commands
[enterprisedb@Base ~]$ psql -h /tmp -p 5444 -U enterprisedb -d edb
Type "help" for help.
edb=# \q

61 © Copyright EnterpriseDB Corporation. All rights reserved.

 Connecting to a Database
psql Connection Options:

• -d <Database Name>
• -h <Hostname>
• -p <Database Port>
• -U <Database Username>

Environmental Variables

• PGDATABASE, PGHOST, PGPORT and PGUSER

62 © Copyright EnterpriseDB Corporation. All rights reserved.

 Conventions

• psql has its own set of commands, all of which start with
a backslash (\).
• Some commands accept a pattern. This pattern is a
modified regex. Key points:
• * and ? are wildcards
• Double-quotes are used to specify an exact name, ignoring all
special characters and preserving case

63 © Copyright EnterpriseDB Corporation. All rights reserved.

 On Startup…

• psql will execute commands from $HOME/.psqlrc, unless

option -X is specified
• -f FILENAME will execute the commands in FILENAME,
then exit
• -c COMMAND will execute COMMAND (SQL or internal) and
then exit
• --help will display all the startup options, then exit
• --version will display version info and then exit

64 © Copyright EnterpriseDB Corporation. All rights reserved.

 Entering Commands

• psql uses the command line editing capabilities that are

available in the native OS. Generally, this means:
• Up and Down arrows cycle through command history
• On UNIX, there is tab completion for various things, such as SQL
commands

65 © Copyright EnterpriseDB Corporation. All rights reserved.

 History and Query Buffer

• \s will show the command history

• \s FILENAME will save the command history
• \e will edit the query buffer and then execute it
• \e FILENAME will edit FILENAME and then execute it
• \w FILENAME will save the query buffer to FILENAME

66 © Copyright EnterpriseDB Corporation. All rights reserved.

 Controlling Output

• psql -o FILENAME or meta command \o FILENAME

will send query output (excluding STDERR) to FILENAME
• \g FILENAME executes the query buffer sending output
to FILENAME
• \watch <seconds> can be used to run previous query
repeatedly

67 © Copyright EnterpriseDB Corporation. All rights reserved.

 Advanced Features - Variables

• psql provides variable substitution

• Variables are simply name/value pairs
• Use \set meta command to set a variable
=> \set city Edmonton
=> \echo :city
Edmonton
• Use \unset to delete a variable
=> \unset city

68 © Copyright EnterpriseDB Corporation. All rights reserved.

 Advanced Features - Special Variables

• Settings can be changed at runtime by altering special variables

• Some important special variables include:
• AUTOCOMMIT, ENCODING, HISTFILE, ON_ERROR_ROLLBACK,
ON_ERROR_STOP, PROMPT1 and VERBOSITY

• Example:
=# \set AUTOCOMMIT off
• Once AUTOCOMMIT is set to off use COMMIT/ROLLBACK to complete the
running transaction

69 © Copyright EnterpriseDB Corporation. All rights reserved.

 Conditional Commands

• Conditional commands primarily helpful for scripting

• \if EXPR begin conditional block
• \elif EXPR alternative within current conditional block
• \else final alternative within current conditional block
• \endif end conditional block

70 © Copyright EnterpriseDB Corporation. All rights reserved.

 Information Commands

• \d[(i|s|t|v|b|S)][+] [pattern]
• List of objects (indexes, sequences, tables, views, tablespaces and
dictionaries)
• \d[+] [pattern]
• Describe structure details of an object
• \l[ist][+]
• Lists of databases in a database cluster

71 © Copyright EnterpriseDB Corporation. All rights reserved.

 Information Commands (continued)

• \dn+ [pattern]
• Lists schemas (namespaces)
• + adds permissions and description to output
• \df[+] [pattern]
• Lists functions
• + adds owner, language, source code and description to
output

72 © Copyright EnterpriseDB Corporation. All rights reserved.

 Common psql Meta Commands
• \q or ^d or quit or exit
• Quits the psql program
• \cd [ directory ]
• Change current working directory
• Tip - To print your current working directory, use \! pwd
• \! [ command ]
• Executes the specified command
• If no command is specified, escapes to a separate Unix shell (CMD.EXE in
Windows)

73 © Copyright EnterpriseDB Corporation. All rights reserved.

 Help
• \conninfo
• Current connection information
• \?
• Shows help information about psql commands
• \h [command]
• Shows information about SQL commands
• If command isn't specified, lists all SQL commands
• psql --help
• Lists command line options for psql

74 © Copyright EnterpriseDB Corporation. All rights reserved.

EDB*Plus
 EDB*Plus

• EDB*Plus is a command line user interface to the EDB Postgres

Advanced Server
• EDB*Plus accepts SQL commands, SPL anonymous blocks, and
EDB*Plus commands
• EDB*Plus commands are compatible with Oracle SQL*Plus
commands
• edb-asXX-edbplus package is available to install EDB*Plus
using yum command and requires java runtime

76 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB*Plus Features

• EDB*Plus can be used for:

• Querying certain database objects
• Executing stored procedures
• Formatting output from SQL commands
• Executing batch scripts
• Executing OS commands
• Recording output

77 © Copyright EnterpriseDB Corporation. All rights reserved.

 Installing EDB*Plus

• You can also use yum package manager for installing EDB*Plus
• EnterpriseDB Repos must be installed prior to installing EDB*Plus
packages
• Login as root or sudo user and run:
• yum install -y edb-edbplus

78 © Copyright EnterpriseDB Corporation. All rights reserved.

 Starting EDB*Plus
• The EDB*Plus program can be invoked by running edbplus from
the edbplus subdirectory
• Syntax:
edbplus:[ -S[ILENT ] ] [ login | /NOLOG ] [ @scriptfile[.ext ] ]
login: username[/password][@{connectstring | variable } ]
connectstring: host[:port][/dbname ] ]

• Example:

79 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB*Plus SET Command
• Sets a session level variable to control certain aspects of EDB*Plus behavior
as follows:
SET AUTO[COMMIT] {ON | OFF | IMMEDIATE | statement_count}
• EDB*Plus always autocommits ddl statements
• IMMEDIATE has the same effect as ON
• statement_count causes EDB*Plus to issue a commit after N successful SQL
statements
• SET ECHO {ON | OFF} Determines whether SQL and EDB*Plus script
statements are shown to the screen as they are executed. The default is OFF
• Type HELP SET in the EDB*Plus terminal to view the entire list of special
variables

80 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB*Plus Commands
• EDB*Plus supports a variety of Oracle SQL*Plus compatible commands
• Example: SQL> ACCEPT city
Enter value for city: Edmonton
SQL> DEFINE city
DEFINE CITY = "Edmonton"

CL[EAR ] [ BUFF[ER ] | SQL | COL[UMNS ] | SCR[EEN ] ]

• CLEAR can be used to clear the buffers, column settings and screen

CON[NECT] username[/password][@{connectstring | variable } ]

• CONNECT can be used to change the database connection

81 © Copyright EnterpriseDB Corporation. All rights reserved.

 List of EDB*Plus Commands

82 © Copyright EnterpriseDB Corporation. All rights reserved.

 Setup Login Variable
• The database login variable can be setup in login.sql
• login.sql can be created in current folder
• Syntax for login.sql file:
define <databasename>=“host:port/databasename”
• Example:

83 © Copyright EnterpriseDB Corporation. All rights reserved.

 Calling a SQL Script

84 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Introduction to psql
• Connecting to Database
• psql Command Line Parameters
• psql Meta-Commands
• Conditional and Information Commands
• EDB*Plus
• Installing and Starting EDB*Plus
• EDB*Plus Commands

85 © Copyright EnterpriseDB Corporation. All rights reserved.

 Prepare Lab Environment

• In the training materials provided by EnterpriseDB there is a script

file edbstore.sql that can be executed using edb-psql to create a
sample edbstore database. Here are the steps:
• Download the edbstore.sql file and place in a directory which is accessible to the
enterprisedb user
• Login as enterprisedb OS user
• Run the edb-psql command with the -f option to execute the edbstore.sql file
and install all the sample objects required for this training
$ edb-psql -p 5444 -f edbstore.sql –d edb –U enterprisedb
• Note - The above command will prompt for edbuser password. The password is edbuser

86 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1
• In this lab exercise you will have a chance to practice what you have learned
through using command line interfaces:

1. Connect to a database using psql 9. Do the same thing, just saving data, not the
2. Switch databases column headers
3. Describe the customers table 10. Create a script via another method, and
4. Describe the customers table including execute from psql
description 11. Turn on the expanded table formatting mode
5. List all databases 12. Lists tables, views and sequences with their
6. List all schemas associated access privileges
7. List all tablespaces 13. Which meta command displays the SQL text
8. Execute a sql statement, saving the output to for a function?
a file 14. View the current working directory

87 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise – 2
1. What is the use of EDB*Plus?
2. Open EDB*Plus and connect to the edb database
3. Write a statement to view the structure of the orders table
4. Configure your EDB*Plus session so that a commit is issued after 10 successful
statements
5. Write a statement to turn on autocommit
6. Write a statement to clear the buffers, column settings and screen
7. Setup a default login variable so that edbplus connects to the EDB Postgres Advanced
Server database named edbstore running on host localhost and port 5444

88 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 5
Database Clusters
 Module Objectives

• Database Clusters
• Creating a Database Cluster
• Starting and Stopping the Server (pg_ctl)
• Connecting to the Server Using psql

90 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Clusters
• A Database Cluster is a collection of databases managed by a single server
instance
• Database Clusters are comprised of:
• Data directory
• Port
• Default databases are created named:
• template0
• template1
• postgres
• edb

91 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating a Database Cluster

• Choose the data directory location for new cluster

• Initialize the database cluster storage area (data directory)
using the initdb utility
• initdb will create the data directory if it doesn’t exist
• You must have permissions on the parent directory so that
initdb can create the data directory
• The data directory can be created manually by superuser and
the ownership can be given to enterprisedb user

92 © Copyright EnterpriseDB Corporation. All rights reserved.

 initdb Utility
$ initdb [OPTION]... [DATADIR]
• Options:
• -D, --pgdata location for this database cluster
• -E, --encoding set default encoding for new databases
• -U, --username database superuser name
• -W, --pwprompt prompt for a password for the new superuser
• -X, --waldir location for the write-ahead log directory
• --wal-segsize size of wal segments , in megabytes
• -k, --data-checksums use data page checksums
• --no-redwood-compat Do not install Oracle compatibility features
• --redwood-like Use Oracle compatible behavior
• -?, --help show this help, then exit

• If the data directory is not specified, the environment variable PGDATA is used

93 © Copyright EnterpriseDB Corporation. All rights reserved.

 Enabling Transparent Data Encryption

Transparent Data Encryption (TDE) can be used Following initdb command options can be used
to encrypt data files, WAL and temporary files to enable TDE:

-y, --data-encryption
--copy-key-from=<file>
--key-wrap-command=<command>
--key-unwrap-command=<command>
--no-key-wrap

94 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - initdb
[root@Base ~]# mkdir /edbstore
[root@Base ~]# chown enterprisedb:enterprisedb /edbstore
[root@Base ~]# su – enterprisedb

[enterprisedb@Base ~]$ initdb -D /edbstore --wal-segsize 1024 -W

• In the above example the • --wal-segsize 1024

database system will be owned
MB specifies the write-ahead
by user enterprisedb
log file segment size
• The enterprisedb user is
the database superuser • -W is used to force initdb to
• The default server config file prompt for the superuser
will be created in /edbstore password
named postgresql.conf
95 © Copyright EnterpriseDB Corporation. All rights reserved.
 pg_ctl Utility

• pg_ctl is a command line utility provided by Postgres to initialize,

start, stop and control a Postgres instance
• It provide options for redirecting start log, controlled startup and
shutdown
• -D option or environmental variable PGDATA can be used to specify
cluster data directory
pg_ctl -D datadir

start stop restart reload status promote init logrotate kill

96 © Copyright EnterpriseDB Corporation. All rights reserved.

 Starting a Database Cluster

• After initializing a database cluster, a unique port must be assigned

• Choose a unique port for postmaster in postgresql.conf
• Start the database cluster using pg_ctl utility
• Example:
[enterprisedb@Base ~]$ vi /edbstore/postgresql.conf
port = 5434

[enterprisedb@Base ~]$ pg_ctl -D /edbstore/ -l /edbstore/startlog start

waiting for server to start.... done
server started

[enterprisedb@Base ~]$ pg_ctl -D /edbstore/ status

pg_ctl: server is running (PID: 62239)

97 © Copyright EnterpriseDB Corporation. All rights reserved.

 Connecting To a Database Cluster

• The edb-psql and PEM clients can be used for connections

[enterprisedb@Base ~]$ edb-psql -p 5434 -d edb -U enterprisedb
Type "help" for help.
edb=# show port;
port
------
5434
(1 row)
edb=# show data_directory;
data_directory
----------------
/edbstore
(1 row)
edb=# \q
[enterprisedb@Base ~]$

98 © Copyright EnterpriseDB Corporation. All rights reserved.

 Reload a Database Cluster

• Some configuration parameter changes do not require a restart

• Changes can be reloaded using the pg_ctl utility
• Changes can also be reloaded using pg_reload_conf()
• Syntax:
$ pg_ctl reload [options]

-D location of the database cluster’s data directory

-s only print errors, no informational messages

99 © Copyright EnterpriseDB Corporation. All rights reserved.

 Stopping a Database Cluster
• pg_ctl supports three modes of shutdown
• smart quit after all clients have disconnected
• Fast quit directly, with proper shutdown (default)
• immediate quit without complete shutdown; will lead to recovery

• Syntax:
$ pg_ctl stop [-W] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-
MODE]
• Example: [enterprisedb@Base ~]$ pg_ctl -D /edbstore/ stop
waiting for server to shut down.... done
server stopped

[enterprisedb@Base ~]$ pg_ctl -D /edbstore/ status

pg_ctl: no server running

100 © Copyright EnterpriseDB Corporation. All rights reserved.

 View Cluster Control Information
• pg_controldata can be used to view the control information for
a database cluster
• It can be run with data directory as an option
[enterprisedb@Base ~]$ pg_controldata /edbstore/
……………………………………………………………………………………………….
Database system identifier: 6724770293870218226
Database cluster state: shut down
Latest checkpoint location: 0/41A3AA40
Latest checkpoint's REDO WAL file: 000000010000000000000001
Latest checkpoint's TimeLineID: 1
Backup start location: 0/0
Backup end location: 0/0
wal_level setting: replica
Database block size: 8192
WAL block size: 8192
Data page checksum version: 0

101 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Database Clusters
• Creating a Database Cluster
• Starting and Stopping the Server (pg_ctl)
• Connecting to the Server Using psql

102 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

1. A new website is to be developed for an online music store.

• Create a new cluster edbdata with ownership of enterprisedb user
• Start your edbdata cluster
• Reload your cluster with pg_ctl utility and using pg_reload_conf()
function
• Stop your edbdata cluster with fast mode

103 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 6
Configuration
 Module Objectives

• Server Parameter File - postgresql.conf

• Viewing and Changing Server Parameters
• Configuration Parameters - Security, Resources and WAL
• Configuration Parameters - Error Logging, Planner and
Maintenance
• Viewing Compilation Settings
• Using File Includes

105 © Copyright EnterpriseDB Corporation. All rights reserved.

 Setting Server Parameters
• There are many configuration parameters that effect the behavior of the database
system
• All parameter names are case-insensitive
• Every parameter takes a value of one of five types:
• boolean
• integer
• floating point
• string
• enum

• One way to set these parameters is to edit the file postgresql.conf, which is normally
kept in the data directory

106 © Copyright EnterpriseDB Corporation. All rights reserved.

 The Server Parameter File - postgresql.conf
• Holds parameters used by a cluster
• Parameters are case-insensitive
• Normally stored in data directory
• initdb installs default copy
• Some parameters only take effect on server restart (pg_ctl restart)
• # used for comments
• One parameter per line
• Use include directive to read and process another file
• Can also be set using the command-line option

107 © Copyright EnterpriseDB Corporation. All rights reserved.

 Viewing and Changing Server Parameters

Configuration parameters can

be viewed using:
• SHOW command Configuration parameters can be
• pg_settings modified for:
• pg_file_settings
• Single session using the SET command
• Database user using ALTER USER
• Single database using ALTER DATABASE

108 © Copyright EnterpriseDB Corporation. All rights reserved.

 Changing Configuration Parameter at Cluster Level

Use ALTER SYSTEM command to edit cluster level

[enterprisedb@pgsrv1 ~] psql edb enterprisedb settings without editing postgresql.conf
edb=# ALTER SYSTEM SET work_mem=20480;
ALTER SYSTEM
edb=# SELECT pg_reload_conf(); ALTER SYSTEM writes new setting to
postgresql.auto.conf file which is read at last during
edb=# ALTER SYSTEM RESET work_mem; server reload/restarts
ALTER SYSTEM
edb=# SELECT pg_reload_conf();
Parameters can be modified using ALTER SYSTEM when
required

109 © Copyright EnterpriseDB Corporation. All rights reserved.

 Connection Settings
• listen_addresses (default *) - Specifies the addresses on which the server is to listen
for connections. Use * for all
• port (default 5444) - The port the server listens on
• max_connections (default 100) - Maximum number of concurrent connections the
server can support
• superuser_reserved_connections (default 3) - Number of connection slots reserved
for superusers
• unix_socket_directory (default /tmp) - Directory to be used for UNIX socket
connections to the server
• unix_socket_permissions (default 0777) - access permissions of the Unix-domain
socket

110 © Copyright EnterpriseDB Corporation. All rights reserved.

 Security and Authentication Settings

• authentication_timeout (default is 1 minute) – Maximum

time to complete client authentication, in seconds
• row_security (default is on) – Controls row security
policy behavior
• password_encryption (default scram-sha-256) –
Determines the algorithm to use to encrypt password
• ssl (default: off) - Enables SSL connections

111 © Copyright EnterpriseDB Corporation. All rights reserved.

 SSL Settings
• ssl_ca_file - Specifies the name of the file containing the SSL server
certificate authority (CA)
• ssl_cert_file - Specifies the name of the file containing the SSL server
certificate
• ssl_key_file - Specifies the name of the file containing the SSL server
private key
• ssl_ciphers - List of SSL ciphers that may be used for secure connections
• ssl_dh_params_file – Specifies file name for custom OpenSSL DH
paramters

112 © Copyright EnterpriseDB Corporation. All rights reserved.

 Memory Settings

maintenance_ autovacuum
shared_buffers temp_buffers work_mem temp_file_limit
work_mem _work_mem

Amount of Amount of
Amount of Amount of
Size of shared memory used memory used Amount of disk
memory used memory used by
buffer pool for a caching sorting and space used for
for maintenance autovacuum
cluster temporary hashing temporary files
commands worker
tables operations

Server Session

113 © Copyright EnterpriseDB Corporation. All rights reserved.

 Dynatune Dynamic Tuning

• Dynatune functionality allows Advanced Server to make

optimal usage of the system resources
• Dynatune automatically configures various resource
parameters at the time of startup
• edb_dynatune (0-100) - This parameter determines how Advanced
Server allocates system resources
• edb_dynatune_profile (oltp | reporting | mixed) - This parameter
controls performance tuning aspects based on the type of work that
the server performs

114 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB Resource Manager

• Prevents any single process from monopolizing resources to

the detriment of other processes
• A resource group can limit the % of cpu or rate of dirty buffer
IO
• Helps in managing multiple kinds of workload on your server
edb_max_resource_groups

This parameter edb_resource_group

controls the
maximum
number of
active resource Set per session to the name of the resource group in EDB Resource Manager
groups

115 © Copyright EnterpriseDB Corporation. All rights reserved.

 Query Planner Settings
• random_page_cost (default 4.0) - Estimated cost of a random page fetch. May
need to be reduced to account for caching effects
• seq_page_cost (default 1.0) - Estimated cost of a sequential page fetch.
• effective_cache_size (default 4GB) - Used to estimate the cost of an index scan.
• enable_hints (default true) - Controls whether Optimizer hints embedded in SQL
commands are utilized or not
• plan_cache_mode (default auto) – Controls custom or generic plan execution for
prepared statements. Can be set to auto, force_custom_plan and
force_generic_plan

116 © Copyright EnterpriseDB Corporation. All rights reserved.

 Write Ahead Log Settings
• wal_level (default replica) - Determines how much information is written to the WAL. Other
values are minimal and logical
• fsync (default on) – Force WAL buffer flush at each commit, Turning this off can cause lead
to arbitrary corruption in case of a system crash
• wal_buffers (default -1, autotune) - The amount of memory used in shared memory for WAL
data. The default setting of -1 selects a size equal to 1/32nd (about 3%) of shared_buffers
• min_wal_size (default 80 MB) – The WAL size to start recycling the WAL files
• max_wal_size (default 1GB) – The WAL size to start checkpoint. Controls the number of WAL
Segments(16MB each) after which checkpoint is forced
• checkpoint_timeout (default 5 minutes) - Maximum time between checkpoints
• wal_compression (default off) – The WAL of Full Page write will be compressed and written

117 © Copyright EnterpriseDB Corporation. All rights reserved.

 Where To Log
log_destination Controls logging type for a database cluster.

Can be set to stderr, csvlog, jsonlog, syslog, and eventlog

logging_collector Enables logger process to capture stderr and csv logging messages

These messages can be redirected based on configuration settings

log_directory - Directory where log files are written

Log File and
log_filename - Format of log file name (e.g. postgresql-%Y-%m-%d_%H%M%S.log)
Directory log_file_mode - permissions for log files
Settings log_rotation_age - Used for file age based log rotation
log_rotation_size - Used for file size based log rotation

118 © Copyright EnterpriseDB Corporation. All rights reserved.

 When To Log
log_min_messages Messages of this severity level or above are sent to the
Duration server log

and log_min_error_statement When a message of this severity or higher is written to

sampling the server log, the statement that caused it is logged
along with it
log_min_duration_statement When a statement runs for at least this long, it is written
to the server log

log_autovacuum_min_duration Logs any Autovacuum activity running for at least this long

log_statement_sample_rate Percentage of queries(above

log_autovacuum_min_duration) to be logged

log_transaction_sample_rate Sample a percentage of transactions by logging

statements

119 © Copyright EnterpriseDB Corporation. All rights reserved.

 What To Log
log_connections Log successful connections to the server log

log_disconnections Log some information each time a session disconnects, including the duration of the session

log_temp_files Log temporary files of this size or larger, in kilobytes

log_checkpoints Causes checkpoints and restart points to be logged in the server log

log_lock_waits Log information if a session is waits longer then deadlock_timeout to acquire a lock

log_error_verbosity How detailed the logged message is. Can be set to default, terse or verbose

log_line_prefix Additional details to log with each line. Default is '%m [%p] ‘ which logs a timestamp and the process ID

log_statement Legal values are none, ddl, mod (DDL and all other data-modifying statements), or all

120 © Copyright EnterpriseDB Corporation. All rights reserved.

 Auditing

• EDB Postgres Advanced Server allows you to track and analyze

database activities using the EDB Audit Logging functionality
• The audit logs can be configured to record information such as:
• When a role establishes a connection to the database
• What database objects are created, modified or deleted by a role
• When any failed authentication attempts have occurred
• The parameters can be specified in the postgresql.conf or
postgresql.auto.conf files

121 © Copyright EnterpriseDB Corporation. All rights reserved.

 Where To Audit
• edb_audit (default none) – enables or disables database auditing. Possible
values are none or xml or csv
• edb_audit_directory (default PGDATA/edb_audit) – specifies where the audit
log files will be created
• edb_audit_destination (default file) - Specifies whether the audit log
information is to be recorded in the directory as given by the
edb_audit_directory parameter or to the directory and file managed by the
syslog process. Set to syslog to use the syslog process and its location as
configured in the /etc/syslog.conf file
• edb_audit_filename (default audit-%Y%m%d_%H%M%S) – file name of the audit
file where the auditing information will be stored

122 © Copyright EnterpriseDB Corporation. All rights reserved.

 What To Audit
• edb_audit_connect (default failed) – enables auditing of database connection
attempts by users. Possible values are none, failed or all
• edb_audit_disconnect (default none) – enables auditing of disconnections by
connected users. Possible values are all or none
• edb_audit_statement (default ddl,error) – specifies auditing of different
categories of SQL statements. Various combinations of these values may be
specified: none, dml, insert, update, delete, truncate, select, error,
rollback, ddl, create, drop, alter, grant, revoke or all
• edb_audit_tag (default none) – specifies a string value that will be included in
the audit log when the edb_audit parameter is set to csv or xml

123 © Copyright EnterpriseDB Corporation. All rights reserved.

 Controlling Audit Trail

• edb_audit_rotation_day (default every) – specifies the day

of the week on which to rotate the audit files. Possible values
are sun,mon,tue,wed,thu,fri,sat,sun,every or
none
• edb_audit_rotation_size (default 0MB) – specifies a file size
threshold in MB when file rotation will be forced to occur
• edb_audit_rotation_seconds (default 0) – specifies the
rotation time in seconds when a new logfile should be created

124 © Copyright EnterpriseDB Corporation. All rights reserved.

 Background Writer Settings

• bgwriter_delay (default 200 ms) - Specifies time between

activity rounds for the background writer
• bgwriter_lru_maxpages (default 100) - Maximum number of
pages that the background writer may clean per activity round
• bgwriter_lru_multiplier (default 2.0) - Multiplier on buffers
scanned per round. By default, if system thinks 10 pages will
be needed, it cleans 10 * bgwriter_lru_multiplier of 2.0 = 20
• Primary tuning technique is to lower bgwriter_delay

125 © Copyright EnterpriseDB Corporation. All rights reserved.

 Statement Behavior

• search_path - This parameter specifies the order in which schemas are searched.
The default value for this parameter is "$user", public
• default_tablespace - Name of the tablespace in which objects are created by
default
• temp_tablespaces - Tablespaces name(s) in which temporary objects are created
• statement_timeout - Postgres will abort any statement that takes over the
specified number of milliseconds A value of zero (the default) turns this off
• idle_in_transaction_session_timeout – Terminates any session with an open
transaction that has been idle for longer than the specified duration in
milliseconds
126 © Copyright EnterpriseDB Corporation. All rights reserved.
 Parallel Query Scan Settings
• Advanced Server supports parallel execution of read-only queries
• Can be enabled and configured by using configuration parameters
• max_parallel_workers_per_gather (default 2): Enables parallel query scan
• parallel_tuple_cost (default 0.1): Estimated cost of transferring one tuple from a parallel worker
process to another process
• parallel_setup_cost (default 1000): Estimates cost of launching parallel worker processes
• min_parallel_table_scan_size (default 8MB): Sets minimum amount of table data that must be
scanned in order for a parallel scan
• min_parallel_index_scan_size (default 512 KB): Sets the minimum amount of index data that must
be scanned in order for a parallel scan
• force_parallel_mode (default off): Useful when testing parallel query scan even when there is no
performance benefit

127 © Copyright EnterpriseDB Corporation. All rights reserved.

 Parallel Maintenance Settings

• PostgreSQL supports parallel processes for creating an index

• Currently this feature is only available for btree index type
• max_parallel_maintenance_workers (default 2): Enables
parallel index creation

128 © Copyright EnterpriseDB Corporation. All rights reserved.

 Vacuum Cost Settings
• vacuum_cost_delay (default 0 ms) - The length of time, in milliseconds, that
the process will wait when the cost limit is exceeded
• vacuum_cost_page_hit (default 1) - The estimated cost of vacuuming a
buffer found in the buffer pool
• vacuum_cost_page_miss (default 10) - The estimated cost of vacuuming a
buffer that must be read into the buffer pool
• vacuum_cost_page_dirty (default 20) - The estimated cost charged when
vacuum modifies a buffer that was previously clean
• vacuum_cost_limit (default 200) - The accumulated cost that will cause the
vacuuming process to sleep

129 © Copyright EnterpriseDB Corporation. All rights reserved.

 Autovacuum Settings
• autovacuum (default on) - Controls whether the autovacuum launcher
runs, and starts worker processes to vacuum and analyze tables
• log_autovacuum_min_duration (default -1) - Autovacuum tasks running
longer than this duration are logged
• autovacuum_max_workers (default 3) - Maximum number of
autovacuum worker processes which may be running at one time
• autovacuum_work_mem (default -1, to use maintenance_work_mem) -
Maximum amount of memory used by each autovacuum worker

130 © Copyright EnterpriseDB Corporation. All rights reserved.

 Just-in-Time Compilation

• Just-in-Time(JIT) is a core feature of EDB Postgres for

accomplishing high performance
• edb-as15-server-llvmjit package must be installed
• JIT in EDB Postgres supports accelerating expression
evaluation and tuple deforming
• JIT configuration parameters:

131 © Copyright EnterpriseDB Corporation. All rights reserved.

 Read-only Parameters

• EDB Postgres sources are compiled using various setting.

• Various read-only configuration parameters can be used to
view build settings
block_size
wal_block_size
segment_size
wal_segment_size
data_checksums
data_directory_mode
server_encoding
lc_collate
max_function_args
max_index_keys
ssl_library

132 © Copyright EnterpriseDB Corporation. All rights reserved.

 Configuration File Includes

• The postgresql.conf file can now contain include

directives
• Allows configuration file to be divided in separate files
• Usage in postgresql.conf file:
• include ‘filename’
• include_dir ‘directory name’

133 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Server Parameter File - postgresql.conf

• Viewing and Changing Server Parameters
• Configuration Parameters - Security, Resources and WAL
• Configuration Parameters - Error Logging, Planner and
Maintenance
• Viewing Compilation Settings
• Using File Includes

134 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

1. You are working as a DBA. It is recommended to keep a

backup copy of the postgresql.conf file before making any
changes. Make the necessary changes in the server
parameter file for the following settings:
• Allow up to 200 connected users on the server
• Reserve 10 connection slots for DBA users on the server
• Maximum time to complete client authentication will be 10 seconds

135 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2

1. Working as a DBA is a challenging job and to track down certain

activities on the database server, logging has to be implemented. Go
through the server parameters that control logging and implement the
following:
• Save all the error message in a file inside the log folder in your cluster data
directory (e.g. c:\edbdata or /edbdata)
• Log all queries which are taking more than 5 seconds to execute, and their
time
• Log the users who are connecting to the database cluster
• Make the above changes and verify them

136 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 3

1. Perform the following changes recommended by a senior

DBA and verify them. Set:
• Shared buffer to 256MB
• Effective cache for indexes to 512MB
• Maintenance memory to 64MB
• Temporary memory to 8MB

137 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 4

1. Vacuuming is an important maintenance activity and needs to

be properly configured. Change the following autovacuum
parameters on the production server. Set:
• Autovacuum workers to 6
• Autovacuum threshold to 100
• Autovacuum scale factor to 0.3
• Auto analyze threshold to 100
• Autovacuum cost limit to 100

138 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 7
Data Dictionary
 Module Objectives

• The System Catalog Schema

• System Information Tables and Views
• System Information and Administration Functions
• Oracle-like Dictionaries
• Oracle Compatible built-in Packages

140 © Copyright EnterpriseDB Corporation. All rights reserved.

 The System Catalog Schema

• Stores information about table and other objects

• Created and maintained automatically in pg_catalog
schema
• pg_catalog is always effectively part of the search_path
• Contains:
• System Tables like pg_class etc.
• System Function like pg_database_size() etc.
• System Views like pg_stat_activity etc.

141 © Copyright EnterpriseDB Corporation. All rights reserved.

 System Information Tables

• \dS in psql prompt will give you the list of pg_* tables and
views
• This list is from pg_catalog schema

pg_tables • list of tables

pg_constraints • list of constraints
pg_indexes • list of indexes
pg_trigger • list of triggers
pg_views • list of views

142 © Copyright EnterpriseDB Corporation. All rights reserved.

 More System Information Tables

• Provides summary of the contents of the server

pg_file_settings configuration file

pg_policy • Stores row level security for tables

• Provides access to useful information about each row-

pg_policies level security policy in the database

143 © Copyright EnterpriseDB Corporation. All rights reserved.

 System Information Functions

current_database() current_schema[()] pg_postmaster_start_time() version()

current_user current_schemas(boolean) pg_current_logfile() txid_status()

pg_conf_load_time() pg_jit_available()

144 © Copyright EnterpriseDB Corporation. All rights reserved.

 System Administration Functions
current_setting, set_config • Return or modify configuration variables

pg_cancel_backend • Cancel a backend's current query

pg_terminate_backend • Terminates backend process

pg_reload_conf • Reload configuration files

pg_rotate_logfile • Rotate the server's log file

pg_start_backup, pg_stop_backup • Used with point-in time recovery

pg_ls_logdir() • Returns the name, size, and last modified time of each file in the log directory

pg_ls_waldir() • Returns the name, size, and last modified time of each file in the WAL directory

145 © Copyright EnterpriseDB Corporation. All rights reserved.

 More System Administration Functions
• Disk space used by a tablespace, database, relation or
pg_*_size total_relation (includes indexes and toasted data)

pg_column_size • Bytes used to store a particular value

pg_size_pretty • Convert a raw size to something more human-readable

• File operation functions. Restricted to superuser use and only

pg_ls_dir, pg_read_file on files in the data or log directories

pg_blocking_pids() • Function to reliably identify which sessions block others

146 © Copyright EnterpriseDB Corporation. All rights reserved.

 System Information Views
pg_stat_activity • details of open connections and running transactions

pg_locks • list of current locks being held

pg_stat_database • details of databases

pg_stat_user_* • details of tables, indexes and functions

pg_stat_archiver • status of the archiver process

pg_stat_progress_basebackup • view pg_basebackup progress

pg_stat_progress_vacuum • provides progress reporting for VACUUM operations

pg_stat_progress_analyze • provides progress details for ANALYZE operations

• provides a summary of the contents of the client authentication
pg_hba_file_rules
configuration file, pg_hba.conf

147 © Copyright EnterpriseDB Corporation. All rights reserved.

 Built-in Packages
• EDB Postgres Advanced Server provides built-in packages compatible with Oracle
• Over 24 of most commonly used Oracle built-in packages are available in EDB Postgres
Advanced Server
• These built-in packages provide administration and maintenance utilities
DBMS_ALERT DBMS_AQ DBMS_AQADM DBMS_CRYPTO DBMS_JOB

DBMS_LOB DBMS_LOCK DBMS_MVIEW DBMS_OUTPUT DBMS_PIPE

DBMS_PROFILER DBMS_RANDOM DBMS_RLS DBMS_SESSION DBMS_SCHEDULER

DBMS_SQL DBMS_UTILITY DBMS_REDACT UTL_ENCODE UTL_FILE

UTL_HTTP UTL_MAIL UTL_SMTP UTL_URL UTL_RAW

148 © Copyright EnterpriseDB Corporation. All rights reserved.

 System Catalog Views
• EDB Postgres Advanced Server database stores user data in user tables
• EDB Postgres Advanced Server database stores internal information about all the
objects and events in the catalog tables
• System catalog tables are kept in separate internal schemas
• EDB Postgres Advanced Server provides Oracle compatible catalog view (data
dictionaries)
• These views can be queried to find:
• Definition of schema objects
• Users information
• Privileges
• Other general database information

149 © Copyright EnterpriseDB Corporation. All rights reserved.

 sys Schema

• sys schema contains Oracle compatible Catalog Views

View Description
user_* User’s view (what is in your
schema; what you own)
all_* Expanded user’s view (what you
can access)
dba_* Database administrator’s view
(what is in everyone’s schemas)

edb$ Performance-related data

150 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• The System Catalog Schema

• System Information Tables and Views
• System Information and Administration Functions
• Oracle-like Dictionaries
• Oracle Compatible built-in Packages

151 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

1. You are working with different schemas in a database.

After a while you need to determine all the schemas in
your search path. Write a query to find the list of schemas
currently in your search path.

152 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2

1. You need to determine the names and definitions of all of

the views in your schema. Create a report that retrieves
view information - the view name and definition text.

153 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 3

1. Create a report of all the users who are currently

connected. The report must display total session time of
all connected users.

2. You found that a user has connected to the server for a

very long time and have decided to gracefully kill its
connection. Write a statement to perform this task.

154 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 4

1. Write a query to display the name and size of all the

databases in your cluster. Size must be displayed using a
meaningful unit.

155 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 8
Creating and Managing Databases
 Module Objectives

• Object Hierarchy
• Users and Roles
• Tablespaces
• Databases
• Access Control
• Creating Schemas
• Schema Search Path

157 © Copyright EnterpriseDB Corporation. All rights reserved.

 Object Hierarchy
Database
Cluster

Users/Groups
Database Tablespaces
(Roles)

Catalogs Schema Extensions

Table View Sequence Functions Event Triggers

158 © Copyright EnterpriseDB Corporation. All rights reserved.

Users and Roles
 Database Users

• Are global across a database cluster

• Are not the operating system users
• Are used for connecting to a database
• Have a unique name not starting with pg_
• enterprisedb is a predefined superuser

160 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating Users Using psql

• How to create? CREATE USER sql command

• How to delete? DROP USER sql command
• superuser or createrole privilege is required for creating
a database user
Syntax: Example:
CREATE USER name [ [ WITH ] option [ ... ] ]
where option can be:
SUPERUSER | CREATEDB | CREATEROLE | LOGIN |
REPLICATION | BYPASSRLS CONNECTION LIMIT
connlimit
| PASSWORD 'password'| VALID UNTIL 'timestamp'
| PROFILE profile_name
| ACCOUNT { LOCK | UNLOCK }
| LOCK TIME 'timestamp'
| PASSWORD EXPIRE [ AT 'timestamp' ]

161 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating Users Using createuser

• The createuser utility can also be used to create a user

• Syntax:
$ createuser [OPTION]... [ROLENAME]
• Use --help option to view the full list of options available
• Example:

162 © Copyright EnterpriseDB Corporation. All rights reserved.

 User Profile Management

• A profile is a named set of • Password attributes:

• FAILED_LOGIN_ATTEMPTS
password attributes
• PASSWORD_LOCK_TIME
• User profiles can be used to • PASSWORD_LIFE_TIME
• PASSWORD_GRACE_TIME
manage account status and
• PASSWORD_REUSE_TIME
password expiration • PASSWORD_REUSE_MAX
• PASSWORD_VERIFY_FUNCTION
• The default profile is
• PASSWORD_ALLOW_HASHED
assigned to all users

163 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating A User Profile

• How to create? CREATE PROFILE command

• How to delete? DROP PROFILE command
• Example:

164 © Copyright EnterpriseDB Corporation. All rights reserved.

 Roles

• Role is a collection of cluster and object level privileges

• Role makes it easier to manage multiple privileges
• How to create? CREATE ROLE statement
• How to assign? GRANT statement
• Who it can be assigned to? user or a group

165 © Copyright EnterpriseDB Corporation. All rights reserved.

 Default Roles
aq_administrator_role
pg_checkpoint • Provide certain
pg_database_owner
pg_execute_server_program
administrative capabilities
pg_monitor using these default roles
pg_read_all_data
pg_read_all_settings • For Example, create a new
pg_read_all_stats
pg_read_server_files
user with read only access
pg_signal_backend or a new user with access to
view monitoring data only
pg_stat_scan_tables
pg_write_all_data
pg_write_server_files

166 © Copyright EnterpriseDB Corporation. All rights reserved.

Tablespaces
 Tablespaces and Data Files
• Data is stored logically in tablespaces and physically in data files
• Tablespaces:
• Can belong to only one database cluster
• Consist of multiple data files
• Can be used by multiple databases
• Data Files:
• Can belong to only one tablespace
• Are used to store database objects
• Cannot be shared by multiple tables (one or more per table)

168 © Copyright EnterpriseDB Corporation. All rights reserved.

 Advantages of Tablespaces

• Control the disk layout for a database cluster

• Store indexes and data physically separated for performance

Indexes
Tablespace A
Database Instance

Fast Storage
Transactional Tables

Historical Tables
Tablespace B
Slow Storage
Seldom Used Partition

169 © Copyright EnterpriseDB Corporation. All rights reserved.

 Pre-Configured Tablespaces

PGDATA/global directory
Database Instance pg_global
Tablespace
Cluster-wide tables and catalog objects

PGDATA/base directory
pg_default
Tablespace
Databases, schemas and other objects

170 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating Tablespaces

Tablespace Physical Cluster Data

• How to create? CREATE Directory Directory

TABLESPACE command
• The tablespace directory must Directory(Database
pg_tblspc
Catalogue Version)
be existing with permissions
• Syntax: Database Directory
Symbolic
Link(Tablespace
for each Database
• CREATE TABLESPACE OID)
tablespace_name [ OWNER
user_name ]
LOCATION 'directory‘; Database
Objects(Files)

171 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - CREATE TABLESPACE
[training@Base ~]$ sudo mkdir /newtab1
[training@Base ~]$ sudo chown enterprisedb:enterprisedb /newtab1
[training@Base ~]$ su - enterprisedb
[enterprisedb@Base ~]$ psql -p 5444 edb enterprisedb

edb=# CREATE TABLESPACE fast_tab LOCATION '/newtab1';

CREATE TABLESPACE
edb=# \db
List of tablespaces
Name | Owner | Location
------------+--------------+----------
fast_tab | enterprisedb | /newtab1
pg_default | enterprisedb |
pg_global | enterprisedb |
(3 rows)

172 © Copyright EnterpriseDB Corporation. All rights reserved.

 Using Tablespaces

• Use the TABLESPACE keyword while creating databases,

tables and indexes
edb=# CREATE TABLE account(acno NUMBER PRIMARY KEY,
ac_hldr_fname VARCHAR2(20)) TABLESPACE fast_tab;
CREATE TABLE

173 © Copyright EnterpriseDB Corporation. All rights reserved.

 Default and Temp Tablespace
• default_tablespace server parameter sets default tablespace
• default_tablespace parameter can also be set using the SET command at the session
level
• temp_tablespaces parameter determines the placement of temporary tables and
indexes and temporary files
• temp_tablespaces can be a list of tablespace names
edb=# show default_tablespace;
default_tablespace
--------------------

(1 row)
edb=# show temp_tablespaces;
temp_tablespaces
------------------

(1 row)

174 © Copyright EnterpriseDB Corporation. All rights reserved.

 Altering Tablespaces

• ALTER TABLESPACE can be used to rename a

tablespace, change ownership and set a custom value for
a configuration parameter
• Only the owner or superuser can alter a tablespace
• The seq_page_cost and random_page_cost
parameters can be altered for a tablespace

175 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Alter Tablespace
Syntax:
ALTER TABLESPACE name RENAME TO new_name
ALTER TABLESPACE name OWNER TO { new_owner | CURRENT_USER | SESSION_USER }
ALTER TABLESPACE name SET ( tablespace_option = value [, ... ] )
ALTER TABLESPACE name RESET ( tablespace_option [, ... ] )

edb=# ALTER TABLESPACE fast_tab RENAME TO new_tab;

ALTER TABLESPACE
edb=# \db
List of tablespaces
Name | Owner | Location
------------+--------------+----------
new_tab | enterprisedb | /newtab
pg_default | enterprisedb |
pg_global | enterprisedb |

176 © Copyright EnterpriseDB Corporation. All rights reserved.

 Dropping a Tablespace

• DROP TABLESPACE removes a tablespace from the system

• Only the owner or superuser can drop a tablespace
• The tablespace must be empty
• If a tablespace is listed in the temp_tablespaces
parameter, make sure current sessions are not using the
tablespace
• DROP TABLESPACE cannot be executed inside a transaction

177 © Copyright EnterpriseDB Corporation. All rights reserved.

Databases
 What Is a Database?

• A database is a named collection of SQL objects

• A running Postgres instance can manage multiple
databases
• How to create? CREATE DATABASE command
• How to delete? DROP DATABASE command
• To determine the set of existing databases:
• SQL - SELECT datname FROM pg_database;
• psql META COMMAND - \l (backslash lowercase L)

179 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating Databases

• Database can be created using:

1. createdb utility program
2. CREATE DATABASE SQL command
• SQL Command syntax:
CREATE DATABASE name [ [ WITH ] [ OWNER [=] user_name ]
[ TEMPLATE [=] template ]
[ ENCODING [=] encoding ]
[ TABLESPACE [=] tablespace_name ]
[ ALLOW_CONNECTIONS [=] allowconn ]
[ CONNECTION LIMIT [=] connlimit ]

180 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Creating Databases

181 © Copyright EnterpriseDB Corporation. All rights reserved.

 Accessing a Database

• PEM Web Client or psql can be used to access a database

• To use psql, open a terminal and execute:
$ psql –U postgres –d prod

Note: If PATH is not set you can execute psql command from the bin directory
of postgres installation

182 © Copyright EnterpriseDB Corporation. All rights reserved.

 Privileges

• Cluster level
• Granted to a user during CREATE or later using ALTER USER
• These privileges are granted by superuser
• Object Level
• Granted to user using GRANT command
• These privileges allow a user to perform particular actions on a database
object, such as tables, views, or sequence
• Can be granted by owner, superuser or someone who has been given
permission to grant privileges (WITH GRANT OPTION)

183 © Copyright EnterpriseDB Corporation. All rights reserved.

 GRANT Statement

• Grants object level privileges to database users, groups or

roles
• GRANT can also be used to grant a role to a user
• How to view syntax and available privileges?
• Type \h GRANT in psql

184 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – GRANT Statement

185 © Copyright EnterpriseDB Corporation. All rights reserved.

 REVOKE Statement

• Revokes object level privileges from database users,

groups or roles
• REVOKE [ GRANT OPTION FOR ] can be used to
revoke only the grant option without revoking the actual
privilege
• How to view syntax and available privileges?
• Type \h REVOKE in psql

186 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - REVOKE Statement

187 © Copyright EnterpriseDB Corporation. All rights reserved.

Database Schemas
 What is a Schema
SCHEMA

Tables Views

Sequences Functions Owns

USER
Domains

189 © Copyright EnterpriseDB Corporation. All rights reserved.

 Benefits of Schemas

• A database can contain one or more named schemas

• By default, all databases contain a public schema
• There are several reasons why one might want to use schemas:
• To allow many users to use one database without interfering with each other
• To organize database objects into logical groups to make them more
manageable
• Third-party applications can be put into separate schemas so they cannot
collide with the names of other objects

190 © Copyright EnterpriseDB Corporation. All rights reserved.

 Creating Schemas
• Schemas can be added using the CREATE SCHEMA SQL command
• Syntax:
CREATE SCHEMA IF NOT EXISTS schema_name [ AUTHORIZATION
role_specification ]
• Example:

191 © Copyright EnterpriseDB Corporation. All rights reserved.

 What is a Schema Search Path

• The schema search path determines which schemas are

searched for matching table names
• Search path is used when fully qualified object names are
not used in a query
• Example:
 SELECT * FROM employee;

This statement will find the first employee table from the schemas listed in the search path

192 © Copyright EnterpriseDB Corporation. All rights reserved.

 Determine the Schema Search Path

• To show the current search path, use the following

command:
• => SHOW search_path;

• Default search_path is "$user",public

• Search path can be changed using SET command:
• => SET search_path TO myschema, public;

193 © Copyright EnterpriseDB Corporation. All rights reserved.

 Object Ownership

Database
Cluster
Owner

Users/Groups
Database Tablespaces
(Roles)

Catalogs Schema Extensions

Event
Table View Sequence Functions
Triggers

194 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Object Hierarchy
• Users and Roles
• Tablespaces
• Databases
• Access Control
• Creating Schemas
• Schema Search Path

195 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

1. An e-music online store website application developer wants to add an

online buy/sell facility and has asked you to separate all tables used in
online transactions. Here you have suggested to use schemas.
Implement the following suggested options:
• Create an ebuy user with password ‘lion’
• Create an ebuy schema which can be used by user ebuy
• Login as the ebuy user, create a table sample1 and check whether that
table belongs to the ebuy schema or not

196 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2

1. Retrieve a list of databases using a SQL query

2. Retrieve a list of databases using the psql meta command
3. Retrieve a list of tables in the edbstore database and check which
schema and owner they have

197 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 9
Database Security
 Module Objectives

• Database Security Requirements and Protection Plan

• Levels of Security in Postgres
• Access Control using pg_hba.conf
• Introduction to Row Level Security
• Data Encryption
• Advanced Features - Data Redaction, SQL/Protect and EDB*Wrap
• General Security Recommendations

199 © Copyright EnterpriseDB Corporation. All rights reserved.

 Why Database Security

• Databases are a core component of many computing systems

• Confidential data like SIN(Social Insurnace Number),
Healthcare, Banking details is stored and shared using
databases
• It is very critical to protect stored information from hackers,
insiders, and other groups who intend to steal valuable data
• Database Security is a mechanism to protect the data against
threats

200
200 © Copyright EnterpriseDB Corporation. All rights reserved.
 Data Security Requirements

• Stopping improper disclosure, modification and denial of

access to information is very important
• Who wants an employee finding out boss’s salary, changing
his/her salary or stopping HR from printing paychecks
• Database Security Requirements includes:
• Confidentiality
• Integrity
• Availability

201 © Copyright EnterpriseDB Corporation. All rights reserved.

 Protection Plan – We all need one

• Access Control
Prevent
• Authentication and Authorization

• Data Control
• Views, Row Level Security,
Encryptions

• Network Control Attacks

• SSL Connections, Firewalls
Protect Discover
• Auditing
• Monitoring

202 © Copyright EnterpriseDB Corporation. All rights reserved.

 Levels of Security

Server and • Check Client IP

Application • pg_hba.conf

• User/Password
Database • Connect Privilege
• Schema Permissions

• Table Level Privileges

Object
• Grant/Revoke

203 © Copyright EnterpriseDB Corporation. All rights reserved.

Host Based Access Control
 Host Based Access Control
pg_hba.conf

Postmaster
Client
IP: 10.8.99.30
User: appuser1

• pg_hba.conf can be used to restrict the ability to connect to a database

• SSL can be forced for selected clients based on hostname or IP address
• Different authentication methods can be used
• Superuser access can be locked down to certain IPs using pg_hba.conf

205 © Copyright EnterpriseDB Corporation. All rights reserved.

 pg_hba.conf - Access Control
• Host based access control file
• Located in the cluster data directory
• Read at startup, any change requires reload
• Contains a set of records, one per line
• Each record specifies a connection type, database name, user name, client IP and method of
authentication
• Top to bottom read
• Hostnames, IPv6 and IPv4 supported
• Authentication methods - trust, reject, md5, password, gss, sspi, krb5, ident, peer, pam, ldap,
radius, bsd, scram or cert

206 © Copyright EnterpriseDB Corporation. All rights reserved.

 pg_hba.conf Example
# TYPE DATABASE USER ADDRESS METHOD

# "local" is for Unix domain socket connections only

local all all peer
# IPv4 local connections:
host all all 127.0.0.1/32 ident
# IPv6 local connections:
host all all ::1/128 ident
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all peer
host replication all 127.0.0.1/32 ident
host replication all ::1/128 ident

207 © Copyright EnterpriseDB Corporation. All rights reserved.

 Authentication Problems
FATAL: no pg_hba.conf entry for host "192.168.10.23", user “edbstore", database “edbuser“
FATAL: password authentication failed for user “edbuser“
FATAL: user “edbuser" does not exist
FATAL: database “edbstore" does not exist

• Self-explanatory message is displayed

• Verify database name, username and Client IP in pg_hba.conf
• Reload Cluster after changing pg_hba.conf
• Check server log for more information

208 © Copyright EnterpriseDB Corporation. All rights reserved.

Row Level Security
 Row Level Security (RLS)

• GRANT and REVOKE can be used at table level

• PostgreSQL supports security policies for limiting
access at row level
• By default, all rows of a table are visible
• Once RLS is enabled on a table, all queries must
go through the security policy
• Security policies are controlled by DBA rather
than application
• RLS offers stronger security as it is enforced by
the database

210 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Row Level Security
• For example, to enable row level security for the table accounts :
• Create the table first
edb=# CREATE TABLE accounts (manager text, company text, contact_email text);

• Then alter the table

edb=# ALTER TABLE accounts ENABLE ROW LEVEL SECURITY;

• Syntax:
CREATE POLICY name ON table_name
[ AS { PERMISSIVE | RESTRICTIVE } ]
[ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ]
[ TO{ role_name | PUBLIC | CURRENT_USER | SESSION_USER}[,...] ]
[ USING ( using_expression ) ]
[ WITH CHECK ( check_expression ) ]

211 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Row Level Security (continued)

• To create a policy on the accounts table to allow the

managers role to view the rows of their accounts, the
CREATE POLICY command can be used:
edb=# CREATE POLICY account_managers ON accounts TO managers USING
(manager = current_user);

• To allow all users to view their own row in a user table, a

simple policy can be used:
edb=# CREATE POLICY user_policy ON users USING (user =
current_user);

212 © Copyright EnterpriseDB Corporation. All rights reserved.

Data Encryption
 Transparent Data Encryption

• Transparent Data Encryption was introduced in EDB Postgres Advanced

Server version 15
• Helps securing user data by encrypting data files, write-ahead logs and
temporary files
• TDE is transparent to user and doesn’t require any application-level change
• With TDE, Database Server and backup storage files are unintelligible for
unauthorized users
• Does not encrypt transaction status metadata, data directory structure,
foreign tables, logs and configuration files
• General overhead of AES is expected

214 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Level Encryption

• Encrypting everything does not make data secure

• Resources are consumed when you query encrypted data
• pgcrypto provides mechanism for encrypting selected
columns
• pgcrypto supports one-way and two-way data encryption
• Install pgcrypto using CREATE EXTENSION command
• CREATE EXTENSION pgcrypto;

215 © Copyright EnterpriseDB Corporation. All rights reserved.

Added Security in EDB
Postgres Advanced Server
 Added Security in EDB Postgres Advanced Server

o EDB Postgres Advanced Server supports all the security

options available in PostgreSQL, plus:
• Transparent Data Encryption
• User Profiles and Password Policy Manager
• SQL/Protect - Protection against SQL Injections
• DBMS_CRYPTO, DBMS_RLS, DBMS_REDACT
• Data redaction
• EDB*Wrap - Obfuscate Source Code
• Enhanced Auditing using edb_audit

217 © Copyright EnterpriseDB Corporation. All rights reserved.

 Data Redaction

• Data Redaction can be used to conceal

data values from selected users
• The redaction function is incorporated
into redaction policy using CREATE
REDACTION POLICY
• Data Redaction is controlled by
edb_data_redaction
configuration parameter
• Useful for compliance to GDPR, PCI
and HIPAA standards

218 © Copyright EnterpriseDB Corporation. All rights reserved.

 SQL/PROTECT – Track, Warn and Protect

219 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB*Wrap – Protects Your Code

• Safeguards sensitive code from

prying eyes inside your firewall
• Protects critical algorithms,
processes, seed values and more
• Restricts access to intellectual
property on customer sites
• Additional layer of security
beyond standard user ACLs

220 © Copyright EnterpriseDB Corporation. All rights reserved.

General Security
Recommendations
 General Recommendations - Database Server

• Always keep your system patched to the latest version

• Don't put a postmaster port on the Internet
• Firewall this port appropriately
• If that's not possible, make a read-only Replica database available on the port, not
a R/W master
• Isolate the database port from other network traffic
• Don't rely solely on your front-end application to prevent
unauthorized access to your database
• Avoid using trust authentication in pg_hba.conf

222 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Recommendations - Database Users

• Provide each user with their own login

Shared credentials make auditing more complicated and violate
HIPAA, PCI, etc.
• Allow users the minimum access to do their jobs
• Use Roles and classes of privileges
• Use Views and View Security Barriers
• Use Row Level Security

223 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Recommendations - Connection Pooling

• When not practical to provide each user with their own

login (i.e. connection pooling is in use):
• Have one or more logins related to the application
• Limit access to the database by the specific IP addresses where
the application is certified to run
• Ensure the login(s) have minimum rights needed to do their work
(e.g. SELECT rights and only to specified tables)

224 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Recommendations - Database Superuser

• Only allow the database superuser to log in from the server

machine itself, with local or localhost connection
• Reserve use of superuser accounts for tasks or roles where it is
absolutely required
• Make as few objects owned by the superuser as necessary
• Restrict access to configuration files (postgresql.conf and
pg_hba.conf) and error log files to administrators
• Disallow host system login by database superuser roles
('postgres‘)

225 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Recommendations - Database Superuser
(continued)
• Do not allow superuser to log into database server OS.
Use personal OS login and then “sudo” to create an audit
trail
• Use a separate database login to own each database and
own everything in it

226 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Recommendations - Database Backups

• Keep backups and have a tested recovery plan. No matter how

well you secure things, it's still possible an intruder could get
in and delete or modify your data
• Have scripts perform backups and immediately test them and
alert DBA on any failures
• Keep backups physically separate from the database server. A
disaster can strike and take out an entire location, whether
that’s environmental (e.g. earthquake), malicious (e.g. hacker,
insider), or human error

227 © Copyright EnterpriseDB Corporation. All rights reserved.

 General Recommendations - Think AAA

• Authenticate - verify the user is who she claims to be

• Authorize - verify the user is allowed access
• Audit - record which user did what and when they did it

228 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Database Security Requirements and Protection Plan

• Levels of Security in Postgres
• Access Control using pg_hba.conf
• Introduction to Row Level Security
• Data Encryption
• Advanced Features - Data Redaction, SQL/Protect and EDB*Wrap
• General Security Recommendations

229 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

1. You are working as an EDB Postgres Advanced Server DBA. Your server
box has 2 network cards with ip addresses 192.168.30.10 and
10.4.2.10. 192.168.30.10 is used for the internal LAN and 10.4.2.10 is
used by the web server to connect users from an external network.
Your server should accept TCP/IP connections both from internal and
external users.
• Configure your server to accept connections from external and internal
networks.

230 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2

1. You are working as an EDB Postgres Advanced Server DBA. A developer

showed you the following error:
psql: could not connect to server: Connection refused
(0x0000274D/10061)

• Is the server running on host 192.168.30.22 and accepting

TCP/IP connections on port 5444?

2. Diagnose the problem and suggest the solution

231 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 3

1. A new developer has joined the team with ID number 89

• Create a new user by name dev89 and password password89
• Then assign the necessary privileges to dev89 so they can connect to the
edbstore database and view all tables

232 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 4
1. A new developer joins e-music corp. They have an ip address 192.168.30.89.
They are not able to connect from their machine to the EDB Postgres
Advanced Server and gets the following error on the server:

FATAL: no pg_hba.conf entry for host “192.168.30.89", user

“dev89", database “edbstore", SSL off

2. Configure your server so that the new developer can connect from their
machine

233 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 10
Monitoring and Admin Tools
 Module Objectives

• Overview and Features of EDB Postgres Enterprise Manager

• Access EDB Postgres Enterprise Manager Client
• Register and Connect to a Database Server
• General Database Administration
• Object Browser - View Data, Query Tool, Server Status
• Overview of pgAdmin

235 © Copyright EnterpriseDB Corporation. All rights reserved.

 Postgres Enterprise Manager

Manage, monitor, and tune Postgres at scale

Manage from one Optimize database Monitor system Integrate with other
interface performance health tools

One place to visualize In-depth diagnostics for Built-in dashboards and APIs and webhooks to
and manage everything database reports and customizable alert fetch data, send alerts,
tuning thresholds and manage servers

236 © Copyright EnterpriseDB Corporation. All rights reserved.

 PEM - Features
Manage, Monitor and Tune PostgreSQL and EDB Postgres Advanced Server running on multiple Platforms

Management Monitoring Tuning

• Integrated SQL IDE • Customizable charts • Detailed performance

and dashboards
• Built-in query debugger diagnostics
• Predefined and custom
• User/group access alerts via email or • SQL profiler
management SNMP
• Capacity management
• Schema Diff • User-defined metrics
log analysis • Audit log manager
• Session profiling
• Database and OS level
• Expert wizards for
• Job scheduling monitoring
configuration setup
• Backup and failover • Web hooks and REST
management API for integrations

237 © Copyright EnterpriseDB Corporation. All rights reserved.

 PEM Architecture
PEM Web Application PostgreSQL EPAS

HTTPD

Monitoring Data
PEM Agent PEM Agent

Managed Host Machine

Client(Browser)
Monitoring Data
EPAS

PEM Storage
(Backend Database: pem) Monitoring Data PEM Agent

Managed Host Machine

PEM Server Host Machine

238 © Copyright EnterpriseDB Corporation. All rights reserved.

 Install and Configure PEM

• PEM Server can be installed using RPM or yum package

manager :
• yum install edb-pem
• After installation, PEM Server can be configured using
configure-pem-server.sh script file
• /usr/edb/pem/bin/configure-pem-server.sh
• PEM installation is covered in detail in our PEM Monitoring
Course.

239 © Copyright EnterpriseDB Corporation. All rights reserved.

 Open Postgres Enterprise Manager Client
• Postgres Enterprise Manager (PEM) Client can be run using the a
supported web browser on your OS

240 © Copyright EnterpriseDB Corporation. All rights reserved.

 PEM Client First Look

241 © Copyright EnterpriseDB Corporation. All rights reserved.

 Register a Database Cluster
• Go to PEM Server Directory
• Right Click on PEM Enterprise Manager Server and select Create Server

242 © Copyright EnterpriseDB Corporation. All rights reserved.

 Common Connection Problems

• There are 2 common error messages encountered when

connecting to an EDB Postgres Advanced Server database:
• Could not connect to Server - Connection refused
• This error occurs when either the database server isn't running OR the server isn't
configured to accept external TCP/IP connections
• FATAL - no pg_hba.conf entry
• This means your server can be contacted over the network, but is not configured to
accept the connection. Your client is not detected as a legal user for the database. You
will have to add an entry for each of your clients to the pg_hba.conf file

243 © Copyright EnterpriseDB Corporation. All rights reserved.

 Changing a Server’s Registration

• Right-click on a server entry

to modify its properties
• Click on the Delete/Drop to
remove a server’s entry

244 © Copyright EnterpriseDB Corporation. All rights reserved.

 Viewing Data

• Expand Databases 
Schemas  Tables
• Right-click on a table
• Select View Data

245 © Copyright EnterpriseDB Corporation. All rights reserved.

 Filtering and Sorting Data

Filter/Sort Data

246 © Copyright EnterpriseDB Corporation. All rights reserved.

 Query Tool

247 © Copyright EnterpriseDB Corporation. All rights reserved.

 Databases

• Create a new database or run a

report on the databases in a cluster with
the databases menu
• Perform the following option with an
individual database menu:
• Create a new object in the database
• Drop the database
• Open the Query Tool with a script to re-create the
database
• Run reports
• Perform maintenance
• Backup or Restore
• Modify the databases properties

248 © Copyright EnterpriseDB Corporation. All rights reserved.

 Schemas

249 © Copyright EnterpriseDB Corporation. All rights reserved.

 Tables

250 © Copyright EnterpriseDB Corporation. All rights reserved.

 Tables - Maintenance

251 © Copyright EnterpriseDB Corporation. All rights reserved.

 Tablespaces

252 © Copyright EnterpriseDB Corporation. All rights reserved.

 Roles

253 © Copyright EnterpriseDB Corporation. All rights reserved.

 Server Status

254 © Copyright EnterpriseDB Corporation. All rights reserved.

Overview of pgAdmin
 Introduction to pgAdmin

• Open source graphical user interface for Postgres

• Create, manage and maintain database objects
• pgAdmin is web based and requires Apache HTTP server
• Download and Install:
https://www.pgadmin.org/download/

256 © Copyright EnterpriseDB Corporation. All rights reserved.

 pgAdmin Features
Multi-platform
Supports PostgreSQL and EDB Postgres Advanced Server
Multi-deployment Mode – Desktop, Server
Integrated SQL IDE
pl/pgsql and edb-spl Debugger
Schema Diff Tool
ERD Tool
Perform Maintenance Tasks- Vacuum, Backups, Restore etc.
Job Scheduler
Multibyte server-side encoding support

257 © Copyright EnterpriseDB Corporation. All rights reserved.

 First Look - pgAdmin

258 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Overview and Features of EDB Postgres Enterprise Manager

• Access EDB Postgres Enterprise Manager Client
• Register and Connect to a Database Server
• General Database Administration
• Object Browser - View Data, Query Tool, Server Status
• Overview of pgAdmin

259 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 11
SQL Primer
 Module Objectives

• Data Types • Sequences

• Structured Query Language • Domains

(SQL) • SQL Joins and Functions

• DDL, DML and DCL Statements • Explain Plans

• Quoting in PostgreSQL
• Transaction Control Statements
• Indexes
• Tables and Constraints
• Oracle Compatibility and Tools
• Views and Materialized Views

261 © Copyright EnterpriseDB Corporation. All rights reserved.

 Data Types

• Common Data Types:

Numeric Character Date/Time Other Advanced
Types Types Types Types Server
TIMESTAMP BYTEA CLOB
NUMERIC CHAR
BOOL
BLOB
DATE MONEY
INTEGER VARCHAR VARCHAR2
TIME XML
NUMBER
JSON
SERIAL TEXT INTERVAL JSONB XMLTYPE

262 © Copyright EnterpriseDB Corporation. All rights reserved.

 Oracle Compatible Data Types
• The built-in general-purpose data types:

BLOB BOOLEAN CHAR CLOB DATE DOUBLE

BINARY VARBINARY INTEGER NUMBER TIMESTAMP VARCHAR2

NVARCHAR2 ROWID INTERVAL XML

263 © Copyright EnterpriseDB Corporation. All rights reserved.

 Structured Query Language

Data Definition Data Manipulation Data Control Transaction

Language Language Language Control Language
• CREATE • INSERT • GRANT • COMMIT
• ALTER • UPDATE • REVOKE • ROLLBACK
• DROP • DELETE • SAVEPOINT
• TRUNCATE • SET
TRANSACTION

264 © Copyright EnterpriseDB Corporation. All rights reserved.

 DDL Statements
Statement Syntax
CREATE TABLE CREATE [TEMPORARY][UNLOGGED] TABLE table_name
( [column_name data_type [ column_constraint] )
[ INHERITS ( parent_table) ]
[ TABLESPACE tablespace_name ]
[ USING INDEX TABLESPACE tablespace_name ]
[ PARTITION BY { RANGE | LIST | HASH } (column_name|( expression) ]

ALTER TABLE ALTER TABLE [IF EXISTS] [ONLY] name [*] action [,…]

DROP TABLE DROP TABLE [ IF EXISTS ] name [, …] [ CASCADE | RESTRICT ]

TRUNCATE TRUNCATE [ TABLE ] [ ONLY ] name [ * ] [, ….]

TABLE

265 © Copyright EnterpriseDB Corporation. All rights reserved.

 DML Statements
Statement Syntax
INSERT INSERT INTO table_name [ ( column_name [, ...] ) ]
{ DEFAULT VALUES | VALUES ( { expression | DEFAULT } [, ...] ) [,...] | query }

UPDATE UPDATE [ ONLY ] table_name

SET column_name = { expression | DEFAULT }
[ WHERE condition]

DELETE DELETE FROM [ ONLY ] table_name

[ WHERE condition ]

SELECT SELECT [ ALL | DISTINCT ] [ * | expression ]

[FROM table [,.. ]

266 © Copyright EnterpriseDB Corporation. All rights reserved.

 DCL Statements
Statement Syntax

GRANT GRANT { { SELECT | INSERT | UPDATE ……} [, … ] | ALL [PRIVILEGES ] }

ON { [ TABLE ] table_name [, …] | ALL TABLES IN SCHEMA schema_name [ ,…] }
TO role_specification [, …] [ WITH GRANT OPTION ]

REVOKE REVOKE [ GRANT OPTION FOR ]

{ { SELECT | INSERT | UPDATE ……} [, … ] | ALL [PRIVILEGES ] }
ON { [ TABLE ] table_name [, …] | ALL TABLES IN SCHEMA schema_name [ ,…] }
FROM { [ GROUP ] role_name | PUBLIC } [, …]

267 © Copyright EnterpriseDB Corporation. All rights reserved.

 Transaction Control Language
Statement Syntax
COMMIT COMMIT [ WORK | TRANSACTION ]

ROLLBACK ROLLBACK [ WORK | TRANSACTION ]

SAVEPOINT SAVEPOINT savepoint_name

SET TRANSACTION SET TRANSACTION transaction_mode [, …]

268 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Objects
Object Description
TABLE Named collection of rows
VIEW Virtual table, can be used to hide complex queries
SEQUENCE Used to automatically generate integer values that follow a pattern
INDEX A common way to enhance query performance
DOMAIN A data type with optional constraints

269 © Copyright EnterpriseDB Corporation. All rights reserved.

 Tables
• A table is a named collection of rows
• Each table row has same set of columns
• Each column has a data type
• Tables can be created using the CREATE TABLE statement
• Syntax:

270 © Copyright EnterpriseDB Corporation. All rights reserved.

 Types of Constraints
• Constraints are used to enforce data integrity
• EDB Postgres Advanced Server supports different types of constraints:
• NOT NULL
• CHECK
• UNIQUE
• PRIMARY KEY
• FOREIGN KEY
• Constraints can be defined at the column level or table level
• Constraints can be added to an existing table using the ALTER TABLE statement
• Constraints can be declared DEFERRABLE or NOT DEFERRABLE
• Constraints prevent the deletion of a table if there are dependencies

271 © Copyright EnterpriseDB Corporation. All rights reserved.

 Views

• A View is a Virtual Table and can be used to hide complex queries

• Can also be used to represent a selected view of data
• Simple views are updatable and allow non-updatable columns
• Views can be created using the CREATE VIEW statement
• Syntax:
=> CREATE [ OR REPLACE ] VIEW name [ ( column_name [, ...] ) ]
[ WITH ( view_option_name [= view_option_value] [, ... ] ) ]
AS query

272 © Copyright EnterpriseDB Corporation. All rights reserved.

 Sequences
• A sequence is used to automatically generate integer values that follow a
pattern
• A sequence has a name, start point and an end point
• Sequence values can be cached for performance
• Sequence can be used using CURRVAL and NEXTVAL functions
• Syntax:
=> CREATE SEQUENCE name [ INCREMENT [ BY ] increment ]
[ MINVALUE minvalue] [ MAXVALUE maxvalue]
[ START [ WITH ] start ] [ CACHE cache ] [ [ NO ] CYCLE ]
[ OWNED BY { table_name.column_name | NONE } ]

273 © Copyright EnterpriseDB Corporation. All rights reserved.

 Domains

• A domain is a data type with optional constraints

• Domains can be used to create a data type which allows a
selected list of values Table: emp
Column: cityname
Data Type: city

Domain: city Table: shop

Allowed Values: Edmonton, Column: shoplocation

Calgary, Red Deer Data Type: city

Table: clients
Column: res_city
Data Type: city
274 © Copyright EnterpriseDB Corporation. All rights reserved.
 Types of JOINS
Type Description

INNER JOIN Returns all matching rows from both tables

LEFT OUTER JOIN Returns all matching rows and rows from left-hand table even if there is no
corresponding row in the joined table

RIGHT OUTER JOIN Returns all matching rows and rows from right-hand table even if there is no
corresponding row in the joined table

FULL OUTER JOIN Returns all matching as well as not matching rows from both tables

CROSS JOIN Returns all rows of both tables with Cartesian product on number of rows

275 © Copyright EnterpriseDB Corporation. All rights reserved.

 Using SQL Functions
• Can be used in SELECT statements and WHERE clauses
• Includes
• String Functions
• Format Functions
• Date and Time Functions
• Aggregate Functions
• Example:
=> SELECT lower(name)FROM departments;
=> SELECT * FROM departments
WHERE lower(name) = 'development';

276 © Copyright EnterpriseDB Corporation. All rights reserved.

 SQL Format Functions
Function Return Type Description Example
to_char(timestamp, convert time stamp to to_char(current_timestamp,
text
text) string 'HH12:MI:SS')
to_char(interval, convert interval to to_char(interval
text
text) string '15h 2m 12s', 'HH24:MI:SS')
convert integer to
to_char(int, text) text to_char(125, '999')
string
to_char(double real/double precision to to_char(125.8::real,
text
precision, text) strconvert ing '999D9')
to_char(numeric, convert numeric to
text to_char(-125.8, '999D99S')
text) string
to_date('05 Dec 2000',
to_date(text, text) date convert string to date
'DD Mon YYYY')
to_number(text, convert string to to_number('12,454.8-',
numeric
text) numeric '99G999D9S')
to_timestamp(text, timestamp with convert string to time to_timestamp('05 Dec 2000',
text) time zone stamp 'DD Mon YYYY')
to_timestamp(double timestamp with convert Unix epoch to
to_timestamp(1284352323)
precision) time zone time stamp

277 © Copyright EnterpriseDB Corporation. All rights reserved.

 Execution Plan

• An execution plan shows the detailed steps necessary to execute a SQL

statement
• Planner is responsible for generating the execution plan
• The Optimizer determines the most efficient execution plan
• Optimization is cost-based, cost is estimated resource usage for a plan
• Cost estimates rely on accurate table statistics, gathered with ANALYZE
• Costs also rely on seq_page_cost, random_page_cost, and others
• The EXPLAIN command is used to view a query plan
• EXPLAIN ANALYZE is used to run the query to get actual runtime stats

278 © Copyright EnterpriseDB Corporation. All rights reserved.

 Execution Plan Components

Execution Plan Components: Syntax:

• Cardinality - Row Estimates =# EXPLAIN [ ( option [, ...] ) ] statement
EXPLAIN [ ANALYZE ] [ VERBOSE ] statement
• Access Method - Sequential where option can be one of:
or Index ANALYZE [ boolean ]

• Join Method - Hash, Nested VERBOSE [ boolean ]

COSTS [ boolean ]
Loop etc.
SETTINGS [ boolean ]
• Join Type, Join Order BUFFERS [ boolean ]

• Sort and Aggregates WAL [ boolean ]

TIMING [ boolean ]

SUMMARY [ boolean ]

FORMAT { TEXT | XML | JSON | YAML }

279 © Copyright EnterpriseDB Corporation. All rights reserved.

 Explain Example

• Example
postgres=# EXPLAIN SELECT * FROM emp;
QUERY PLAN
------------------------------------------------------
Seq Scan on emp (cost=0.00..1.14 rows=14 width=145)

• The numbers that are quoted by EXPLAIN are:

• Estimated start-up cost
• Estimated total cost
• Estimated number of rows output by this plan node
• Estimated average width (in bytes) of rows output by this plan node

280 © Copyright EnterpriseDB Corporation. All rights reserved.

 PEM - Query Tool’s Visual Explain

281 © Copyright EnterpriseDB Corporation. All rights reserved.

 Quoting
• Single quotes and dollar quotes are used to specify non-numeric
values
• Example:
'hello world'
'2011-07-04 13:36:24'
'{1,4,5}'
$$A string "with" various 'quotes' in.$$
$foo$A string with $$ quotes in $foo$
• Double quotes are used for names of database objects which either
clash with keywords, contain mixed case letters, or contain
characters other than a-z, 0-9 or underscore
• Example:
SELECT * FROM "select“
CREATE TABLE "HelloWorld" ...
SELECT * FROM "Hi everyone and everything"

282 © Copyright EnterpriseDB Corporation. All rights reserved.

 Indexes

• Indexes are a common way to enhance performance

• Postgres supports several index types:
B-tree (default)

Hash

Block Range Index (BRIN)

GIN

GIST

SP-GiST Indexes

Index on Expressions

283 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example Index
• Syntax:
CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] name ] ON [ ONLY ] table_name [ USING method ]
( { column_name | ( expression ) | constant } [ COLLATE collation ] [ opclass [ ( opclass_parameter =
value [, ... ] ) ] ] [ ASC | DESC ] [ NULLS { FIRST | LAST } ] [, ...] )
[ INCLUDE ( column_name [, ...] ) ]
[ NULLS [ NOT ] DISTINCT ]
[ WITH ( storage_parameter [= value] [, ... ] ) ]
[ LOGGING | NOLOGGING ]
[ LOCAL ]
[ TABLESPACE tablespace_name ]
[ PARALLEL [ integer | ( degree {integer | DEFAULT } ) ] | NOPARALLEL ]
[ WHERE predicate ]

• Example:

284 © Copyright EnterpriseDB Corporation. All rights reserved.

Oracle Compatibility and
Migration
 Oracle Compatibility
• Oracle compatibility helps an application running in an Oracle environment to run
in an EDB Postgres Advanced Server environment with minimal or no changes
• Oracle Compatibility in EDB Postgres Advanced Server offers:
Oracle Compatible Tools
Data types
SQL statements
Oracle Compatible Catalog Views
Stored Procedure Language (SPL)
Built-in Packages
Triggers
Table Partitioning
Optimizer Hints
Open Client Library (OCL) for Oracle Call Interface (OCI)

286 © Copyright EnterpriseDB Corporation. All rights reserved.

 Oracle Compatible Tools

EDB*Plus

EDB*Loader
EDB Postgres
Advanced
Server
EDB*Wrap
Oracle DBA

DRITA

287 © Copyright EnterpriseDB Corporation. All rights reserved.

 Table Partitioning

• Table partitioning can be used to break one logically large table into
smaller physical pieces
• EDB Postgres Advanced Server supports Oracle compatible syntax for
table partitioning with support for:
• List Partitioning
• Range Partitioning
• Hash Partitioning
• Sub Partitioning
• Interval Partitioning

288 © Copyright EnterpriseDB Corporation. All rights reserved.

 Stored Procedure Language
• SPL is the procedural language extension to SQL
• SPL is compatible with Oracle PL/SQL
• Can be used to create functions, procedures and packages
• Can be used to create sub-programs like sub-procedures and sub-functions inside
standalone SPL Programs
• Provides procedural constructs such as:
• Variables, constants, and types
• Conditional statements
• Dynamic SQL
• Cursors
• Triggers

289 © Copyright EnterpriseDB Corporation. All rights reserved.

 Built-in Packages
• EDB Postgres Advanced Server provides built-in packages compatible with Oracle
• Over 24 most commonly used Oracle built-in packages are available in EDB Postgres
Advanced Server
• These built-in packages provide administration and maintenance utilities
DBMS_ALERT DBMS_AQ DBMS_AQADM DBMS_CRYPTO DBMS_JOB

DBMS_LOB DBMS_LOCK DBMS_MVIEW DBMS_OUTPUT DBMS_PIPE

DBMS_PROFILER DBMS_RANDOM DBMS_RLS DBMS_SESSION DBMS_SCHEDULER

DBMS_SQL DBMS_UTILITY DBMS_REDACT UTL_ENCODE UTL_FILE

UTL_HTTP UTL_MAIL UTL_SMTP UTL_URL UTL_RAW

290 © Copyright EnterpriseDB Corporation. All rights reserved.

 Migration Process
Identify Use Migration
Results,
Assessment candidate Portal for
Solutions, Plan
database(s) Assessment

Database Migrate the

Prepare Migrate the
schema
Migration environments data
definition

Application Connect the Test and

Revise
Migration application Verify

Deployment Monitor Manage Tune

291 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB Migration Portal

• Migration Portal Combines:

• Native Oracle Compatibility
• Schemas
• DB Code
• Application interfaces
• Rich knowledge base from 10+ years of migrations
• Cloud-based machine learning of new code translations
• Supports assessment and migration from Oracle 11g to 19c to EDB
Advanced Server 11 to 15

292 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Migration Toolkit

• MTK is a powerful command

line tool
• It simplifies the process of
migrating from other
databases to EDB Postgres
Advanced Server
• The EDB Postgres Migration
Toolkit can be used to migrate
the entire schema including
triggers and stored procedures

293 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Migration using Replication Server
Single Master Replication (SMR) between Oracle and EDB Postgres for Migration

• Replicate Oracle or SQL Server

data to EDB Postgres Advanced
Server
• Distributed multi-
Publication/Subscription
Architecture
• Synchronize data across
geographies
• Replicate selected tables and
sequences

294 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Data Types • Sequences

• Structured Query Language • Domains
(SQL) • SQL Joins and Functions
• DDL, DML and DCL Statements • Explain Plans
• Transaction Control Statements • Quoting in PostgreSQL

• Tables and Constraints • Indexes

• Views and Materialized Views • Oracle Compatibility and Tools

295 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

Test your knowledge:

1. Initiate a psql session
2. edb-psql commands access the database True/False
3. The following SELECT statement executes successfully: True/False
=> SELECT ename, job, sal AS Salary FROM emp;
4. The following SELECT statement executes successfully: True/False
=> SELECT * FROM emp;
5. There are coding errors in the following statement. Can you identify them?
=> SELECT empno, ename, sal * 12 annual salary FROM emp;

296 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2

1. The staff in the HR department wants to hide some of the data in the
EMP table. They want a view called EMPVU based on the employee
numbers, employee names, and department numbers from the EMP
table. They want the heading for the employee name to be EMPLOYEE.
2. Confirm that the view works. Display the contents of the EMPVU view.
3. Using your EMPVU view, write a query for the SALES department to
display all employee names and department numbers.

297 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 3

1. You need a sequence that can be used with the primary key column of
the dept table. The sequence should start at 60 and have a maximum
value of 200. Have your sequence increment by 10. Name the
sequence dept_id_seq.
2. To test your sequence, write a script to insert two rows in the dept
table.

298 © Copyright EnterpriseDB Corporation. All rights reserved.

Module 12
Backup, Recovery and PITR
 Module Objectives
• Backup Types
• Database SQL Dumps
• Restoring SQL Dumps
• Offline Physical Backups
• Continuous Archiving
• Online Physical Backups Using pg_basebackup
• Point-in-time Recovery
• Recovery Settings
• Backup Tools – Barman and pgBackRest

300 © Copyright EnterpriseDB Corporation. All rights reserved.

 Types of Backup
• As with any database, PostgreSQL databases should be backed up regularly

Logical Backups

• Database SQL Dumps using pg_dump

• Database Cluster SQL Dump using pg_dumpall

Physical Backups

• Offline File System Level Backups using OS commands

• Online File System Level Backups using pg_basebackup
• Backup Tool – Barman and pgBackRest

301 © Copyright EnterpriseDB Corporation. All rights reserved.

Logical Backups
 Database SQL Dump
• Generate a text file with SQL commands
• EDB Postgres Advanced Server provides the utility program pg_dump for this
purpose
• pg_dump does not block readers or writers
• pg_dump does not operate with special permissions
• Dumps created by pg_dump are internally consistent, that is, the dump
represents a snapshot of the database as of the time pg_dump begins running
• Syntax:
$ pg_dump [options] [dbname]

303 © Copyright EnterpriseDB Corporation. All rights reserved.

 pg_dump Options
-a - Data only. Do not dump the data definitions (schema)
-s - Data definitions (schema) only. Do not dump the data
-n <schema> - Dump from the specified schema only
-t <table> - Dump specified table only
-f <file name> - Send dump to specified file. Filename can be specified using absolute or relative location
-Fp - Dump in plain-text SQL script (default)
-Ft - Dump in tar format
-Fc - Dump in compressed, custom format
-Fd - Dump in directory format
-j njobs - dump in parallel by dumping n jobs tables simultaneously. Only supported with –Fd
-B, --no-blobs - Excludes large objects in dump
-v - Verbose option

304 © Copyright EnterpriseDB Corporation. All rights reserved.

 SQL Dump - Large Databases

• If the operating system has maximum file size limits, it can

cause problems when creating large pg_dump output files
• Standard Unix tools can be used to work around this potential
problem
• Use a compression program, for example gzip:
$ pg_dump dbname | gzip > filename.gz

• The split command allows you to split the output into smaller files:
$ pg_dump dbname | split -b 1m - filename

305 © Copyright EnterpriseDB Corporation. All rights reserved.

 Restore – SQL Dump

• Backups taken using pg_dump with plain text

format(Fp) psql client
• Backups taken using pg_dumpall

• Backup taken using pg_dump with custom(Fc),

tar(Ft) or director(Fd) formats
• Supports parallel jobs for during restore pg_restore utility
• Selected objects can be restored

306 © Copyright EnterpriseDB Corporation. All rights reserved.

 pg_restore Options
-l - Display TOC of the archive file

-F [c|d|t] - Backup file format

-d <database name> - Connect to the specified database. Also restores to this database if -C option is omitted

-C - Create the database named in the dump file and restore directly into it

-a - Restore the data only, not the data definitions (schema)

-s - Restore the data definitions (schema) only, not the data

-n <schema> - Restore only objects from specified schema

-N <schema> - do not restore objects in this schema

-t <table> - Restore only specified table

-v - Verbose option

307 © Copyright EnterpriseDB Corporation. All rights reserved.

 Entire Cluster - SQL Dump

• pg_dumpall is used to dump an entire database cluster in

plain-text SQL format
• Dumps global objects - users, groups, and associated
permissions
• Use psql to restore
• Syntax:
$ pg_dumpall [options…] > filename.backup

308 © Copyright EnterpriseDB Corporation. All rights reserved.

 pg_dumpall Options
-a - Data only. Do not dump schema
-s - Data definitions (schema) only
-g - Dump global objects only - not databases
-r - Dump only roles
-c - Clean (drop) databases before recreating
-O - Skip restoration of object ownership
-x - do not dump privileges (grant/revoke)
-v - Verbose option
--disable-triggers - disable triggers during data-only restore
--no-role-passwords - do not dump passwords for roles. This allows use of pg_dumpall by non-superusers
--exclude-database -exclude database whose name match with given pattern

309 © Copyright EnterpriseDB Corporation. All rights reserved.

Physical Backups
 Backup - File system level backup
• An alternative backup strategy is to directly copy the files that Postgres uses to store the
data in the database
• You can use whatever method you prefer for doing usual file system backups, for example:
$ tar -cf backup.tar /usr/local/edb/data

• The database server must be shut down or in backup mode in order to get a usable backup
• File system backups only work for complete backup and restoration of an entire database
cluster
• Two types of File system backup
• Offline backups
• Online backups

311 © Copyright EnterpriseDB Corporation. All rights reserved.

 File System Backups
Offline Backups

• Taken using OS Copy command

• Database Server must be shutdown
• Cluster Level Backup and Restore

Online Backups

• Continuous archiving must be enabled

• Database server start/end backup mode
• Cluster Level Backup and Restore with PITR
• Methods - pg_basebackup, Barman, pgBackRest

312 © Copyright EnterpriseDB Corporation. All rights reserved.

 Continuous Archiving
• Postgres maintains WAL files for all transactions in pg_wal directory
• Postgres automatically maintains the WAL logs which are full and
switched
• Continuous archiving can be setup to keep a copy of switched WAL
Logs which can be later used for recovery
• It also enables online file system backup of a database cluster
• Requirements:
- wal_level must be set to replica
- archive_mode must be set to on (can be set to always)
- archive_command must be set in postgresql.conf which archives WAL
logs and supports PITR

313 © Copyright EnterpriseDB Corporation. All rights reserved.

 Continuous Archiving Methods
•Parameters in postgresql.conf file
•wal_level = replica
•archive_mode = on
Archiver Process •archive_command = ‘cp -i %p /edb/archive/%f’
•Restart the database server
•Archive files are generated after every log switch

•Parameters in postgresql.conf file

•wal_level = replica
•archive_mode = on
•max_wal_senders = 3
Streaming WAL •Restart the database server
•pg_receivewal –h localhost –D /edb/archive
•Transactions are streamed and written to archive files

314 © Copyright EnterpriseDB Corporation. All rights reserved.

 Base Backup Using pg_basebackup Tool

• pg_basebackup can take an online base backup of a

database cluster
• This backup can be used for PITR or Streaming
Replication
• pg_basebackup makes a binary copy of the database
cluster files
• System is automatically put in and out of backup mode

315 © Copyright EnterpriseDB Corporation. All rights reserved.

 pg_basebackup - Online Backup
• Steps require to take Base Backup:
• Modify pg_hba.conf
host replication enterprisedb [Ipv4 address of client]/32 md5

• Modify postgresql.conf
wal_level = replica
archive_command = 'cp -i %p /users/enterprisedb/archive/%f‘
archive_mode = on
max_wal_senders = 3
wal_keep_size = 512

• Backup Command:
$ pg_basebackup [options] ..

316 © Copyright EnterpriseDB Corporation. All rights reserved.

 Options for pg_basebackup command
-D <directory name> - Location of backup
-F <p or t> - Backup files format. Plain(p) or tar(t)
-R - write standby.signal and append postgresql.auto.conf
-T OLDDIR=NEWDIR - relocate tablespace in OLDDIR to NEWDIR
--waldir - Write ahead logs location
-z - Enable compression(tar) for files
-Z - Compress backup based on setting set to none, client or server
-P - Progress Reporting
-h host - host on which cluster is running
-p port - cluster port

• To create a base backup of the server at localhost and store it in the local directory
/usr/local/edb/backup
$ pg_basebackup -h localhost -D /usr/local/edb/backup

317 © Copyright EnterpriseDB Corporation. All rights reserved.

 Verify Base Backups
• Verify backup taken by pg_basebackup using pg_verifybackup utility
• Backup is verified against a backup_manifest generated by the server at
the time of the backup
• Only plain format backups can be verified

318 © Copyright EnterpriseDB Corporation. All rights reserved.

Restoring Physical Backups
 Point-in-time Recovery

• Point-in-time recovery (PITR) is the ability to restore a

database cluster up to the present or to a specified point
of time in the past
• Uses a full database cluster backup and the write-ahead
logs found in the /pg_wal subdirectory
• Must be configured before it is needed (write-ahead log
archiving must be enabled)

320 © Copyright EnterpriseDB Corporation. All rights reserved.

 Performing Point-in-Time Recovery
Prepare Restore Configure Recover
Stop the server Copy data cluster Configure Start the server
Take a file system files and folders recovery settings using service or
level backup if from backup in pg_ctl utility
possible location to the postgresql.conf
Check error log
data directory file
Clean the data for any issue
directory Use cp -rp to Create
recovery.signal
preserve recovery.signal
file is removed
privileges file in the data
automatically
directory
after recovery

321 © Copyright EnterpriseDB Corporation. All rights reserved.

 Point-in-Time Recovery Settings
• Restoring archived WAL using restore_command
parameter:
• Unix:
restore_command = 'cp /mnt/server/archivedir/%f "%p"'
• Windows:
restore_command = 'copy c:\\mnt\\server\\archivedir\\"%f" "%p"'

• Recovery target settings:

• recovery_target_name
• recovery_target_time
• recovery_target_xid
• recovery_target_action

322 © Copyright EnterpriseDB Corporation. All rights reserved.

Backup and Recovery Tools
 Backup And Recovery Manager(Barman)

• Open-source administration tool for remote backups and

disaster recovery
• Manage backups and the recovery phase of multiple
servers from one location
• Distributed under GNU GPL 3 and maintained by EDB

324 © Copyright EnterpriseDB Corporation. All rights reserved.

 Barman Architecture

• One Barman for multiple Primary Replica

Postgres servers
• Standard connection to
Postgres for management,
coordination and monitoring
• Standard replication Barman

connection for running

Backup
Server Processing Tier Remote Tier

pg_basebackup and
pg_receivewal S3/Azure

Local Tier (Barman

• Supports rsync/SSH Storage)

http://docs.pgbarman.org/
325 © Copyright EnterpriseDB Corporation. All rights reserved.
 Barman - Features

• Remote backup and restore with rsync and the PostgreSQL protocol
• Support for file level incremental backups with rsync
• Retention policy support
• WAL Archive Compression with gzip, bzip2, or pigz
• Backup data verification
• Backup with RPO=0 using a synchronous physical streaming
replication connection
• Rate limiting
https://www.pgbarman.org/about/

326 © Copyright EnterpriseDB Corporation. All rights reserved.

 Postgres Backup And Restore
pgBackRest

Solves common bottleneck

problems with parallel Support capabilities like Fully supported Open Source
processing for backup, symmetric encryption, and backup tool with
compression, restoring and partial restore troubleshooting support
archiving

327 © Copyright EnterpriseDB Corporation. All rights reserved.

 -

Feature comparison
Capability Added value Barman pgBackRest Pg_basebackup

SSH protocol support Yes Yes -

PostgreSQL protocol Works without passwordless ssh. Yes - Yes
Incremental backups Yes Yes -
RPO=0 Restore up to the last commit Yes - -
Rate limiting Preserve IO for Postgres Yes - Yes
Retention and List backups Yes Yes -
Backup compression Less backup space required - Yes -
Symmetric encryption Lower security footprint for the backup data - Yes -
Partial restore (only selected databases) Restore required data for analysis purposes - Yes -
S3 and Azure Blob Support Use flexible Cloud Storage for backup storage Yes Yes -
Nagios integration Monitor your backups with Nagios Yes Yes -

328 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary
• Backup Types
• Database SQL Dumps
• Restoring SQL Dumps
• Offline Physical Backups
• Continuous Archiving
• Online Physical Backups Using pg_basebackup
• Point-in-time Recovery
• Recovery Settings
• Backup Tools – Barman and pgBackRest

329 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

1. The edbstore website database is all setup and as a DBA

you need to plan a proper backup strategy and implement
it
- As the root user, create a folder /pgbackup and assign
ownership to the Postgres user using the chown utility or the
Windows security tab in folder properties
- Take a full database dump of the edbstore database with the
pg_dump utility. The dump should be in plain text format
- Name the dump file as edbstore_full.sql and store it in the
/pgbackup directory

330 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2
1. Take a dump of the edbuser schema from the edbstore database
and name the file as edbstore_schema.sql

2. Take a data-only dump of the edbstore database, disable all triggers

for a faster restore, use the INSERT command instead of COPY, and
name the file as edbstore_data.sql

3. Take a full dump of customers table and name the file as

edbstore_customers.sql

331 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 3

1. Take a full database dump of edbstore in compressed

format using the pg_dump utility, name the file as
edbstore_full_fc.dmp

2. Take a full database cluster dump using pg_dumpall.

Remember pg_dumpall supports only plain text format;
name the file edbdata.sql

332 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 4
In this exercise you will demonstrate your ability to restore a database.
1. Drop database edbstore.
2. Create database edbstore with owner edbuser.
3. Restore the full dump from edbstore_full.sql and verify all the objects
and their ownership.
4. Drop database edbstore.
5. Create database edbstore with edbuser owner.
6. Restore the full dump from the compressed file edbstore_full_fc.dmp
and verify all the objects and their ownership.

333 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 5

1. Create a directory /opt/arch or c:\arch and give

ownership to the Postgres user.

2. Configure your cluster to run in archive mode and set the

archive log location to be /opt/arch or c:\arch.

3. Take a full online base backup of your cluster in the

/pgbackup directory using the pg_basebackup utility.

334 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 13
Routine Maintenance Tasks
 Module Objectives

• Updating Optimizer Statistics

• Handling Data Fragmentation using Routine Vacuuming
• Preventing Transaction ID Wraparound Failures
• Automatic Maintenance using Autovacuum
• Re-indexing in Postgres

336 © Copyright EnterpriseDB Corporation. All rights reserved.

 Database Maintenance

• Data files become fragmented as data is modified and deleted

• Database maintenance helps reconstruct the data files
• If done on time nobody notices but when not done everyone
knows
• Must be done before you need it
• Improves performance of the database
• Saves database from transaction ID wraparound failures
337 © Copyright EnterpriseDB Corporation. All rights reserved.
 Maintenance Tools
• Maintenance thresholds can be configured using the PEM Client
• Postgres maintenance thresholds can be configured in postgresql.conf
• Manual scripts can be written watch stat tables like pg_stat_user_tables
• Maintenance commands:
• ANALYZE
• VACUUM
• CLUSTER
• Maintenance command vacuumdb can be run from OS prompt
• Autovacuum can help in automatic database maintenance

338 © Copyright EnterpriseDB Corporation. All rights reserved.

 Optimizer Statistics

• Optimizer statistics play a vital role in query planning

• Not updated in real time
• Collects information for relations including size, row counts, average
row size and row sampling
• Stored permanently in catalog tables
• The maintenance command ANALYZE updates the statistics
• Thresholds can be set using PEM Client to alert you when statistics
are not collected on time

339 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Updating Statistics

340 © Copyright EnterpriseDB Corporation. All rights reserved.

 Data Fragmentation and Bloat

• Data is stored in data file pages

• An update or delete of a row does not immediately
remove the row from the disk page
• Eventually this row space becomes obsolete and causes
fragmentation and bloating
• Set PEM Alert for notifications

341 © Copyright EnterpriseDB Corporation. All rights reserved.

 Routine Vacuuming

• Obsoleted rows can be removed or reused using vacuuming

• Helps in shrinking data file size when required
• Vacuuming can be automated using autovacuum
• The VACUUM command locks tables in access exclusive mode
• Long running transactions may block vacuuming, thus it
should be done during low usage times

342 © Copyright EnterpriseDB Corporation. All rights reserved.

 Vacuuming Commands

• When executed, the VACUUM command:

• Can recover or reuse disk space occupied by obsolete rows
• Updates data statistics
• Updates the visibility map, which speeds up index-only scans
• Protects against loss of very old data due to transaction ID wraparound
• The VACUUM command can be run in two modes:
- VACUUM
- VACUUM FULL

343 © Copyright EnterpriseDB Corporation. All rights reserved.

 Vacuum and Vacuum Full
• VACUUM
• Removes dead rows and marks the space available for future reuse
• Does not return the space to the operating system
• Space is reclaimed if obsolete rows are at the end of a table

• VACUUM FULL
• More aggressive algorithm compared to VACUUM
• Compacts tables by writing a complete new version of the table file with no dead space
• Takes more time
• Requires extra disk space for the new copy of the table, until the operation completes

344 © Copyright EnterpriseDB Corporation. All rights reserved.

 VACUUM Syntax
• VACUUM [ ( option [, ...] ) ] [ table_and_columns [, ...] ]
where option can be one of:
• FULL [ boolean ]
• FREEZE [ boolean ]
• VERBOSE [ boolean ]
• ANALYZE [ boolean ]
• DISABLE_PAGE_SKIPPING [ boolean ]
• SKIP_LOCKED [ boolean ]
• INDEX_CLEANUP [ boolean ]
• TRUNCATE [ boolean ]
• PARALLEL integer

345 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Vacuuming

346 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – Vacuuming (continued)

347 © Copyright EnterpriseDB Corporation. All rights reserved.

 Preventing Transaction ID Wraparound Failures

• MVCC depends on transaction ID numbers

• Transaction IDs have limited size (32 bits at this writing)
• A cluster that runs for a long time (more than 4 billion
transactions) would suffer transaction ID wraparound
• This causes a catastrophic data loss
• To avoid this problem, every table in the database must be
vacuumed at least once for every two billion transactions

348 © Copyright EnterpriseDB Corporation. All rights reserved.

 Vacuum Freeze
• VACUUM FREEZE will mark rows as frozen
• Postgres reserves a special XID, FrozenTransactionId
• FrozenTransactionId is always considered older than every normal XID
• VACUUM FREEZE replaces transaction IDs with FrozenTransactionId, thus
rows will appear to be “in the past”
• vacuum_freeze_min_age controls when a row will be frozen
• VACUUM normally skips pages without dead row versions, but some rows may
need FREEZE
• vacuum_freeze_table_age controls when a whole table must be scanned

349 © Copyright EnterpriseDB Corporation. All rights reserved.

 The Visibility Map

• Each heap relation has a Visibility Map which keeps track of

which pages contain only tuples
• Stored at <relfilenode>_vm
• Helps vacuum to determine whether pages contain dead rows
• Can also be used by index-only scans to answer queries
• VACUUM command updates the visibility map
• The visibility map is vastly smaller, so can be cached easily

350 © Copyright EnterpriseDB Corporation. All rights reserved.

 vacuumdb Utility

• The VACUUM command has a command-line executable

wrapper called vacuumdb
• vacuumdb can VACUUM all databases using a single command
• Syntax:
• vacuumdb [OPTION]... [DBNAME]

• Available options can be listed using:

• vacuumdb --help

351 © Copyright EnterpriseDB Corporation. All rights reserved.

 Autovacuuming
• Highly recommended feature of Postgres
• It automates the execution of VACUUM, FREEZE and ANALYZE commands
• Autovacuum consists of a launcher and many worker processes
• A maximum of autovacuum_max_workers worker processes are allowed
• Launcher will start one worker within each database every autovacuum_naptime
seconds
• Workers check for inserts, updates and deletes and execute VACUUM and/or ANALYZE as
needed
• track_counts must be set to TRUE as autovacuum depends on statistics
• Temporary tables cannot be accessed by autovacuum

352 © Copyright EnterpriseDB Corporation. All rights reserved.

 Autovacuuming Parameters
Autovacuum Launcher Process
•autovacuum
Autovacuum Worker Processes
•autovacuum_max_workers
•autovacuum_naptime
Vacuuming Thresholds
•autovacuum_vacuum_scale_factor
•autovacuum_vacuum_threshold
•autovacuum_analyze_scale_factor
•autovacuum_analyze_threshold
•autovacuum_vacuum_insert_scale_threshold
•autovacuum_vacuum_insert_threshold
•autovacuum_freeze_max_age

353 © Copyright EnterpriseDB Corporation. All rights reserved.

 Per-Table Thresholds
• Autovacuum workers are resource intensive
• Table-by-table autovacuum parameters can be configured for large tables
• Configure the following parameters using ALTER TABLE or CREATE TABLE:
• autovacuum_enabled
• autovacuum_vacuum_threshold
• autovacuum_vacuum_scale_factor
• autovacuum_analyze_threshold
• autovacuum_analyze_scale_factor
• autovacuum_vacuum_insert_scale_threshold
• autovacuum_vacuum_insert_threshold
• autovacuum_freeze_max_age

354 © Copyright EnterpriseDB Corporation. All rights reserved.

 Routine Reindexing
• Indexes are used for faster data access
• UPDATE and DELETE on a table modify underlying index entries
• Indexes are stored on data pages and become fragmented over time
• REINDEX rebuilds an index using the data stored in the index's table
• Time required depends on:
• Number of indexes
• Size of indexes
• Load on server when running command

355 © Copyright EnterpriseDB Corporation. All rights reserved.

 When to Reindex

• There are several reasons to use REINDEX:

• An index has become "bloated", meaning it contains many empty or nearly-
empty pages
• You have altered a storage parameter (such as fillfactor) for an index
• An index built with the CONCURRENTLY option failed, leaving an "invalid"
index
• Syntax:
=> REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE |
SYSTEM } [ CONCURRENTLY ] name

356 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Updating Optimizer Statistics

• Handling Data Fragmentation using Routine Vacuuming
• Preventing Transaction ID Wraparound Failures
• Automatic Maintenance using Autovacuum
• Re-indexing in Postgres

357 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1
1. While monitoring table statistics on the edbstore database, you found
that some tables are not automatically maintained by autovacuum. You
decided to perform manual maintenance on these tables. Write a SQL
script to perform the following maintenance:
• Reclaim obsolete row space from the customers table.
• Update statistics for emp and dept tables.
• Mark all the obsolete rows in the orders table for reuse.

2. Execute the newly created maintenance script on edbstore database.

358 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 2

1. The composite index named ix_orderlines_orderid

on (orderid, orderlineid) columns of the
orderlines table is performing very slowly. Write a
statement to reindex this index for better performance.

359 © Copyright EnterpriseDB Corporation. All rights reserved.

Module - 14
Moving Data Using COPY Command
 Module Objectives

• Loading flat files

• Import and export data using COPY
• Examples of COPY Command
• Using COPY FREEZE for performance
• EDB*Loader
• Data Loading Methods
• Invoking EDB*Loader and Control File
• EDB*Loader Exit Codes

361 © Copyright EnterpriseDB Corporation. All rights reserved.

 Loading Flat Files into Database Tables

• A "flat file" is a plain text or mixed text file which usually

contains one record per line
• EDB Postgres Advanced Server offers two options to load
flat files into a database table:
• EDB*Loader
• COPY Command

362362 © Copyright EnterpriseDB Corporation. All rights reserved.

COPY Command
 The COPY Command

• COPY moves data between EDB Postgres Advanced

Server tables and standard file-system files
• COPY TO copies the contents of a table or a query to a file
• COPY FROM copies data from a file to a table
• The file must be accessible to the server

364 © Copyright EnterpriseDB Corporation. All rights reserved.

 COPY Command Syntax
Copy From:
•COPY table_name [(column list)] FROM 'filename'|PROGRAM 'command'|STDIN [options][WHERE
cond.]
Copy To:
•COPY table_name[(column list])|(query) TO 'filename'|PROGRAM 'command'|STDOUT [options]

Copy Command Options

•FORMAT format_name
•OIDS [ boolean ]
•FREEZE [ boolean ]
•DELIMITER 'delimiter_character'
•NULL 'null_string'
•HEADER [ boolean ]
•QUOTE 'quote_character'
•ESCAPE 'escape_character'
•FORCE_QUOTE { ( column_name [, ...] ) | * }
•FORCE_NOT_NULL ( column_name [, ...] )
•FORCE_NULL ( column_name [, ...] )
•ENCODING 'encoding_name'

365 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example Export to File
=> COPY emp (empno,ename,job,sal,comm,hiredate) TO '/tmp/emp.csv' CSV HEADER;
COPY
=> \! cat /tmp/emp.csv
empno,ename,job,sal,comm,hiredate
7369,SMITH,CLERK,800.00,,17-DEC-80 00:00:00
7499,ALLEN,SALESMAN,1600.00,300.00,20-FEB-81 00:00:00
7521,WARD,SALESMAN,1250.00,500.00,22-FEB-81 00:00:00
7566,JONES,MANAGER,2975.00,,02-APR-81 00:00:00
7654,MARTIN,SALESMAN,1250.00,1400.00,28-SEP-81 00:00:00
7698,BLAKE,MANAGER,2850.00,,01-MAY-81 00:00:00
7782,CLARK,MANAGER,2450.00,,09-JUN-81 00:00:00
7788,SCOTT,ANALYST,3000.00,,19-APR-87 00:00:00
7839,KING,PRESIDENT,5000.00,,17-NOV-81 00:00:00
7844,TURNER,SALESMAN,1500.00,0.00,08-SEP-81 00:00:00
7876,ADAMS,CLERK,1100.00,,23-MAY-87 00:00:00
7900,JAMES,CLERK,950.00,,03-DEC-81 00:00:00
7902,FORD,ANALYST,3000.00,,03-DEC-81 00:00:00
7934,MILLER,CLERK,1300.00,,23-JAN-82 00:00:00

366 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example Import from File
edb=# CREATE TEMP TABLE empcsv (LIKE emp);
CREATE TABLE
edb=# COPY empcsv (empno, ename, job, sal, comm, hiredate)
edb-# FROM '/tmp/emp.csv' CSV HEADER;
COPY
edb=# SELECT * FROM empcsv;
empno | ename | job | mgr | hiredate | sal | comm | deptno
-------+--------+-----------+-----+--------------------+---------+---------+--------
7369 | SMITH | CLERK | | 17-DEC-80 00:00:00 | 800.00 | |
7499 | ALLEN | SALESMAN | | 20-FEB-81 00:00:00 | 1600.00 | 300.00 |
7521 | WARD | SALESMAN | | 22-FEB-81 00:00:00 | 1250.00 | 500.00 |
7566 | JONES | MANAGER | | 02-APR-81 00:00:00 | 2975.00 | |
7654 | MARTIN | SALESMAN | | 28-SEP-81 00:00:00 | 1250.00 | 1400.00 |
7698 | BLAKE | MANAGER | | 01-MAY-81 00:00:00 | 2850.00 | |
7782 | CLARK | MANAGER | | 09-JUN-81 00:00:00 | 2450.00 | |
7788 | SCOTT | ANALYST | | 19-APR-87 00:00:00 | 3000.00 | |
7839 | KING | PRESIDENT | | 17-NOV-81 00:00:00 | 5000.00 | |
7844 | TURNER | SALESMAN | | 08-SEP-81 00:00:00 | 1500.00 | 0.00 |
7876 | ADAMS | CLERK | | 23-MAY-87 00:00:00 | 1100.00 | |
7900 | JAMES | CLERK | | 03-DEC-81 00:00:00 | 950.00 | |
7902 | FORD | ANALYST | | 03-DEC-81 00:00:00 | 3000.00 | |
7934 | MILLER | CLERK | | 23-JAN-82 00:00:00 | 1300.00 | |
(14 rows)

367 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - COPY Command on Remote Host

• COPY command on remote host using psql

$ cat emp.csv | ssh 192.168.192.83 “psql –U edbstore
edbstore -c ‘copy emp from stdin;’ “

368 © Copyright EnterpriseDB Corporation. All rights reserved.

 COPY FREEZE

• FREEZE is a new option in the COPY statement

• Add rows to a newly created table and freezes them
• Table must be created or truncated in current subtransaction
• Improves performance of initial bulk load
• Does violate normal rules of MVCC
• Usage:
=> COPY tablename FROM filename FREEZE;

369 © Copyright EnterpriseDB Corporation. All rights reserved.

EDB*Loader
 EDB*Loader

• EDB*Loader is a high-performance bulk data loader

• Supports Oracle SQL*Loader data loading methods:
• Conventional Control Discard
File
File

• Direct
Log File
Parallel
Data File EDB*Loader
•

Param Bad File

File
EPAS
Table

371 © Copyright EnterpriseDB Corporation. All rights reserved.

 Data Loading Methods
• Conventional path loading uses basic insert processing and is used to add rows to the
table
• Constraints, indexes and triggers are enforced during conventional path data loading
• Direct path loading is faster than conventional path loading, but is non-recoverable
• Direct path loading also requires removal of constraints and triggers from the table
• Conventional path data loading is slower than direct path loading, but is fully
recoverable
• A parallel direct path load provides even greater performance

372 © Copyright EnterpriseDB Corporation. All rights reserved.

 Invoking EDB*Loader

• Use the following command to invoke EBD*Loader from the command line:
edbldr [-d DBNAME] [-p PORT] [-c "CONNECTION_STRING"]
userid={dbuser[/dbpass]|/} direct={true|false} parallel={true|false}
control=control_file_name log=log_file_name
errors=num_errors
skip_index_maintenance={true|false}
skip=num_skips bad=bad_file_name parfile=par_file_name
freeze={true|false}
handle_conflicts={true|false}

373373 © Copyright EnterpriseDB Corporation. All rights reserved.

 The EDB*Loader Control File

• The Oracle SQL*Loader has compatible syntax for control file

directives
• The control file includes the instructions that EDB*Loader uses to
build the table (or tables) from the input file. It includes information
such as:
• The fully qualified name of the input file
• The name of the table or tables
• The name of the columns within the table or tables
• The delimiters or other selection criteria used to choose the column content
• The fully qualified names of the bad and discarded files

374374 © Copyright EnterpriseDB Corporation. All rights reserved.

 Control File Syntax
• The syntax for the EDB*Loader control file is as follows:
[ OPTIONS (param=value [, param=value ] ...) ]
LOAD DATA
[ CHARACTERSET charset ]
[ INFILE '{ data_file | stdin }' ] [ BADFILE 'bad_file' ] [ DISCARDFILE 'discard_file' ]
[ { DISCARDMAX | DISCARDS } max_discard_recs ]
[ INSERT | APPEND | REPLACE | TRUNCATE ] [ PRESERVE BLANKS ]
{ INTO TABLE target_table
[ WHEN field_condition [ AND field_condition ] ...] [ FIELDS TERMINATED BY 'termstring'
[ OPTIONALLY ENCLOSED BY 'enclstring' ] ] [ RECORDS DELIMITED BY 'delimstring' ] [ TRAILING
NULLCOLS ]
(field_def [, field_def ] ...)
} ...

375 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB*Loader Example
• This example loads data from a file named /tmp/mydata.csv into a table
named emp
• The data within the input file is delimited by a comma
• Create the control file:
LOAD DATA INFILE '/tmp/mydata.csv'
BADFILE '/tmp/mydata.bad'
DISCARDFILE '/tmp/mydata.dsc'
INSERT INTO TABLE emp
FIELDS TERMINATED BY ","
OPTIONALLY ENCLOSED BY '”' (empno, empname, sal, deptno)
• Run the edbldr command:
$ edbldr -d edb CONTROL=emp.ctl BAD=/tmp/emp.bad LOG=/tmp/emp.log SKIP=1 ERRORS=10

376376 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB*Loader Exit Codes

• EDB*Loader will return one of the following exit codes:

0 • Indicates that all rows loaded successfully

1 • EDB*Loader encountered syntax errors or aborted due to an unrecoverable error

2 • Load completed with some rejected or discarded rows

3 • Indicates EDB*Loaded stopped due to an OS error

377 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary

• Loading flat files

• Import and export data using COPY
• Examples of COPY Command
• Using COPY FREEZE for performance
• EDB*Loader
• Data Loading Methods
• Invoking EDB*Loader and Control File
• EDB*Loader Exit Codes

378378 © Copyright EnterpriseDB Corporation. All rights reserved.

 Lab Exercise - 1

• In this lab exercise you will demonstrate your ability to

copy data:

1. Unload the emp table from the edbuser schema to a csv file, with column headers

2. Create a copyemp table with the same structure as the emp table

3. Load the csv file (from step 1) into the copyemp table

379 © Copyright EnterpriseDB Corporation. All rights reserved.

Module 15
Replication and High Availability Tools
 Module Objectives
• Data Replication
• Data Replication in Postgres
• Streaming Replication and Architecture
• Synchronous, Asynchronous and Cascaded Replication
• Setup Streaming Replication
• Logical Replication Architecture
• Failover Manager
• Replication Manager (repmgr)
• Postgres-BDR
• Replication Server (xdb)

381 © Copyright EnterpriseDB Corporation. All rights reserved.

 Data Replication

• Replication is the process of copying data and changes to a

secondary location for data safety and availability
• Data loss can occur due to several reasons
• Replication is aimed towards availability of the data when a
primary source goes offline
• Data can be recovered from backup but downtimes are costly
• Replication aims towards lowering downtime
• Failovers can be configured to such a level where application
may not notice the primary source is offline

382 © Copyright EnterpriseDB Corporation. All rights reserved.

 Data Replication in Postgres

• Data replication options:

• Log-Shipping Standby Servers
• Streaming Replication
• Logical Replication
• Postgres-BDR
• EDB Replication Server
• Cluster management tools:
• High Availability Routing for Postgres(HARP)
• EDB Failover Manager
• Replication manager(repmgr)

383 © Copyright EnterpriseDB Corporation. All rights reserved.

 Streaming Replication

• Streaming Replication (Hot Standby) is a major feature of

Postgres
• Replica connects to the primary node using REPLICATION
protocol
• WAL segments are streamed to replica server
• No log shipping delays, stream WAL content across to replica
immediately
• Synchronous/Asynchronous options available
• Supports cascading replication

384 © Copyright EnterpriseDB Corporation. All rights reserved.

 Hot Streaming Architecture
Production Replica
WAL Sender WAL Receiver

WAL stream

Reports

Primary database
Replica
database

385 © Copyright EnterpriseDB Corporation. All rights reserved.

 Asynchronous Replication

• Streaming replication is asynchronous by default but can

be configured as synchronous
• Asynchronous
• Disconnected architecture
• Transaction is committed on primary and flushed to WAL segment
• Later transaction is transmitted to replica server(s) using stream
• Some data loss is possible
• Replication using WAL Archive method is always asynchronous

386 © Copyright EnterpriseDB Corporation. All rights reserved.

 Synchronous Replication

• Synchronous Replication
• A 2-safe replication method offering zero data loss
• Transaction must apply changes to primary and
synchronously replicated replicas using two-phase commit
actions
• User gets a commit message after confirmation from both
primary and replica
• This will introduce a delay in committing transactions

387 © Copyright EnterpriseDB Corporation. All rights reserved.

 Cascading Replication

• Streaming replication supports single Primary

master node
• Cascade replication can be used to share
the replication overhead of primary with
other replicas Replica 1

• Replica can stream changes to other Replica 2 Replica 3

Replicas
• Helps minimize inter-site bandwidth
overheads on primary node
• Asynchronous only

388 © Copyright EnterpriseDB Corporation. All rights reserved.

Setup Streaming Replication
 Primary Server Configuration

• For Physical Streaming Replication:

• Change WAL Content parameter:
• wal_level = replica #Default is replica
• Two options to allow streaming connection:
• max_wal_senders
• max_replication_slots
• Set only the minimum number of segments retained in pg_wal
• wal_keep_size = 1024

390 © Copyright EnterpriseDB Corporation. All rights reserved.

 Synchronous Streaming Replication Configuration

• Default level of Streaming Replication is Asynchronous

• Synchronous level can also be configured using additional
parameters:
synchronous_commit=on
synchronous_standby_names
• If the synchronous replica stops responding, then COMMITs will
be blocked forever until someone manually intervenes
• Transactions can be configured not to wait for replication by
setting the synchronous_commit parameter to local or
off

391 © Copyright EnterpriseDB Corporation. All rights reserved.

 Configure Authentication

• Authentication setting on the primary server must allow

replication connections from the replica(s)
• Provide a suitable entry or entries in pg_hba.conf with the
database field set to replication
• Open pg_hba.conf of primary server:
host replication all 192.168.56.2/32 md5

Note - You will need to reload the primary server

392 © Copyright EnterpriseDB Corporation. All rights reserved.

 Take a Full Backup of the Primary Server

• Backup the Primary Server using pg_basebackup:

pg_basebackup -h localhost -U postgres –p 5444
-D /backup/data1 -R

-R option
• creates a default copy of standby.signal file
• Add primary server connection info to postgresql.auto.conf

393 © Copyright EnterpriseDB Corporation. All rights reserved.

 Replica Configuration
hot_standby •Set this parameter to “ON” for read-only replica

primary_conninfo •Set connection string to connect with primary or cascaded replica

primary_slot_name •Specify replication slot name to be used for connection

promote_trigger_file •Set trigger file name for replica promotion

max_standby_streaming_delay •Duration for which replica has to wait during query conflicts

wal_receiver_create_temp_slot •Authorize WAL receiver process to be able to create a temporary replication slot

recovery_min_apply_delay •Parameter used for delayed replication

394 © Copyright EnterpriseDB Corporation. All rights reserved.

 Replica Recovery Settings

• Replica configuration settings must be set in

postgresql.conf or postgresql.auto.conf
• Create a file name standby.signal in the data directory
• standby.signal indicates the server should start as a
replica
• Last step - start the replica using system services or
pg_ctl

395 © Copyright EnterpriseDB Corporation. All rights reserved.

Logical Replication
 Logical Replication

• Logical replication is a method of

replicating selected data objects and
Subscription 1
Publication 1

their changes Publication 2 Subscription 2

• Based on publications and WAL

Sender
Standby
subscriptions Primary Logical
• Can be used to consolidate data Replication
Worker
Subscription 1

• Portable across hardware and Logical Subscription 2

software version
Replication
Launcher
Publication 3 Subscription 3
Reporting
• Tables on standby server which are Consolidate
part of a subscription must be treated Publication 4 Subscription 4 Upgrades

as read only to avoid conflicts Primary

397 © Copyright EnterpriseDB Corporation. All rights reserved.

 When to Use Logical Replication

• Sending incremental changes in a single database or a

subset of a database to subscribers
• Consolidating multiple databases into a single one
• Replicating between different major versions of Postgres
• Giving access to replicated data to different groups of
users
• Sharing a subset of the database between multiple
databases

398 © Copyright EnterpriseDB Corporation. All rights reserved.

 Setting Up Logical Replication
Change wal_level to logical in postgresql.conf

Add pg_hba.conf entry in each server to allow connection

Connect to database in publication instance

Create a publication using CREATE PUBLICATION statement

A published table must have a “replica identity” configured in order to be able to

replicate UPDATE and DELETE operations
Connect to database in subscription instance and create a subscription using CREATE
SUBSCRIPTION statement

399 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – Logical Replication Setup

• Initialize sample publication(primarypub) and

subscription(primarysub) instance
[postgres@localhost ~]$ initdb --version
[postgres@localhost ~]$ initdb –D primarypub –U pubdba
[postgres@localhost ~]$ initdb –D primarysub –U subdba
• Edit postgresql.conf parameter for both instances
[postgres@localhost ~]$ vi primarypub/postgresql.conf
• port=5420
• wal_level=logical
[postgres@localhost ~]$ vi primarysub/postgresql.conf
• port=5421
• wal_level=logical

400 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – HBA Entries and Starting Instances

• Add pg_hba.conf entries for connections

[enterprisedb@localhost ~]$ vi primarysub/pg_hba.conf
host all pubdba 192.168.56.101/32 md5
[enterprisedb@localhost ~]$ vi primarypub/pg_hba.conf
host all subdba 192.168.56.101/32 md5

• Start both instances

[enterprisedb@localhost ~]$ pg_ctl –D primarypub/ start
[enterprisedb@localhost ~]$ pg_ctl –D primarysub/ start

401 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – Create Tables and Publication

• Connect to default database in publication instance

[postgres@localhost ~]$ psql –p 5420 –U pubdba postgres

• Create a sample table and publication

=# CREATE TABLE pubexample(id INT PRIMARY KEY,
name VARCHAR(30));
=# INSERT INTO pubexample
VALUES(generate_series(1,5000),’Test1’);
=# SELECT count(*) FROM pubexample;
=# CREATE PUBLICATION testpub FOR TABLE pubexample;

402 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – Create Tables and Subscription

• Connect to default database in subscription instance

[postgres@localhost ~]$ psql –p 5421 –U subdba postgres

• Create a sample table and subscription

=# CREATE TABLE pubexample(id INT PRIMARY KEY,
name VARCHAR(30));
=# CREATE SUBSCRIPTION testsub CONNECTION
‘host=localhost port=5420 user=pubdba dbname=postgres’
PUBLICATION testpub;
=# SELECT count(*) FROM pubexample;

403 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example – Test Logical Replication

• Add data to publication

[postgres@localhost ~]$ psql –p 5420 –U pubdba postgres
postgres=# INSERT INTO pubexample
VALUES (generate_series(5001,10000),’Test1’);
postgres=# \q

• Check changes on Subscription

[postgres@localhost ~]$ psql –p 5421 –U subdba postgres
postgres=# SELECT count(*) FROM pubexample;

404 © Copyright EnterpriseDB Corporation. All rights reserved.

Monitoring Basics
 Monitoring Replication

• pg_stat_replication
• Show connected replicas and their status on the primary
• pg_stat_subscription
• Shows the status of subscription when using logical replication
• pg_stat_wal_receiver
• Shows the WAL receiver process status on Replica
• Recovery information functions:
• pg_is_in_recovery()
• pg_current_wal_lsn
• pg_last_wal_receive_lsn
• pg_last_xact_replay_timestamp()

406 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example - Monitoring Replication

• Execute:
=# SELECT * FROM pg_stat_replication;
• Find lag (bytes):
=# SELECT pg_wal_lsn_diff(sent_lsn, replay_lsn) FROM
pg_stat_replication;
• Find lag (seconds):
=# SELECT CASE WHEN pg_last_wal_receive_lsn() =
pg_last_wal_replay_lsn()
THEN 0 ELSE
EXTRACT (EPOCH FROM now() -pg_last_xact_replay_timestamp())
END AS stream_delay;

407 © Copyright EnterpriseDB Corporation. All rights reserved.

 Recovery Control Functions

Name Return Type Description

pg_is_wal_replay_paused() bool True if recovery is paused.

pg_wal_replay_pause() void Pauses recovery immediately.

pg_wal_replay_resume() void Restarts recovery if it was paused.

408 © Copyright EnterpriseDB Corporation. All rights reserved.

EDB Postgres Distributed -
Overview
 EDB Postgres Distributed
The most advanced replication solution for Postgres

Maintain extreme high Upgrade with Choose the level of

availability near zero downtime consistency
Postgres clusters deployed with Rolling upgrades of application Robust capabilities provide
EDB Postgres Distributed keep and database software flexibility to meet application
top tier enterprise applications eliminate the largest source of data loss requirements
running downtime

410 © Copyright EnterpriseDB Corporation. All rights reserved.

 Always ON
Top-tier enterprise applications are critical to an organization’s success in all
regions where business is conducted, whether a single region or globally

411 © Copyright EnterpriseDB Corporation. All rights reserved.

 BDR is more than bi-directional replication
Multi-master replication enabling highly available and geographically distributed
Postgres clusters
• Logical replication of data and schema
enabled via standard Postgres extension

• Data consistency options that span from

immediate to eventual consistency

• Robust tooling to manage conflicts, monitor

performance, and validate consistency

• Deploy natively to cloud, virtual, or bare

metal environments

• Geo-fencing, allowing selectively replicate

data for security compliance and jurisdiction
control.

412 © Copyright EnterpriseDB Corporation. All rights reserved.

 Postgres-BDR Features

Multi-Master
Synchronous or Flexible Always-ON
Row Level
Asynchronous Deployment DDL Replication DDL and Row Filters
Consistency
Replication for Architectures
Postgres

Configurable Conflict-free
Database Rolling
Parallel Apply Auto Partitioning Column-level Replicated Data
Upgrades
Conflict Resolution Types (CRDTs)

Transactions State Next Generation

Subscriber-only Open Telemetry
Tracking Across PGD CLI Connection Routing
Nodes Integration
Failovers using PGD-Proxy

413 © Copyright EnterpriseDB Corporation. All rights reserved.

 Deployment - Single Location

• Locations = 1, local redundancy = 3, nodes =

3, active locations = 1
• Global group with single data group of A1,
A2 and A3
• Lead Primary A1 receiving all writes but
changes can also be received by A2 and A2
• Shadow Primary A2, A3 receiving writes
• Can be 3 data nodes (recommended)
• Can be 2 data nodes and 1 witness that
doesn't hold data (not depicted)

414 © Copyright EnterpriseDB Corporation. All rights reserved.

 Example Deployment - Multiple Location

Locations = 2, local redundancy = 3, nodes = 3, active locations = 1

415 © Copyright EnterpriseDB Corporation. All rights reserved.

Replication Server Overview
 EDB Postgres Replication Server
Single Master Replication (SMR)
for Reporting or Migration

417 © Copyright EnterpriseDB Corporation. All rights reserved.

 Replication Server
Replicates between Postgres and non-Postgres databases

● Integrate with Oracle or SQL Server databases to offload

reporting or to feed data to legacy applications
● Flexibility to replicate a subset of data from the source
database
● Graphical user interface provides easy configuration and
management
● Includes utility to validate data consistency between the
source and target databases

418 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB Replication Server Features

• Replicate Oracle or SQL Server • Supports cascading replication

data to EDB Postgres Advanced
Server • Trigger and Log-based replication

• Distributed multi- • Snapshot and continuous modes

Publication/Subscription • Define and apply row filters
Architecture
• Flexible replication scheduler
• Synchronize data across
geographies • Replication History Viewer
• Replicate tables and sequences • Graphical Replication Console and
• Controlled switchover and failover CLI

419 © Copyright EnterpriseDB Corporation. All rights reserved.

Failover Manager Overview
 Why Failover Manager

Ensure business Maintain high Upgrade with

continuity availability minimal downtime

Monitor health databases and Meet your SLAs by switching Switchover on demand to move
identify failures quickly over to the most recent standby the primary to standby for
maintenance

421 © Copyright EnterpriseDB Corporation. All rights reserved.

 EDB Postgres Failover Manager
Automatically detect failures

• Monitors database health - detects

failures and takes action

• Automatically fails over to the most

current standby, reconfigures others

• Reconfigures load balancers on

failover - integrates with pgPool and
others

• Avoids “split brain” scenarios -

Prevents two nodes from thinking that
each is primary

422 © Copyright EnterpriseDB Corporation. All rights reserved.

 EFM Features

• Multiple health checks for Primary & Replica nodes

• Automatic Failover from Primary to Replica node
• Controlled switchovers for planned events on primary
• Configurable fencing operations
• User configurable failure detection wait times
• Witness node protects against ‘split brain’ scenarios
• Support for multiple streaming replicas
• Replica promotion based on WAL location and node priority
• Real-time notifications to chat rooms, SNMP and SMTP for all cluster
status changes

423 © Copyright EnterpriseDB Corporation. All rights reserved.

 Setup an EFM Cluster
Client Connection Pools and Load
Application Balancer
1. Set up Streaming replication
between the two servers
2. Install EFM
3. Configure the efm.properties Streaming Replication

file

Replication
Streaming
4. Start EFM
5. Add nodes to EFM cluster Primary Replica - 2
EFM Agent
6. Monitor the EFM and database EFM Agent

servers Replica - 1
EFM Agent

424 © Copyright EnterpriseDB Corporation. All rights reserved.

Replication Manager
Overview
 Replication Manager(repmgr)
Cluster Management tool for Postgres

Maintain high Perform upgrades Open Source

availability using switchovers

Automatic Failover to Add/remove replicas Open Source from

Replica in a Streaming and switchover of EnterpriseDB and
Replication Environment primary instance licensed under GPL

426 © Copyright EnterpriseDB Corporation. All rights reserved.

 repmgr Features

• Open source tool for managing replication and failover

• Supports Postgres Streaming Replication
• repmgr tool for setup:
• Add/remove replicas
• Perform switchovers
• Promote a replica
• repmgrd tool:
• Monitor replication
• Automatic failover detection with witness protection
• Email notification

427 © Copyright EnterpriseDB Corporation. All rights reserved.

 repmgr Architecture

Streaming Replication
Streaming Replication
Replica - 2
Replica - 1 Primary

repmgr user and metadata

repmgr
repmgr repmgr

repmgrd repmgrd repmgrd

428 © Copyright EnterpriseDB Corporation. All rights reserved.

 Module Summary
• Data Replication
• Data Replication in Postgres
• Streaming Replication and Architecture
• Synchronous, Asynchronous and Cascaded Replication
• Setup Streaming Replication
• Logical Replication Architecture
• Failover Manager
• Replication Manager (repmgr)
• Postgres-BDR
• Replication Server (xdb)

429 © Copyright EnterpriseDB Corporation. All rights reserved.

 Course Summary
• Introduction and Architectural • Creating and Managing Database
Overview Objects
• System Architecture • Database Security
• EDB Postgres Advanced Server • Monitoring and Admin Tools Overview
Installation • SQL Primer
• User Tools - Command Line Interfaces • Backup and Recovery
• Database Clusters • Routine Maintenance Tasks
• Database Configuration • Data Loading
• Data Dictionary • Data Replication and High Availability

430 © Copyright EnterpriseDB Corporation. All rights reserved.

 Next Steps
• Certify your Postgres skills with EDB Certifications for Postgres
• Continue your skills development with the following classes:
• Advanced Database Administration
• Monitoring and Alerting with Postgres Enterprise Manager
• Tuning and Maintenance
• See the Training Portal for the full library of Postgres training classes
• Get familiar with the EDB Tools available as part of the EDB Postgres Platform
• For any questions related to EDB Postgres Trainings and Certifications,
or for additional information, write to:
• [email protected]

431 © Copyright EnterpriseDB Corporation. All rights reserved.

THANK YOU

[email protected]
www.enterprisedb.com