Oracle Scene 63 N Chandler Getting Started With Oracle Goldengate
Oracle Scene 63 N Chandler Getting Started With Oracle Goldengate
SPRING 17
Technology
Whatever your reasons for moving a test environment. For the purposes Architectural Overview
data; migrating to the Cloud or to a of this introduction, I will concentrate GoldenGate consists of several processes.
new system/platform, feeding a Data on showing how we can setup a simple
Warehouse, performing an upgrade Oracle-to-Oracle Master-to-Slave The EXTRACT process connects to the
with minimal or zero downtime, or replication of an entire schema. database, captures transactions and writes
even implementing a bespoke Business This example replication will be done on the transactions to a TRAIL FILE. The TRAIL
Continuity system where only a fraction Oracle VM VirtualBox “Developer Days” FILE can either be local to the EXTRACT to
of the data is required for DR, GoldenGate Servers, downloadable here: be used by a DATAPUMP, or directly sent to
will allow you to implement data a remote destination server.
movement across platforms and different http://www.oracle.com/technetwork/
storage engines easily and quickly, with database/enterprise-edition/ The DATAPUMP process is used to pick
built-in resilience. databaseappdev-vm-161299.html up transactions which have been written
to a local TRAIL FILE and sends them to a
GoldenGate is a platform independent
data extraction, transformation and
load tool (ETL). I have used it to reliably
replicate data from Mainframe SQL/
MX databases, transform it into Oracle,
and modify and transform it again into
SQL Server, as well as designing and
implementing one-to-one and one-
to-many data migrations and feeds
on Oracle-centric systems. I have also
implemented a multi-master ACTIVE/
ACTIVE solution.
Basic uni-directional replication, which
will encompass most GoldenGate
implementations, is straightforward
to setup. The best way to understand
GoldenGate is to install it and use it in
46 www.ukoug.org
Technology: Neil Chandler
$ cd /home/oracle/install
$ unzip fbo_ggs_Linux_x64_shiphome.zip
$ cd /home/oracle/install/fbo_ggs_Linux_x64_shiphome/Disk1/ogg.rsp
$ cat ogg.rsp
#-------------------------------------------------------------------------------
# Do not change the following system generated value.
#-------------------------------------------------------------------------------
oracle.install.responseFileVersion=/oracle/install/rspfmt_ogginstall_response_schema_v12_1_2
$./runInstaller -silent -responseFile /home/oracle/install/fbo_ggs_Linux_x64_shiphome/Disk1/ogg.rsp
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 23701 MB Passed
Checking swap space: must be greater than 150 MB. Actual 2063 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2017-01-07_01-30-23PM. Please wait ...
You can find the log of this install session at:
/home/oracle/app/oraInventory/logs/installActions2017-01-07_01-30-23PM.log
FIGURE 1
The REPLICAT process reads the TRAIL FILE on the destination Initial Configuration
server and applies the change records to the destination First of all we need to configure some global settings, directories
database. and the GoldenGate Manager on the source and target servers.
We should create a GLOBALS file in the GoldenGate Home
The MANAGER process looks after all of the other processes, installation directory /home/oracle/app/goldengate. The
and can start and restart them automatically. It also cleans up GLOBALS file is read each time we use the GoldenGate
old TRAIL FILES and listens on a TCP port (7809) for incoming Command Interpreter “ggsci” and contains parameters which
connections from source EXTRACT and DATAPUMP processes. apply to the entire GoldenGate instance.
www.ukoug.org 47
OracleScene
SPRING 17
Technology: Neil Chandler
dirdat – this is the default directory for all of the trail files contain a unique identifier, and you are able to modify the
produced by the extract and datapump/collector processes. schema, that a surrogate key column be added and populated.
The files in this directory will contain all of the data for every It can be a simple population using a DEFAULT sequence
transaction which is replicated, and so we need to ensure it has next-value to capture new values automatically ( e.g. create
sufficient size and performance resources sequence <table_seq>; alter table <table> add
surrogate_unique_col default <table_seq>.
To complete the basic setup, we need to create the Manager nextval).
parameter file and start the Manager.
There are a few other schema-based problems which may need
$ ggsci
to be overcome, such as deferred constraints. There is a script
Oracle GoldenGate Command Interpreter for Oracle within MOS article 1296168.1 which performs a check of your
schema for replication compliance and provides some advice
GGSCI 1 > create subdirs
Creating subdirectories under current directory /home/oracle/ and metrics too.
app/goldengate
48 www.ukoug.org
Technology:Neil Chandler
In the Database – GoldenGate accounts The EXTRACT parameter file looks like this:
There needs to be a GoldenGate account in both the source and
the target databases. The Source EXTRACT will be mainly reading -- /home/oracle/app/goldengate/dirprm/e_hr.prm
from the in-memory REDO stream to capture transactions.
The target REPLICAT will be playing those transaction into the -- Setup Environment Variables so we login to the database
correctly
database. SETENV (ORACLE_HOME=’/home/oracle/app/oracle/product/12.1.0/
dbhome_1’)
SETENV (ORACLE_SID=’cdb1’)
The source database consists of a container database called
“cdb1” and a pluggable database called “orcl” (this pre-exists in -- Name the extract
the downloaded VM image). EXTRACT e_hr
the EXTRACT, with GoldenGate-specific and PDB-level privileges: -- Name the Trail File. We only get 2 characters!
EXTTRAIL ./dirdat/AA
SYS@cdb1 > create user c##goldengate identified by goldengate ; -- Makes trail files smaller and helps with primary key updates
User created. UPDATERECORDFORMAT COMPACT
NOTE: If you are not using Pluggable Databases, setup both GGSCI 2> edit param p_hr
GoldenGate users the same as the target GoldenGate user in the GGSCI 3> add extract p_hr, exttrailsource /home/oracle/app/
PDB. goldengate/dirdat/AA
EXTRACT added.
www.ukoug.org 49
OracleScene
SPRING 17
Technology: Neil Chandler
The DATAPUMP parameter file looks like this: HANDLECOLLISIONS parameter in the REPLICAT. This temporary
parameter may be used to get your REPLICAT started should
-- /home/oracle/app/goldengate/dirprm/p_hr.prm
you have transactions in your TRAIL FILE which are already in
the database, and it endeavours to align your source and target
-- Setup Environment Variables so we login to the database using some sensible rules to handle clashes e.g. if a record to
correctly
SETENV (ORACLE_HOME=’/home/oracle/app/oracle/product/12.1.0/ be deleted does not exist, just ignore the fact it is not there
dbhome_1’) as it will have the same outcome as if we deleted it. However
SETENV (ORACLE_SID=’cdb1’)
HANDLECOLLISION does not cope with all potential scenarios
-- Name the datapump. Note that its’ really a special type of and it should be switched off [NOHANDLECOLLISIONS] as soon
extract as possible after initial REPLICAT synchronisation otherwise it
EXTRACT p_hr
may slowly corrupt the target dataset.
-- Add our standard reporting options for every extract and
replicat
include /home/oracle/app/goldengate/dirprm/i_report.prm
Setup the REPLICAT
-- Specify where the trail file is being transmitted-to First of all we need to check that the GoldenGate DATAPUMP
RMTHOST DevDaysTargetGG , MGRPORT 7809
RMTTRAIL /home/oracle/app/goldengate/dirdat/AA is transmitting changes to the target server by looking for
the TRAIL FILE. Running multiple ls -l commands will show if
-- If you are not doing any transformation in the datapump
-- this parameter increases performance by up to 30%
transactions are being transmitted as the TRAIL FILE grows:
PASSTHRU
If you are unable to ensure definitive extract points, such GGSCI 7> start r_hr aftercsn 6302652
as when you are using flat files extract and load to perform
Sending START request to MANAGER ...
an initial population, it may be necessary to use the
50 www.ukoug.org
Technology: Neil Chandler
MANAGER RUNNING
REPLICAT RUNNING R_HR 00:00:00 00:00:02 REPLICAT r_hr STATS
The REPLICAT parameter file looks like this: GGSCI 1> stats r_hr, total
-- Add our standard reporting options for every extract and *** Total statistics since 2017-01-11 15:04:59 ***
replicat Total inserts 3.00
include /home/oracle/app/goldengate/dirprm/i_report.prm Total updates 0.00
Total deletes 0.00
-- Controlling REPLICAT memory use and parallelism Total discards 0.00
DBOPTIONS INTEGRATEDPARAMS (max_sga_size 200, parallelism 1) Total operations 3.00
-- Key file used to show failed records. Needed when Replicating from ORCL.HR.JOB_SUBTASKS to ORCLTARGET.HR_TARGET.
troubleshooting problems. JOB_SUBTASKS:
DISCARDFILE ./dirrpt/p_orcl2.dsc, PURGE
*** Total statistics since 2017-01-11 15:04:59 ***
-- This is how we map the tables across from source to target Total inserts 93605.00
MAP orcl.hr.*, TARGET hr_target.*; Total updates 0.00
Total deletes 0.00
Total discards 0.00
And the standard reporting i_report.prm file we have included in Total operations 93605.00
End of Statistics.
every group parameter file looks like this:
GGSCI 1> stats e_hr, total Row “IT_DBA” was inserted at SCN: 6284117 and therefore
captured by the EXTRACT, but not needed as the Export
Sending STATS request to EXTRACT E_HR ...
Start of Statistics at 2017-01-11 15:37:00. Datapump was executed with SCN: 6302652.
www.ukoug.org 51
OracleScene
SPRING 17
Technology: Neil Chandler
Blog: https://chandlerdba.com
www.linkedin.com/in/nchandler
@ChandlerDBA
52 www.ukoug.org