Oracle GoldenGate Directories

Directory Contents
dirbdb * Berkeley DB for Monitor facility
dirchk Oracle GoldenGate checkpoint files
dircrd * Credential Store for users and passwords
dirdat Oracle GoldenGate trail and Extract files
dirdef Column definitions produced by defgen to translate heterogeneous data
dirjar Java executables to support OGG Monitor and other services
dirpcs Process status files
dirprm Parameter files
dirrpt Process report files
dirsql SQL scripts
dirtmp Temp storage for transactions that exceed allocated memory
dirwlt * Wallet for masterkey; works with dircrd
dirwww * Web artifacts for Monitoring
A few other directories are not shown. * New with 12c

Some key directories:

• dirchk: Contains the checkpoint files created by Extract and Replicat processes,
which
store current read and write positions to support data accuracy and fault tolerance.
These files are written in internal Oracle GoldenGate format and should not be modified.
The file name format is <group_name><sequence_number>.<ext>, where
<sequence_number> is a sequential number appended to aged files and <ext> is
either cpe for Extract checkpoint files or cpr for Replicat checkpoint files. Examples
include ext1.cpe and rep1.cpr.
• dircrd: The default Credential store location for encrypted user IDs and passwords.
You can change this location.
• dirdat: The default location for Oracle GoldenGate trail files and extract files created
by Extract processes to store records of extracted data for further processing, either by
the Replicat process or another application or utility. dirdat is written in internal
GoldenGate format and should not be modified.
The file name format is a user-defined two-character prefix followed by either a six-digit
sequence number (used by trail files; for example, rt000001) or the user-defined name
of the associated Extract process group (used by extract files; for example, finance).
dirdef: The default location for data definition files created by the defgen utility to
contain source or target data definitions used in a heterogeneous synchronization
environment. These files are written in plain text ASCII format.
The file name format is a user-defined name specified in the defgen parameter file (for
example, defs.dat). These files can be edited to add definitions for newly created
tables. If you are unsure how to edit a definitions file, contact technical support.
• dirjar: Contains the Java executable files that support Oracle GoldenGate Monitor
• dirpcs: The default location for status files. The file name format is
<group>.<ext>,
where <group> is the name of the group and <ext> is an extension of pce (Extract),
pcr (Replicat), or pcm (Manager).
These files are created only while a process is running. The file shows the program
name, the process name, the port, and process ID that is running. Do not edit these
files. Examples include mgr.pcm and ext.pce.
• dirprm: The default location for Oracle GoldenGate parameter files created by Oracle
GoldenGate users to store run-time parameters for Oracle GoldenGate process groups
or utilities. These files are written in plain text ASCII format.
The file name format is <group_name/user-defined_name>.prm or mgr.prm.
These files can be edited to change Oracle GoldenGate parameter values. They can be
edited directly from a text editor, such as gedit or Notepad, or by using the EDIT
PARAMS command in GGSCI. Examples include defgen.prm and finance.prm.
• dirrpt: The default location for process report files created by Extract, Replicat, and
Manager processes to report statistical information about a processing run. These files
are written in plain text ASCII format.
The file name format is <group_name><seq_num>.rpt, where <seq_num> is a
sequential number appended to aged files. Do not edit these files. Examples include
fin2.rpt and mgr4.rpt.
• dirsql: The default location for SQL scripts
• dirtmp: The default location for storing large transactions when the size exceeds the
allocated memory size. Do not edit these files.
• dirwlt: The default Oracle Wallet location for the masterkey; you can change this
location. Existing GUI Wallet utilities that come with other products, such as the Oracle
Database “Oracle Wallet Manager” (owm), do not work on this version of the Wallet.
• dirwww: Artifacts for Oracle Monitoring software; separately licensed
• jagent and jdk: Used for Java bits of the Monitor facility
GGSCI Commands
Objects
• Manager, Extract, Replicat: Oracle GoldenGate processes
• ER: Multiple Extract and Replicat processes
• ExtTrail: Local Extract trail
• RmtTrail: Remote trail
• TranData: Enables supplemental logging of Primary Keys and/or indexes for a
schema
or table in the transaction logs. Also, use SchemaTranData for all the present and
future tables in a schema.
• CheckpointTable: Checkpoint table (on target database)
• TraceTable: Oracle trace table (on target database)
Commands
• Add: Creates an object or enables TranData or SchemaTranData capture
• Alter: Changes the attributes of an object
• Cleanup: Deletes the run history of a process or removes records from a checkpoint
Table
Delete: Deletes an object or disables TranData or SchemaTranData capture
• Info: Displays information about an object (status, and so on)
• Kill: Forces a process to stop (no restart)
• Lag: Displays the lag between when a record is processed by the process and the
source
record timestamp
• Refresh: Refreshes Manager parameters (except port number) without stopping
Manager
• Send: Sends commands to a running process
• Start: Starts a process
• Stats: Displays statistics for one or more processes
• Status: Indicates whether a process is running
• Stop: Stops a process gracefully
Parameter Commands
• Set Editor: Changes the default text editor for the current GGSCI session from
Notepad (Windows) or vi (Linux) to any ASCII editor
• Edit Params: Edits a parameter file with an implied extension of .prm
• View Params: Displays the contents of a parameter file with an implied extension of
.prm
Database Commands
• DBLogin and MiningDBLogin: Establish a database connection through GGSCI to
a
local or remote database
• Encrypt Password: Encrypts a database login password in a parameter file
• List Tables: Lists all tables in the database that match a wildcard string (asterisks
and question marks)
DDL Commands
• DumpDDL: Saves the Oracle GoldenGate DDL history table to file
• Show: Displays the DDL information in standard output format
Miscellaneous Commands
• !command: Executes a previous GGSCI command without modification
• Create Subdirs: Creates default directories within the Oracle GoldenGate home
directory
• FC: Edits a previously issued GGSCI command
• Help: Displays information about a GGSCI command
• History: Lists the most recent GGSCI commands issued
• Info All: Displays the status and lag for all Oracle GoldenGate online processes on
a
system (Info All AllProcesses also displays tasks.)
• Obey: Runs a file containing a list of GGSCI commands
• Shell: Runs shell commands from within GGSCI
• Show: Displays the Oracle GoldenGate environment
• Versions: Displays OS and database versions
• View GGSEvt: Displays the Oracle GoldenGate event/error log
• View Report: Displays a process report for Extract or Replicat
Extract: Overview
• Extract captures all the changes that are made to objects
that you configure for synchronization.
• When a transaction is committed, Extract sends the data
for that transaction to the trail for propagation to the target
system.
• A primary Extract can be configured to:
– Capture changed data from database logs or archive logs
– Capture changed data from JMS
– Capture data directly from source tables for initial data load
– Send the data to be written to a local or remote trail or file
• A secondary Extract, called a data pump, can be
configured to distribute data from local trails to remote
systems.

Data Pump: Overview

• An Extract data pump can write to one or multiple remote
trails and/or servers (not to be confused with Oracle Data
Pump).
• A data pump:
– Reads the local trail
– Manipulates the data or passes it through without change
– Sends the data to one or more targets
• A data pump is useful:
– As a safeguard against network and target failures
– To break complex data filtering and transformation
into phases
– To consolidate data from many sources
– To synchronize one source with multiple targets

Setting Up Change Capture (Extract)

On the source system:
1. Add a primary Extract. This Extract reads from the
transaction logs located on the source and has an
associated parameter file.
2. (Optional) Add a local trail and a data pump Extract that
reads from the transaction logs located on the source and
has an associated parameter file.
3. Add a remote trail.
4. Start the Extract processes.
(The target system is a Replicat, which is covered later.)

Add Extract: Examples

GGSCI > DBLogin UserID myuser, Password mypswd
GGSCI 1> Add Extract finance, TranLog, Begin Now
EXTRACT added.
GGSCI 2> Add Extract atms, TranLog, Begin 2013-01-31 08:00
EXTRACT added.
GGSCI 3> Add Extract pump, ExtTrailSource c:\ggs\dirdat\lt
EXTRACT added.
GGSCI 4> Add Extract load, SourceIsTable
EXTRACT added.
GGSCI 5> Info All
All of these examples presume that a DBLogin has already been done.
1. Create an Extract group named finance that extracts database changes from the
transaction logs. Start extracting with records generated at the time that you add the
Extract group.
2. Create an Extract group named atms that extracts database changes from the
transaction logs. Start extracting with records generated at 8:00 AM on January 31,
2013.
3. Create a data pump Extract group named pump that reads from the Oracle
GoldenGate
Windows trail c:\ggs\dirdat\lt.
4. Create an initial-load Extract named load.
5. After you create all of these Extracts, it is customary to enter Info All to see if
they
are all present. They should be Status=STOPPED at this point.

Editing Extract Parameters

• To edit a parameter file for an online Extract group, issue
the following command:
• Then modify the following parameters in the file:
GGSCI> Edit Params <group name>
--Some Comment Goes Here.
Extract extwest
ExtTrail ./dirdat/ew
UserID ogguser@mysid, Password oracle_4U
--UserIDAlias oggalias
TranLogOptions ExcludeUser ogguser
Table WEST.*;
Table SALES.INVENTORY;

Oracle Wallet:
– Contains user IDs and passwords
– Is used in Extracts, ASM, and so
on
GGSCI> Create Wallet
GGSCI> Add CredentialStore
GGSCI> Alter CredentialStore Add User oggadmin
Password Welcome1 Alias oggalias
GGSCI> Info CredentialStore
GGSCI> DBLogin UserIDAlias oggalias
The Info command lists the user IDs and aliases, but not the passwords.
The following are the additional commands that are not shown in the slide:
GGSCI> Alter CredentialStore Replace User userid …
GGSCI> Alter CredentialStore Delete User userid …
Optionally, you can add “Domain mydom” at the end of each of the Alter commands
to
maintain multiple groups of credentials in the same wallet. As another option, the
userid
can include an Oracle SID in the form of @mysid as a suffix.
The parameter file on the previous page would now look like:
Extract extwest
ExtTrail ./dirdat/ew
UserIDAlias oggalias
TranLogOptions ExcludeUser ogguser
Table WEST.*;
Table SALES.INVENTORY;

Convert to Integrated:
• In the parameter file:
GGSCI> DBLogin UserIDAlias <useralias>
GGSCI> Register Extract my_capt Database
GGSCI> Stop Extract my_capt
GGSCI> Alter Extract my_capt, Upgrade Integrated Tranlog
GGSCI> Start Extract my_capt
--Extract my_capt
:
TranLogOptions IntegratedParams (max_sga_size 900, &
parallelism 3)
:

Adding a Local or Remote Trail

• Add a local or remote trail with the GGSCI command:
• If you are using a data pump:
– The primary Extract needs a local trail (ExtTrail)
– The data pump Extract needs a remote trail (RmtTrail)
GGSCI> Add ExtTrail | RmtTrail <trail_name>
, Extract <group_name>
[, Megabytes <n>]
GGSCI> Add ExtTrail c:\ggs\dirdat\aa,
Extract finance, Megabytes 10
GGSCI> Add RmtTrail c:\ggs\dirdat\bb,
Extract parts, Megabytes 5
On the local system, a trail is known as an Extract trail (or local trail). On a remote
system, it is
known as a remote trail. The primary Extract writes to a local trail. The data pump
Extract
sends changes to a remote trail. The default trail file size is 100 MB. In this example, the
parts Extract file is set to 5 MB.

Starting the Extract

• Start an Extract process with the GGSCI command:
GGSCI> Start Extract <group_name>
GGSCI> Info Extract <group_name>
• If the output trail is remote, this normally triggers the target
Manager process to start a Server Collector process with
default parameters.

Replicat: Overview
• The Replicat process runs on the target system.
• Multiple Replicat processes can be used with multiple
Extract processes in parallel to increase throughput.
• Replicat can:
– Read data out of Oracle GoldenGate trails
– Perform data filtering by table, row, or operation
– Perform data transformation
– Perform database operations just as your application
performed them
Oracle GoldenGate trails are temporary queues for the Replicat process. Each record
header
in the trail provides information about the database change record. Replicat reads these
trail
files sequentially and then processes INSERTs, UPDATEs, and DELETEs that meet your
criteria. Alternatively, you can filter out the rows that you do not want to deliver, as well
as
perform data transformation before applying the data.
Replicat supports a high volume of data-replication activity. As a result, network activity
is
block based, not record-at-a-time. Replicat uses native calls to the database for optimal
performance. You can configure multiple Replicat processes for increased throughput.
When replicating, Replicat preserves the boundaries of each transaction so that the
target
database has the same degree of integrity as the source. Small transactions can be
grouped
into larger transactions to improve performance. Replicat uses a checkpointing scheme,
so
changes are processed exactly once. After a graceful stop or a failure, processing can
be
restarted without repetition or loss of continuity.
Oracle GoldenGate 12c: Fundamentals for Oracle 7 - 5
Copyright