7/2/2019 Document 1304557.

1
Copyright (c) 2019, Oracle. All rights reserved. Oracle Confidential.

Main Note - Oracle GoldenGate - Lag, Performance, Slow and Hung Processes (Doc ID
1304557.1)

In this Document

Details
Actions
Contacts
References

APPLIES TO:

Oracle GoldenGate - Version 5.0.0 and later

Information in this document applies to any platform.

DETAILS

This note collects and categorizes notes and papers on identifying, quantifying and remediation issues associated with
perceived performance problems. This includes issues concerning processed described as slow, hung, having excessive lag,
and having performance issues.

ACTIONS

LAG is the elapsed time between when a transaction is written to a storage medium such as an archive log and the time when
Replicat writes the same transaction to the target database. Generally speaking, all rows within a transaction will have the
same lag. Exceptions can occur when a transaction is broken up and applied by multiple replicats or as multiple transactions.
Parameters such as RANGE are responsible for the first case. MAXTRANSOPS is responsible for the second case.

Lag is introduced by:

- Extract in reading the archive log and writing the data to a trail (or remote host),
- (optional) datapump reading the extract trail and writing (normally) to a remote host,
- network,
- collector (server.exe) on the target receiving network data and writing it to a local trail,
- replicat reading the local trail and writing to the database,
- database itself.

The status and info commands from ggsci show lag, as does the send <object>, lag command.
- The sum of these reported lags is the time OGG considers as LAG.
- Lag from an info command may differ from lag from a send command by small amounts.
- Lag from an info command is returned by the manager based on the last recorded checkpoint.
- Lag from a send command is returned by the <object> based on the row timestamp that the <object> is currently
processing.

Lag is expressed in time units and in kilobytes of data the <object> has to process yet. These generally correlate but one or
the other may be zero and the 'other' be a small but non-zero amount.

The lag is a measure of the difference between row archiving or writing and the time an <object> examines that row's
timestamp. It is not an indication of how long it will take an <object> to 'catch up'.
Millions of bytes and hours of lag can be caught up in minutes or less.

Because lag is based on checkpoints, in the case of Long Running Transactions (LRTs), a transaction may be started at a
given time and hours later not be committed. This transaction will be the oldest outstanding data and will not checkpoint
without a commit. This can cause an <object> processing a LRT to appear to be hours behind when the real issue is that the

https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=u7bffijsy_4&id=1304557.1 1/3
7/2/2019 Document 1304557.1

LRT has yet to commit. In a replicat, this issue can be diminshed (with transaction integrity consequences) by adding
MAXTRANSOPS to the replicat parameter file.

Reference notes:

Note 1273285.1 How To Troubleshoot Oracle Redo Log Reading Extract Slow Performance Issue
Note 969639.1 What Information Does Support Need to Diagnose a Replicat Performance Issue on Open Systems
Note 962592.1 Excessive REPLICAT LAG Times
Note 968710.1 Using A Heartbeat Table To Monitor LAG Between The Source And Target.
Note 1071892.1 Excessive LAG on Data Pump Sending Data Over WAN
Note 968614.1 Why Does GoldenGate Report The Lag Is Unknown?
Note 964705.1 Extract RBA Not Moving And LAG Increasing And Appear Hung
Note 1299679.1 OGG - Heartbeat process to monitor lag and performance in GoldenGate

Note 1478958.1 Extract On Restart While In Recovery Runs Slower Using Lots Of Virtual Memory / Disk File Caching
Note 1493225.1 Extract Is Very Slow In Processing the records
Note 1518793.1 Classic Extract runs slower when there are lots of row chaining
Note 1501813.1 Extract (11.2.1.0.2 or 11.2.1.0.3) is Slow In Restarting
Note 1564093.1 GoldenGate Extract Slow when executing on RAC
Note 1490270.1 Why is my VAM or other extract slow or hung or not checkpointing?
Note 1487557.1 Integrated Extract runs slower upon restart during Recovery and status shows "In recovery[1]: At E
Note 1256884.1 Catalog Data Extract - Items Still Slow After Upgrading to Latest Code Version
Note 1337637.1 OGG Extract in Version 11.1.1.1 is Slow Until Restart
Note 1363266.1 OGG How to address Extract Performance Issue When Reading Archive Logs stored in Oracle ASM
Note 1356524.1 How to estimate Goldengate extract redo processing speed?
Note 1337637.1 OGG Extract in Version 11.1.1.1 is Slow Until Restart
Note 1329640.1 Why replicat (or pump) hangs after upstream pump (or extract) had ETROLLOVER?
Note 1432994.1 OGG Extract Hangs on AIX When Reading Through DBLOGREADER
Note 1192972.1 ALO Extract Hangs and on restart an error 22 (Invalid argument) occurs
Note 1555982.1 OGG Extract Hangs on ASM archived logs with sector_size=4k
Note 1381055.1 GoldenGate NSK Extract Hangs in Starting State And is Not Moving
Note 1343063.1 OGG Extract is Hung Stuck on Archive Log Extseqno 0 Extrba 0 and can't be Al
Note 1528401.1 Classic Extract - Extract Hang Using 100% CPU
Note 1358904.1 Extract Is Running But Stuck At Same RBA And Send Commands Gives Timeout Waiting For Message
Note 1362390.1 Goldengate Extract Stuck On 'Bad' redo log, But Chekcpoint Got Update
Note 1467942.1 Unable to start additional GoldenGate processes after several already started. Stuck in "Starting

Would you like to explore this topic further with other Oracle Customers, Oracle Employees, and
Industry Experts?

Click here to join the discussion where you can ask questions, get help from others, and share your experiences
with this specific article.
Discover discussions about other articles and helpful subjects by clicking here to access the main My Oracle
Support Community page for GoldenGate, Streams and Distributed Database.

CONTACTS

REFERENCES

NOTE:1273285.1 - How To Troubleshoot Oracle Redo Log Reading Extract Slow Performance Issue using TESTMAPPINGSPEED
NOTE:1071892.1 - Excessive LAG on OGG Extract Data Pump Sending Data over WAN, Calculate TCP TCPBUFSIZE
TCPFLUSHBYTES
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=u7bffijsy_4&id=1304557.1 2/3
7/2/2019 Document 1304557.1

NOTE:964705.1 - OGG Extract RBA Not Moving And LAG Increasing And Appears Hung
NOTE:968710.1 - Using A Heartbeat Table To Monitor LAG Between The Source And Target.
NOTE:962592.1 - Excessive REPLICAT LAG Times
NOTE:1299679.1 - Oracle GoldenGate Best Practices: Heartbeat Table for Monitoring Lag times
NOTE:968614.1 - Why Does GoldenGate Report The Lag Is Unknown or a Timestamp Mismatch?
NOTE:969639.1 - What Information Does Support Need to Diagnose a Replicat Slow or Performance Issue on Open Systems
Didn't find what you are looking for?

https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=u7bffijsy_4&id=1304557.1 3/3