Online Patching (ADOP) in Oracle EBS

R12.2
Posted on February 21, 2016 by Brijesh Gogia
Online patching is supported by the capability of storing multiple application editions in
the database, and the provision of a dual application tier file system. At any given point
in time, one of these file systems is designated as run (part of the running system) and
the other as patch (either being patched or awaiting the start of the next patching cycle).

For applying a patch in R12.2 you need to use adop and run through all below phases
in sequence mentioned below.

1) adop phase=prepare
2) adop phase=apply patches=<patch_number1>,<patch_number2>
workers=<number_of_worker>
3) adop phase=finalize workers=<number_of_worker> (called automatically)
4) adop phase=cutover workers=<number_of_worker>
5) adop phase=cleanup (called automatically)

OR

Running all phases in single command:

adop phase=prepare,apply,finalize,cutover,cleanup
patches=<patch_number1>,<patch_number2>
——————————————————————————————————————–

DESCRIPTION OF EACH PHASE

1) PREPARE PHASE DETAILS

Used to start a new online patching cycle

How to execute:
A) Set the environment by executing (sourcing) the run file system environment file:
$ source <EBS install base>/EBSapps.env run
B) Verify envirionment
You can confirm that the environment is properly set by examining the relevant
environment variables:

$ echo $FILE_EDITION
run
$ echo $TWO_TASK
dbSID

C) Download Patches

Download patches to be applied and place then in the $PATCH_TOP directory of your
system. This directory is pre-created by the install in the non-editioned file system
(fs_ne) and should not be changed.

Important: On a multi-node system with non-shared file systems, you must copy the
patch files to each separate $PATCH_TOP directory, so that the patch files are
available from the same location on all nodes.
D) Unzip the patch

$ unzip <patch>.zip

E) Run Prepare Command

Prepare the system for patching by running the following command to start a new
patching cycle:

$ adop phase=prepare

What it will do:

• Checks whether to perform a cleanup, which will be needed if the user failed to invoke
cleanup after the cutover phase of a previous online patching cycle.
• Checks the integrity of the database data dictionary. If any corruption is found, adop
exits with an error.
• Checks system configuration on each application tier node. A number of critical
settings are validated to ensure that each application tier node is correctly registered,
configured, and ready for patching.
• Checks for the existence of the “Online Patching In Progress” (ADZDPATCH)
concurrent program. This program prevents certain predefined concurrent programs
from being started, and as such needs to be active while a patching cycle is in progress
(that is, while a database patch edition exists). If the ADZDPATCH program has not yet
been requested to run, a request is submitted.
Note: ADZDPATCH is cancelled later on when the cutover phase is complete.
• Checks to see if the patch service has been created. adop requires that a special
database service exists for the purpose of connecting to the patch edition. This service
is created automatically, but its continued existence is validated on each prepare.

It can be checked by the database parameter SERVICE_NAME

SQL> show parameter service_name

NAME TYPE VALUE
------------------------------------ ----------- ------------
---
service_names string dba, ebs_patch

Here dba is the SID of our database and ebs_patch is additional service_name which is
required by online patching tool.

If you look at tnsnames.ora file in the Application tier $TNS_ADMIN directory you will
find below kind of entry:

<SID>_patch=
(DESCRIPTION=

(ADDRESS=(PROTOCOL=tcp)(HOST=<your_database_server_name>)(PORT
=<your_database_port>))
(CONNECT_DATA=
(SERVICE_NAME=ebs_patch)
 (INSTANCE_NAME=<your_database_SID>)
)
)

During patching phase, adop will use this tns entry to connect to database.

• Invokes the TXK script

$AD_TOP/patch/115/bin/txkADOPPreparePhaseSynchronize.pl to synchronize the
patches which have been applied to the run APPL_TOP, but not the patch APPL_TOP.
• Checks the database for the existence of a patch edition, and creates one if it does not
find one.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++

2) APPLY PHASE DETAILS

In the apply phase, adop applies the specified patches to the system. Patches are
applied to the patch edition of the database and file system.

How to execute:

Example:

$ adop phase=apply patches=1234,7891 workers=8

Where 1234 and 7891 are the patch numbers

What it will do:

If a post-installation patch step mentions any tasks that need to be performed explicitly,
where they are run from depends on the type of patching:

• In a normal online patching cycle, the steps should be executed from the patch file
system after the apply phase.
• If the patch is being applied in hotpatch mode or downtime mode, the steps should be
executed from the run file system after the apply phase.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++

3) FINALIZE PHASE DETAILS

The finalize phase will be executed while the application is still online. It is used to
perform any remaining processing that is needed to ensure the system is ready for the
fastest possible cutover.

Used to perform the final patching operations that can

How to execute:

$ adop phase=finalize

What it will do:

• Pre-compute DDL that needs to be run at cutover.

• Compile all invalid objects.
• Validate that the system is ready for cutover.
If finalize_mode=full, compute statistics for key data dictionary tables for improved
performance.

VERY IMPORTANT 1 : Up to this phase, you can run a special phase called abort,
which will undo the changes made so far in the patching cycle. After cutover is
complete, however, you cannot do this.

VERY IMPORTANT 2 : In an online patching cycle, the requisite JAR files are initially
stored in the $APPL_TOP/admin/<SID>/out directory, and then uploaded into the
database during the cutover phase. Therefore, the out directory must not be deleted at
least until cutover (next phase) is complete.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++

4) CUTOVER PHASE DETAILS

Used to perform the transition to the patched environment. Shuts down application tier
services, makes the patch edition the new run edition, and then restarts application tier
services. This is the only phase the involves a brief downtime.
Important: No users should remain on the system during cutover, as there will be a
short downtime period while the application tier services are restarted. Also, any third-
party processes connected to the
old run edition of the database should be shut down, or they will be terminated
automatically.

How to execute:

$ adop phase=cutover

What it will do:

• Shut down internal concurrent manager. The adop utility signals the internal
concurrent manager to shut down, but will wait for any existing concurrent requests to
finish before it proceeds with cutover actions.

Note: Cutover will take longer if it has to wait for long-running concurrent processes to
complete. In such a case, you can expect to see an informational message of the form:
[STATEMENT] [END 2013/10/28 23:47:16] Waiting for ICM to go downIf you do not
want to wait for in-progress concurrent requests to finish normally, you can terminate
the internal concurrent manager by executing the adcmctl.sh abort command from a
different shell.

• Shut down application tier services: All application tier services are brought down.
During this period, the system is unavailable to users.
• Cutover database: Promote patch database edition to become the new run database
edition, using adzdpmgr.pl script.
• Cutover file system: Promote patch file system to become the new run file system,
switching the $FILE_EDITION values in the patch and run enviroments. The current
patch APPL_TOP becomes the new run APPL_TOP, and the current run APPL_TOP
becomes the new patch APPL_TOP. Terminate old database sessions: Terminate any
database connections to the old run edition of the database.
• Start application tier services: Application tier services are restarted, on the new run
edition. The system is now available again to users
• ADZDPATCH concurrent program is cancelled when the cutover phase is complete.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++

5) CLEANUP PHASE DETAILS

Important: If you fail to run the cleanup phase explicitly, it will be run automatically on
the next prepare cycle, but this will cause a delay in starting your next online patching
cycle.

This adop phase is used to remove obsolete code and data from old editions.

How to execute:

$ adop phase=cleanup

What it will do:

• Various actions are performed during cleanup, including dropping (removing) obsolete:
Crossedition triggers, Seed data, Editioned code objects (covered objects), Indexes,
Columns, Editions.
Using parameter cleanup_mode:

a) cleanup_mode=quick – Performs minimum cleanup, including removal of obsolete

crossedition triggers and seed data.

Use quick cleanup when you need to start the next patching cycle as soon as possible.
For example, if you want to start a new patching cycle right away, but have not yet run
cleanup from the previous patching cycle, you can use quick cleanup mode to complete
the essential cleanup tasks as fast as possible.

b) cleanup_mode=standard – Does the same as quick mode, and also drops (removes)
obsolete editioned code objects (covered objects).

This is the default mode , so does not need to be specified.

c) cleanup_mode=full – Performs maximum cleanup, which drops all obsolete code and
data from earlier editions

Use full cleanup when you want to recover the maximum amount of space in the
database. If you have run a large number of patching cycles, or applied a very large
patch such as a rollup, significant space may be consumed by obsolete table columns
and recovered by running a full cleanup. A full cleanup should only be performed when
there is no immediate need to start a new patching cycle.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++

THERE ARE TWO SPECIAL PHASES:

A) ABORT PHASE DETAILS

Abort PHASE is conditional phase. This phase cannot be specified with any other
phase.

If for some reason either the prepare or apply phase failed or gave problems, you can
abort the patching cycle at either of these points by running a special phase with the
Command. The actions taken will be discarded (rollbacked).

IMPORTANT: This abort command is only available up to (but not including) the cutover
phase. After cutover, the system is running on the new edition, and abort is no longer
possible for that patching cycle.
How to execute:

The command to perform this operation is:

$ adop phase=abort

What it will do:

• Confirms that there is an in-progress online patching cycle, so the abort call is
therefore valid.
• Checks for the existence of a patch edition and drops one if it exists.
• Cancels the ADZDPATCH concurrent program, if it is running.
• Deletes the rows inserted for the pending session ID from the ad_adop_sessions and
ad_adop_session_patches tables.

VERY IMPORTANT: After running abort, a full cleanup must be performed. The cleanup
command is: adop phase=cleanup cleanup_mode=full). This will remove any columns
that were added by the patch but are no longer needed because of the abort. If they are
not removed, they may cause problems in a later patching cycle.

Alternatively, you can run a combined command to abort the patching cycle and perform
a full cleanup:
 $ adop phase=abort,cleanup cleanup_mode=full

If any attempt was made to apply patches to the patch edition, after abort you must run
the fs_clone phase (adop phase=fs_clone) to recreate the patch file system.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++

B) FS_CLONE PHASE DETAILS

The fs_clone phase is a command (not related to adcfgclone.pl) that is used to

synchronize the patch file system with the run file system. The fs_clone phase should
only be run when mentioned as part of a specific documented procedure.

How to execute:

The fs_clone phase is run using the following command:

$ adop phase=fs_clone

What it will do:

This phase is useful if the APPL_TOPs have become very unsynchronized (meaning
that there would be a large number of delta patches to apply). It is a heavyweight
process, taking a backup of the entire current patch APPL_TOP and then cloning the
run APPL_TOP to create a new patch APPL_TOP. As this method requires more time
and disk space, it should only be used when the state of the patch file system is
unknown. This command must be invoked from the run file system, before the next
prepare phase is run.

Note: The patch file system requires at least 25 GB of free disk space to be available for
adop operations, including fs_clone. If there is insufficient free space, the adop
operation will fail.

If an fs_clone operation fails, you can rerun it with the option force=yes to restart it from
the beginning (with the same session ID), or force=no to restart it from the point where it
failed.

——————————————————————————————————————–
IMPORTANT POINTS REGARDING ONLINE PATCHING:

1. adop utility is put under $APPL_TOP_NE/ad/bin. It is a wrapper script which calls

internally the perl script $AD_TOP/bin/adzdoptl.pl which does actual work of applying
the patch.

2. adop will automatically set its environment as required, but it is the user’s
responsibility to set the environment correctly for any other commands that may be run.
Set the run edition environment whenever executing commands that you intend to affect
the run edition.

For example:

$ . <EBS_ROOT>/EBSapps.env run
$ adstrtal.sh

Set the patch edition environment whenever you intend to execute commands that
affect the patch edition.
For example:

$ . <EBS_ROOT>/EBSapps.env patch
$ sqlplus apps/apps @my_custom_patch_script.sql

3. All the phases need to be completed and you can’t skip any of these. For example, if
you try to skip prepare phase, you may get error message like “Apply phase can only be
run while in a patching cycle, i.e. after prepare phase.”
4. After an online patching cycle is started, you should not perform any configuration
changes in the run edition file system. Any that are made will not be propagated and will
therefore be lost after cutover is complete.
5. You should not attempt to clone an Oracle E-Business Suite system while an online
patching cycle is in progress.
6. The prepare, apply, and fs_clone phases all require at least 10GB of free disk space.
All other phases require 1GB of free space. A warning message will be displayed if less
than the needed amount is available.
7. The directories where you extracted the patches applied in a given patching cycle
must be retained, in the same location and with the same contents, until the next
prepare phase completes. This is also a requirement for patches applied in a hotpatch
session.
8. Maintenance Mode is not needed for online patching, and so Maintenance Mode is
not available in Oracle E-Business Suite Release 12.2.
——————————————————————————————————————-

ADOP ON MULTI-NODE

In a multi-node environment, one application tier node will be designated as the primary
node. This is the node where the Admin Server is located, and will usually also be the
node that runs Oracle HTTP Server. All other application tier nodes are designated as
secondary nodes.

adop commands are invoked by a user on the primary node. Internally, adop uses
Secure Shell (ssh) to automatically execute required patching actions on all secondary
nodes. You must set up passwordless ssh connectivity from the primary node to all
secondary nodes.

If a node unexpectedly becomes inaccessible via ssh, it will be abandoned by adop, and
the appropriate further actions taken. Consider a scenario where the adop
phase=prepare command is run in a system with ten application tier nodes. The
command is successful on nine nodes, but fails on the tenth. In such a case, adop will
identify the services enabled on nodes 1-9. If they are sufficient for Oracle E-Business
Suite to continue to run normally, adop will mark node 10 as abandoned and then
proceed with its patching actions. If they are not sufficient, adop will proceed no further.
——————————————————————————————————————–
Patch 24576605 in CVQA

applebs@cvapq01 patch$ adop phase=prepare

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Please wait. Validating credentials...

RUN file system context file:

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

PATCH file system context file:

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

Execute SYSTEM command : df /EQ02/CVQA/fs2

Worker count determination...

Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.

[EVENT] [START 2017/03/06 23:22:40] Performing database sanity checks

[WARNING] The following required database bug fixes <UPGRADE REQUIRED>

are missing from this node, <cvdbq01>.

Refer to My Oracle Support Knowledge Document 1594274.1 to identify the patch that delivers
this bug fix.

[EVENT] [END 2017/03/06 23:22:41] Finished performing database sanity checks

[START 2017/03/06 23:22:42] adzdoptl.pl run

ADOP Session ID: 39

 Phase: prepare

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170306_232145.log

[EVENT] [START 2017/03/06 23:22:42] Calling task: "Executing txkADOPValidation script on

cvapq01"; script: "/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl";
args: " -contextfile=/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
patchctxfile=/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
phase=prepare -
logloc=/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01 -
promptmsg=hide"

[EVENT] [START 2017/03/06 23:22:42] Executing txkADOPValidation script on cvapq01

[EVENT] Calling: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Oracle E-Business Suite Online Patching

RUN edition environment variables have been set.

calling: /EQ02/CVQA/fs1/FMW_Home/webtier/perl/bin/perl
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkValidateSetup.pl -
contextfile=/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
patchctxfile=/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
logloc=/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01 -
phase=prepare -promptmsg=hide

Program : txkValidateSetup.pl started @ Mon Mar 6 23:22:45 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/ADOPValidation
s_Mon_Mar_6_23_22_45_2017.log

SETUP VALIDATION is in progress. This may take few minutes to complete.

Completed execution of all tests. There are warnings.

=========================== START OF VALIDATION REPORT ====================

Validation Results for Node: cvapq01

---------------------------------------

[WARNING]: There could be issues while validating the ports used for E-Business Suite instance against
ports used in /etc/services. Refer the log file for more details.

[WARNING]: Either some of the required entries in /etc/hosts file might be missing (e.g. localhost or
hostname) OR the file /etc/hosts could not be read.

There are warnings in tests, please check

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/ADOPValidation
s_Mon_Mar_6_23_22_45_2017.log

=========================== END OF VALIDATION REPORT ======================

[EVENT] [END 2017/03/06 23:23:52] Executing txkADOPValidation script on cvapq01

[EVENT] [END 2017/03/06 23:23:52] Executing txkADOPValidation script on cvapq01

[START 2017/03/06 23:23:58] prepare phase

[EVENT] [START 2017/03/06 23:23:59] Validating Configuration

[EVENT] Calling:
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSanityCheck.pl

Program : started @ Mon Mar 6 23:23:59 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SANITY_fil
esystemcheck_Mon_Mar_6_23_23_59_2017/txkADOPPreparePhaseSanityCheck_Mon_Mar_6_23_23_5
9_2017.log

LOG FILE :
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SANITY_fil
esystemcheck_Mon_Mar_6_23_23_59_2017/txkADOPPreparePhaseSanityCheck_Mon_Mar_6_23_23_5
9_2017.log
Program : completed @ Mon Mar 6 23:23:59 2017

[EVENT] [END 2017/03/06 23:24:00] Validating Configuration

[EVENT] [START 2017/03/06 23:24:00] Detecting config changes in FS

[EVENT] Calling task: "Executing adConfigChangeDetector on cvapq01"; script:

"/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/adConfigChangeDetector.pl"; args: " -
detectConfigChanges
contextfile=/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml
promptmsg=hide
log=/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/adConfigCh
anges.log hostname=cvapq01"

[EVENT] [START 2017/03/06 23:24:00] Executing adConfigChangeDetector on cvapq01

[EVENT] Calling:
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/adConfigChangeDetector.pl

The log file is

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/adConfigChange
s.log

[EVENT] [END 2017/03/06 23:24:02] Executing adConfigChangeDetector on cvapq01

[EVENT] [END 2017/03/06 23:24:02] Detecting config changes in FS

[EVENT] [START 2017/03/06 23:24:02] Performing check to see if pending cleanup actions exist

[EVENT] No pending cleanup actions, proceeding with other steps

[EVENT] [END 2017/03/06 23:24:03] Performing check to see if pending cleanup actions exist

[EVENT] [START 2017/03/06 23:24:03] Performing DB Sanity checks

[EVENT] [END 2017/03/06 23:24:03] Performing DB Sanity checks

[START 2017/03/06 23:24:03] Generating Tablespace Report

[EVENT] Report: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWTS.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/adzdshowts.out
 [END 2017/03/06 23:24:03] Generating Tablespace Report

[EVENT] [START 2017/03/06 23:24:04] Start Apps Listner

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

adalnctl.sh version 120.3.12020000.2

Checking for FNDFS executable.

Listener APPS_CVQA has already been started.

adalnctl.sh: exiting with status 2

adalnctl.sh: check the logfile

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adalnctl.txt for more information ...

[EVENT] [END 2017/03/06 23:24:04] Start Apps Listner

[EVENT] [START 2017/03/06 23:24:04] Run the ADZDPATCH concurrent program

[EVENT] [START 2017/03/06 23:24:06] Checking status of the ADZDPATCH concurrent program

[EVENT] Status of ADZDPATCH:

[EVENT] This request is waiting to be processed by the Conflict Resolution

Manager. This request cannot yet begin execution because other requests

may conflict with it. The Conflict Resolution Manager will determine when

this request may begin execution. User SYSADMIN submitted this request on

06-MAR-2017 23:24:05. No action required. This is a normal condition.

[EVENT] [END 2017/03/06 23:24:07] Checking status of the ADZDPATCH concurrent program

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/06 23:24:07] Waiting for ADZDPATCH concurrent program to run
 [EVENT] [END 2017/03/06 23:24:38] Waiting for ADZDPATCH concurrent program to run

[EVENT] [END 2017/03/06 23:24:38] Run the ADZDPATCH concurrent program

[EVENT] [START 2017/03/06 23:24:38] Generating ad_zd_logs before truncating

[EVENT] Report: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/adzdshowlog_cl
eanup.out

[EVENT] [END 2017/03/06 23:24:39] Generating ad_zd_logs before truncating

[EVENT] [START 2017/03/06 23:24:39] Check and create Patch Edition (if required)

[EVENT] [START 2017/03/06 23:24:39] Checking if Patch Edition already exists

[EVENT] Patch Edition does not exist

[START 2017/03/06 23:24:39] Creating Patch Edition

[END 2017/03/06 23:24:53] Creating Patch Edition

[EVENT] [END 2017/03/06 23:24:53] Checking if Patch Edition already exists

[EVENT] [END 2017/03/06 23:24:53] Check and create Patch Edition (if required)

[EVENT] [START 2017/03/06 23:24:53] Performing steps to synchronise the FS

[EVENT] [START 2017/03/06 23:24:54] Sync FS of RUN and PATCH

[EVENT] Calling:
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl

Program : started @ Mon Mar 6 23:24:54 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SYNC_upd
ate_Mon_Mar_6_23_24_54_2017/txkADOPPreparePhaseSynchronize_Mon_Mar_6_23_24_54_2017.lo
g

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Oracle E-Business Suite Online Patching

PATCH edition environment variables have been set.

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Oracle E-Business Suite Online Patching

RUN edition environment variables have been set.

========================================

SUMMARY OF PATCHES BEING SYNCHRONIZED...

========================================

[1] SINGLE PATCH (DRV=u18068851.drv): 18068851

SYNCHRONIZATION is in progress. This may take a little longer depending on the number and size of
patches that are being synchronized.

Check log file

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SYNC_upd
ate_Mon_Mar_6_23_24_54_2017/txkADOPPreparePhaseSynchronize_Mon_Mar_6_23_24_54_2017.lo
g for details.

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Oracle E-Business Suite Online Patching

PATCH edition environment variables have been set.

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

 LOGPATH is set to
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SYNC_upd
ate_Mon_Mar_6_23_24_54_2017/log

Reading product information from file...

Reading language and territory information from file...

Reading language information from applUS.txt ...

Temporarily resetting CLASSPATH to:

"/EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/java/adjava.zip:/EQ02/CVQA/fs2/EBSapps/comn/util/jdk32/
lib/dt.jar:/EQ02/CVQA/fs2/EBSapps/comn/util/jdk32/lib/tools.jar:/EQ02/CVQA/fs2/EBSapps/comn/util/j
dk32/jre/lib/rt.jar:/EQ02/CVQA/fs2/EBSapps/comn/shared-libs/ebs-3rdparty/WEB-
INF/lib/ebs3rdpartyManifest.jar:/EQ02/CVQA/fs2/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-
fmw/WEB-INF/lib/ebsFMWManifest.jar:/EQ02/CVQA/fs2/FMW_Home/Oracle_EBS-app1/shared-
libs/ebs-appsborg/WEB-
INF/lib/ebsAppsborgManifest.jar:/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.uix_1
1.1.1/uix2.jar:/EQ02/CVQA/fs2/EBSapps/comn/java/classes:/EQ02/CVQA/fs2/EBSapps/10.1.2/forms/jav
a:/EQ02/CVQA/fs2/EBSapps/10.1.2/forms/java/frmall.jar:/EQ02/CVQA/fs2/EBSapps/10.1.2/jlib/ewt3.jar
:/EQ02/CVQA/fs2/EBSapps/10.1.2/j2ee/OC4J_BI_Forms/applications/formsapp/formsweb/WEB-
INF/lib/frmsrv.jar"

Calling /EQ02/CVQA/fs2/EBSapps/comn/util/jdk32/jre/bin/java ...

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Oracle E-Business Suite Online Patching

RUN edition environment variables have been set.

cmd = /EQ02/CVQA/fs1/EBSapps/comn/util/jdk32/jre/bin/java -classpath

/EQ02/CVQA/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-
INF/lib/ebsAppsborgManifest.jar:/EQ02/CVQA/fs1/EBSapps/comn/java/classes -mx256m
oracle.apps.ad.tools.configuration.ADOPCustomSyncUp -contextFile
/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -custSyncDrv
/EQ02/CVQA/fs_ne/EBSapps/appl/ad/custom/adop_sync.drv -logFile
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SYNC_upd
ate_Mon_Mar_6_23_24_54_2017/txkADOPCustomSyncUp.log -promptmsg 'hide'
sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Removing <MSC_TOP> PS artifacts from PATCH fs

Copying <MSC_TOP> PS artifacts from RUN to PATCH fs for exectier

Removing <MSC_TOP> SNO artifacts from PATCH fs

Copying <MSC_TOP> SNO artifacts from RUN to PATCH fs for exectier

ADOP Custom Synchorization successfully run

LOG FILE :
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SYNC_upd
ate_Mon_Mar_6_23_24_54_2017/txkADOPPreparePhaseSynchronize_Mon_Mar_6_23_24_54_2017.lo
g

Program : completed @ Mon Mar 6 23:33:12 2017

[EVENT] [END 2017/03/06 23:33:12] Sync FS of RUN and PATCH

[EVENT] [END 2017/03/06 23:33:12] Performing steps to synchronise the FS

[EVENT] [START 2017/03/06 23:33:13] Performing steps to validate configuration

[EVENT] [START 2017/03/06 23:33:13] Validating Configuration

[EVENT] Calling:
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSanityCheck.pl

Program : started @ Mon Mar 6 23:33:13 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SANITY_db
check_Mon_Mar_6_23_33_13_2017/txkADOPPreparePhaseSanityCheck_Mon_Mar_6_23_33_13_2017.
log

LOG FILE :
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/TXK_SANITY_db
check_Mon_Mar_6_23_33_13_2017/txkADOPPreparePhaseSanityCheck_Mon_Mar_6_23_33_13_2017.
log

Program : completed @ Mon Mar 6 23:33:14 2017

[EVENT] [END 2017/03/06 23:33:14] Validating Configuration

[EVENT] [END 2017/03/06 23:33:14] Performing steps to validate configuration

[EVENT] [START 2017/03/06 23:33:14] Synchronizing Snapshots

[EVENT] [END 2017/03/06 23:33:41] Synchronizing Snapshots

[EVENT] [START 2017/03/06 23:33:41] Running edition reports post prepare

[EVENT] Report: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWED.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/adzdshowed.ou
t

[EVENT] [END 2017/03/06 23:33:41] Running edition reports post prepare

[END 2017/03/06 23:33:42] prepare phase

[EVENT] [START 2017/03/06 23:33:42] Generating AD_ZD_LOGS Report

[EVENT] Report: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/prepare_20170306_232145/CVQA_cvapq01/adzdshowlog.ou
t

[EVENT] [END 2017/03/06 23:33:43] Generating AD_ZD_LOGS Report

[EVENT] [START 2017/03/06 23:33:44] Check and Stop patch FS Admin Server

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

You are running adadminsrvctl.sh version 120.10.12020000.9

The AdminServer is already shutdown

adadminsrvctl.sh: exiting with status 2

adadminsrvctl.sh: check the logfile

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adadminsrvctl.txt for more
information ...

[EVENT] [END 2017/03/06 23:33:50] Check and Stop patch FS Admin Server

[EVENT] [START 2017/03/06 23:33:51] Check and Stop patch Node Manager

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

You are running adnodemgrctl.sh version 120.11.12020000.11

The Node Manager is already shutdown

adnodemgrctl.sh: exiting with status 2

adnodemgrctl.sh: check the logfile

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adnodemgrctl.txt for more
information ...

[EVENT] [END 2017/03/06 23:33:57] Check and Stop patch Node Manager

[END 2017/03/06 23:33:58] adzdoptl.pl run

 adop phase=prepare - Completed Successfully

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170306_232145.log

adop exiting with status = 0 (Success)

applebs@cvapq01 patch$ adop phase=apply

patches=24576605:u24576605.drv,24576605_D:u24576605.drv,24576605_E:u24576605.drv,24576605_
F:u24576605.drv,24576605_S:u24576605.drv workers=8

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Please wait. Validating credentials...

RUN file system context file:

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

PATCH file system context file:

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

Execute SYSTEM command : df /EQ02/CVQA/fs2

Worker count determination...

Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.

[EVENT] [START 2017/03/06 23:45:45] Performing database sanity checks

[WARNING] The following required database bug fixes <UPGRADE REQUIRED>

are missing from this node, <cvdbq01>.

 Refer to My Oracle Support Knowledge Document 1594274.1 to identify the patch that delivers
this bug fix.

[EVENT] [END 2017/03/06 23:45:46] Finished performing database sanity checks

Using ADOP Session ID from currently incomplete patching cycle

[START 2017/03/06 23:45:46] adzdoptl.pl run

ADOP Session ID: 39

Phase: apply

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170306_234506.log

[START 2017/03/06 23:45:47] apply phase

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Calling: adpatch workers=8 console=no interactive=no

defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA_patch/adalldefaults.txt
patchtop=/EQ02/CVQA/fs_ne/EBSapps/patch/24576605 driver=u24576605.drv logfile=u24576605.log

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/apply_20170306_234506/CVQA_cvapq01/24576605/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Reading product information from file...

Reading language and territory information from file...

Reading language information from applUS.txt ...

Temporarily resetting CLASSPATH to:

"/EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/java/adjava.zip:/EQ02/CVQA/fs2/EBSapps/comn/util/jdk32/
lib/dt.jar:/EQ02/CVQA/fs2/EBSapps/comn/util/jdk32/lib/tools.jar:/EQ02/CVQA/fs2/EBSapps/comn/util/j
dk32/jre/lib/rt.jar:/EQ02/CVQA/fs2/EBSapps/comn/shared-libs/ebs-3rdparty/WEB-
INF/lib/ebs3rdpartyManifest.jar:/EQ02/CVQA/fs2/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-
fmw/WEB-INF/lib/ebsFMWManifest.jar:/EQ02/CVQA/fs2/FMW_Home/Oracle_EBS-app1/shared-
libs/ebs-appsborg/WEB-
INF/lib/ebsAppsborgManifest.jar:/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.uix_1
1.1.1/uix2.jar:/EQ02/CVQA/fs2/EBSapps/comn/java/classes:/EQ02/CVQA/fs2/EBSapps/10.1.2/forms/jav
a:/EQ02/CVQA/fs2/EBSapps/10.1.2/forms/java/frmall.jar:/EQ02/CVQA/fs2/EBSapps/10.1.2/jlib/ewt3.jar
:/EQ02/CVQA/fs2/EBSapps/10.1.2/j2ee/OC4J_BI_Forms/applications/formsapp/formsweb/WEB-
INF/lib/frmsrv.jar"

Calling /EQ02/CVQA/fs2/EBSapps/comn/util/jdk32/jre/bin/java ...

Calling: adpatch workers=8 console=no interactive=no

defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA_patch/adalldefaults.txt
patchtop=/EQ02/CVQA/fs_ne/EBSapps/patch/24576605_D driver=u24576605.drv
logfile=u24576605.log

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/apply_20170306_234506/CVQA_cvapq01/24576605_D/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Calling: adpatch workers=8 console=no interactive=no

defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA_patch/adalldefaults.txt
patchtop=/EQ02/CVQA/fs_ne/EBSapps/patch/24576605_E driver=u24576605.drv logfile=u24576605.log

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/apply_20170306_234506/CVQA_cvapq01/24576605_E/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Calling: adpatch workers=8 console=no interactive=no

defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA_patch/adalldefaults.txt
patchtop=/EQ02/CVQA/fs_ne/EBSapps/patch/24576605_F driver=u24576605.drv logfile=u24576605.log

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/apply_20170306_234506/CVQA_cvapq01/24576605_F/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

 Calling: adpatch workers=8 console=no interactive=no
defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA_patch/adalldefaults.txt
patchtop=/EQ02/CVQA/fs_ne/EBSapps/patch/24576605_S driver=u24576605.drv logfile=u24576605.log

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/apply_20170306_234506/CVQA_cvapq01/24576605_S/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[END 2017/03/07 00:49:56] apply phase

[START 2017/03/07 00:49:56] Generating Post Apply Reports

[EVENT] [START 2017/03/07 00:49:56] Generating AD_ZD_LOGS Report

[EVENT] Report: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/apply_20170306_234506/CVQA_cvapq01/adzdshowlog.out

[EVENT] [END 2017/03/07 00:49:57] Generating AD_ZD_LOGS Report

[END 2017/03/07 00:49:57] Generating Post Apply Reports

[END 2017/03/07 00:49:57] adzdoptl.pl run

adop phase=apply - Completed Successfully

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170306_234506.log

adop exiting with status = 0 (Success)

applebs@cvapq01 patch$

applebs@cvapq01 patch$

applebs@cvapq01 patch$

applebs@cvapq01 patch$

applebs@cvapq01 patch$
applebs@cvapq01 patch$ adop phase=finalize workers=8

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Please wait. Validating credentials...

RUN file system context file:

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

PATCH file system context file:

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

Execute SYSTEM command : df /EQ02/CVQA/fs2

Worker count determination...

Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.

[EVENT] [START 2017/03/07 00:58:32] Performing database sanity checks

[WARNING] The following required database bug fixes <UPGRADE REQUIRED>

are missing from this node, <cvdbq01>.

Refer to My Oracle Support Knowledge Document 1594274.1 to identify the patch that delivers
this bug fix.

[EVENT] [END 2017/03/07 00:58:34] Finished performing database sanity checks

Using ADOP Session ID from currently incomplete patching cycle

[START 2017/03/07 00:58:34] adzdoptl.pl run

ADOP Session ID: 39

Phase: finalize

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170307_005805.log

 [START 2017/03/07 00:58:35] finalize phase

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] Running Finalize in QUICK mode

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[END 2017/03/07 00:59:01] finalize phase

[START 2017/03/07 00:59:01] Generating Post Finalize Reports

[END 2017/03/07 00:59:01] Generating Post Finalize Reports

[EVENT] [START 2017/03/07 00:59:01] Generating AD_ZD_LOGS Report

[EVENT] Report: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/finalize_20170307_005805/CVQA_cvapq01/adzdshowlog.out

[EVENT] [END 2017/03/07 00:59:01] Generating AD_ZD_LOGS Report

[END 2017/03/07 00:59:02] adzdoptl.pl run

adop phase=finalize - Completed Successfully

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170307_005805.log

adop exiting with status = 0 (Success)

applebs@cvapq01 patch$ adop phase=cutover cm_wait=3 workers=8

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Please wait. Validating credentials...

RUN file system context file:
/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

PATCH file system context file:

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

Execute SYSTEM command : df /EQ02/CVQA/fs2

Worker count determination...

Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.

[EVENT] [START 2017/03/07 01:00:07] Performing database sanity checks

[WARNING] The following required database bug fixes <UPGRADE REQUIRED>

are missing from this node, <cvdbq01>.

Refer to My Oracle Support Knowledge Document 1594274.1 to identify the patch that delivers
this bug fix.

[EVENT] [END 2017/03/07 01:00:09] Finished performing database sanity checks

Using ADOP Session ID from currently incomplete patching cycle

[START 2017/03/07 01:00:09] adzdoptl.pl run

ADOP Session ID: 39

Phase: cutover

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170307_005945.log

[EVENT] [START 2017/03/07 01:00:10] Calling task: "Executing txkADOPValidation script on

cvapq01"; script: "/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl";
args: " -contextfile=/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
patchctxfile=/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
phase=cutover -
logloc=/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01 -
promptmsg=hide"

[EVENT] [START 2017/03/07 01:00:10] Executing txkADOPValidation script on cvapq01

[EVENT] Calling: /EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Oracle E-Business Suite Online Patching

RUN edition environment variables have been set.

calling: /EQ02/CVQA/fs1/FMW_Home/webtier/perl/bin/perl
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkValidateSetup.pl -
contextfile=/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
patchctxfile=/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml -
logloc=/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01 -
phase=cutover -promptmsg=hide

Program : txkValidateSetup.pl started @ Tue Mar 7 01:00:12 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/ADOPValidation
s_Tue_Mar_7_01_00_12_2017.log

SETUP VALIDATION is in progress. This may take few minutes to complete.

Completed execution of all tests.

=========================== START OF VALIDATION REPORT ====================

Validation Results for Node: cvapq01

---------------------------------------

All validations were successful, please check

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/ADOPValidation
s_Tue_Mar_7_01_00_12_2017.log

=========================== END OF VALIDATION REPORT ======================

[EVENT] [END 2017/03/07 01:00:15] Executing txkADOPValidation script on cvapq01

 [EVENT] [END 2017/03/07 01:00:15] Executing txkADOPValidation script on cvapq01

[EVENT] [START 2017/03/07 01:00:16] Checking if all nodes were synchronized in the previous apply
phase

[EVENT] [END 2017/03/07 01:00:16] Checking if all nodes were synchronized in the previous apply
phase

[START 2017/03/07 01:00:16] cutover phase

[EVENT] [START 2017/03/07 01:00:17] Performing Cutover Phase steps

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:00:19] ICM Shutdown

/bin/sh: module: line 1: syntax error: unexpected end of file

/bin/sh: error importing function definition for `BASH_FUNC_module'

You are running adcmctl.sh version 120.19.12020000.3

Shutting down concurrent managers for CVQA ...

ORACLE Password:

Submitted request 10846272 for CONCURRENT FND SHUTDOWN

adcmctl.sh: exiting with status 0

adcmctl.sh: check the logfile

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adcmctl.txt for more information ...
 [EVENT] [END 2017/03/07 01:00:19] ICM Shutdown

[EVENT] [START 2017/03/07 01:00:20] Waiting for ICM to go down

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:03:22] Forcefully bringing down ICM

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:03:22] ICM Shutdown

/bin/sh: module: line 1: syntax error: unexpected end of file

/bin/sh: error importing function definition for `BASH_FUNC_module'

You are running adcmctl.sh version 120.19.12020000.3

Aborting concurrent managers for CVQA ...

ORACLE Password:

Submitted request 10846273 for CONCURRENT FND ABORT

adcmctl.sh: exiting with status 0

adcmctl.sh: check the logfile

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adcmctl.txt for more information ...
 [EVENT] [END 2017/03/07 01:03:23] ICM Shutdown

[EVENT] [END 2017/03/07 01:03:23] Forcefully bringing down ICM

[EVENT] [END 2017/03/07 01:04:23] Waiting for ICM to go down

[EVENT] [START 2017/03/07 01:04:24] Check and Stop patch FS Admin Server

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

You are running adadminsrvctl.sh version 120.10.12020000.9

The AdminServer is already shutdown

adadminsrvctl.sh: exiting with status 2

adadminsrvctl.sh: check the logfile

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adadminsrvctl.txt for more
information ...

[EVENT] [END 2017/03/07 01:04:31] Check and Stop patch FS Admin Server

[EVENT] [START 2017/03/07 01:04:31] Check and Stop patch Node Manager

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

You are running adnodemgrctl.sh version 120.11.12020000.11

The Node Manager is already shutdown

adnodemgrctl.sh: exiting with status 2

adnodemgrctl.sh: check the logfile

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/logs/appl/admin/log/adnodemgrctl.txt for more
information ...

[EVENT] [END 2017/03/07 01:04:36] Check and Stop patch Node Manager

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:04:37] Bringing down Middle Tier Services

[EVENT] Calling:
/EQ02/CVQA/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl

Program : started @ Tue Mar 7 01:04:37 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/TXK_CTRL_force
shutdown_Tue_Mar_7_01_04_37_2017/txkADOPCutOverPhaseCtrlScript_Tue_Mar_7_01_04_37_2017.
log

/bin/sh: module: line 1: syntax error: unexpected end of file

/bin/sh: error importing function definition for `BASH_FUNC_module'

LOG FILE :
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/TXK_CTRL_force
shutdown_Tue_Mar_7_01_04_37_2017/txkADOPCutOverPhaseCtrlScript_Tue_Mar_7_01_04_37_2017.
log

Program : completed @ Tue Mar 7 01:06:12 2017

[EVENT] [END 2017/03/07 01:06:13] Bringing down Middle Tier Services

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Calling: adpatch options=hotpatch,nocompiledb interactive=no console=no workers=8 restart=no

abandon=yes defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA_patch/adalldefaults.txt
patchtop=/EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/patch/115/driver logfile=cutover.log
driver=ucutover.drv

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:08:00] Loading Jar files to Database

[EVENT] No jars found to load

[EVENT] [END 2017/03/07 01:08:00] Loading Jar files to Database

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:08:04] Performing File System Cutover

Program : started @ Tue Mar 7 01:08:04 2017

*** Log File =

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/TXK_CTRL_ctxup
date_Tue_Mar_7_01_08_04_2017/txkADOPCutOverPhaseCtrlScript_Tue_Mar_7_01_08_04_2017.log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

LOG FILE :
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/TXK_CTRL_ctxup
date_Tue_Mar_7_01_08_04_2017/txkADOPCutOverPhaseCtrlScript_Tue_Mar_7_01_08_04_2017.log

Program : completed @ Tue Mar 7 01:09:00 2017

[EVENT] [END 2017/03/07 01:09:00] Performing File System Cutover

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Broadcast message from [email protected] (Tue Mar 7 01:09:01 2017):

CVQA environment has changed.

All users must re-source the environment using below command:

source /EQ02/CVQA/EBSapps.env run|patch

[EVENT] [START 2017/03/07 01:09:01] Start Admin Server

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

NMProcess: <Mar 7, 2017 1:09:13 AM> <INFO> <Loading domains file:

/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1/nodemanager.domai
ns>

NMProcess: Mar 07, 2017 1:09:13 AM weblogic.nodemanager.server.NMServerConfig initDomainsMap

NMProcess: INFO: Loading domains file:
/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1/nodemanager.domai
ns

NMProcess: <Mar 7, 2017 1:09:13 AM> <INFO> <Loaded node manager configuration properties from
'/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1/nodemanager.prope
rties'>

NMProcess: Mar 07, 2017 1:09:13 AM weblogic.nodemanager.server.NMServer <init>

NMProcess: INFO: Loaded node manager configuration properties from

'/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1/nodemanager.prope
rties'

NMProcess: Node manager v10.3

NMProcess:

NMProcess: Configuration settings:

NMProcess:

NMProcess:
NodeManagerHome=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1

NMProcess: ListenAddress=cvapq01.dana.com

NMProcess: ListenPort=5566

NMProcess: ListenBacklog=50

NMProcess: SecureListener=false

NMProcess: AuthenticationEnabled=true

NMProcess: NativeVersionEnabled=true

NMProcess: CrashRecoveryEnabled=false

NMProcess: JavaHome=/EQ02/CVQA/fs2/EBSapps/comn/util/jdk64

NMProcess: StartScriptEnabled=false

NMProcess: StopScriptEnabled=false

NMProcess: StartScriptName=startWebLogic.sh

NMProcess: StopScriptName=

NMProcess:
LogFile=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1/nodemanage
r.log

NMProcess: LogLevel=INFO
NMProcess: LogLimit=0

NMProcess: LogCount=1

NMProcess: LogAppend=false

NMProcess: LogToStderr=true

NMProcess: LogFormatter=weblogic.nodemanager.server.LogFormatter

NMProcess:
DomainsFile=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1/nodema
nager.domains

NMProcess: DomainsFileEnabled=true

NMProcess: StateCheckInterval=500

NMProcess: QuitEnabled=true

NMProcess: UseMACBroadcast=false

NMProcess: DomainRegistrationEnabled=false

NMProcess: DomainsDirRemoteSharingEnabled=false

NMProcess:

NMProcess: Domain name mappings:

NMProcess:

NMProcess: EBS_domain_CVQA ->

/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA

NMProcess:
NMProcess: <Mar 7, 2017 1:09:14 AM> <INFO> <EBS_domain_CVQA> <forms_server1> <Startup
configuration properties loaded from
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/forms_server1/dat
a/nodemanager/startup.properties">

NMProcess: Mar 07, 2017 1:09:14 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Startup configuration properties loaded from

"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/forms_server1/dat
a/nodemanager/startup.properties"

NMProcess: <Mar 7, 2017 1:09:14 AM> <INFO> <EBS_domain_CVQA> <forms-c4ws_server1> <Startup

configuration properties loaded from
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/forms-
c4ws_server1/data/nodemanager/startup.properties">

NMProcess: Mar 07, 2017 1:09:14 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Startup configuration properties loaded from
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/forms-
c4ws_server1/data/nodemanager/startup.properties"

NMProcess: <Mar 7, 2017 1:09:14 AM> <INFO> <EBS_domain_CVQA> <oafm_server1> <Startup

configuration properties loaded from
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/oafm_server1/data
/nodemanager/startup.properties">

NMProcess: Mar 07, 2017 1:09:14 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Startup configuration properties loaded from

"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/oafm_server1/data
/nodemanager/startup.properties"

NMProcess: <Mar 7, 2017 1:09:14 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Startup

configuration properties loaded from
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/data/
nodemanager/startup.properties">

NMProcess: Mar 07, 2017 1:09:14 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Startup configuration properties loaded from

"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/data/
nodemanager/startup.properties"

NMProcess: <Mar 7, 2017 1:09:14 AM> <INFO> <EBS_domain_CVQA> <oacore_server1> <Startup

configuration properties loaded from
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/oacore_server1/da
ta/nodemanager/startup.properties">

NMProcess: Mar 07, 2017 1:09:14 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Startup configuration properties loaded from

"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/oacore_server1/da
ta/nodemanager/startup.properties"

NMProcess: <Mar 7, 2017 1:09:14 AM> <INFO> <Plain socket listener started on port 5566, host
cvapq01.dana.com>

NMProcess: Mar 07, 2017 1:09:14 AM weblogic.nodemanager.server.Listener run

NMProcess: INFO: Plain socket listener started on port 5566, host cvapq01.dana.com

NMProcess: <Mar 7, 2017 1:09:32 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Startup

configuration properties saved to
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/data/
nodemanager/startup.properties">

NMProcess: Mar 07, 2017 1:09:32 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Startup configuration properties saved to
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/data/
nodemanager/startup.properties"

NMProcess: <Mar 7, 2017 1:09:32 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Rotated server
output log to
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/logs/
AdminServer.out00008">

NMProcess: Mar 07, 2017 1:09:32 AM weblogic.nodemanager.server.LogFileRotationUtil log

NMProcess: INFO: Rotated server output log to

"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/logs/
AdminServer.out00008"

NMProcess: <Mar 7, 2017 1:09:32 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Server error log
also redirected to server log>

NMProcess: Mar 07, 2017 1:09:32 AM weblogic.nodemanager.server.LogFileRotationUtil log

NMProcess: INFO: Server error log also redirected to server log

NMProcess: <Mar 7, 2017 1:09:32 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Starting

WebLogic server with command line: /EQ02/CVQA/fs2/EBSapps/comn/util/jdk64/jre/bin/java -
Dweblogic.Name=AdminServer -Djava.security.policy=null -
Djava.library.path=/EQ02/CVQA/fs2/FMW_Home/patch_wls1036/profiles/default/native:/EQ02/CVQA/
fs2/EBSapps/10.1.2/jdk/jre/lib/i386:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386/server:/EQ02/CV
QA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386/native_threads:/EQ02/CVQA/fs2/EBSapps/appl/cz/12.0.0/bin:/
EQ02/CVQA/fs2/EBSapps/10.1.2/lib32:/EQ02/CVQA/fs2/EBSapps/10.1.2/lib:/usr/X11R6/lib:/usr/openwi
n/lib:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i38
6/server:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386/native_threads:/EQ02/CVQA/fs2/EBSapps/a
ppl/sht/12.0.0/lib:/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/native/linux/x86_64:/EQ02/CVQ
A/fs2/FMW_Home/wlserver_10.3/server/native/linux/x86_64/oci920_8:/usr/java/packages/lib/amd64:
/usr/lib64:/lib64:/lib:/usr/lib -
Djava.class.path=/EQ02/CVQA/fs2/FMW_Home/patch_wls1036/profiles/default/sys_manifest_classpat
h/weblogic_patch.jar:/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic_sp.jar:/EQ02/C
VQA/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.jar:/EQ02/CVQA/fs2/FMW_Home/modules/fe
atures/weblogic.server.modules_10.3.6.0.jar:/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/lib/w
ebservices.jar:/EQ02/CVQA/fs2/FMW_Home/modules/org.apache.ant_1.7.1/lib/ant-
all.jar:/EQ02/CVQA/fs2/FMW_Home/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-
contrib.jar:/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.jdbc_11.1.1/ojdbc6dms.jar:/
EQ02/CVQA/fs2/FMW_Home/oracle_common/webcenter/modules/oracle.portlet.server_11.1.1/oracle
-portlet-
api.jar:/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.jrf_11.1.1/jrf.jar:/EQ02/CVQA/fs
2/FMW_Home/wlserver_10.3/common/derby/lib/derbyclient.jar -
Dweblogic.system.BootIdentityFile=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain
_CVQA/servers/AdminServer/security/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -
XX:PermSize=128m -XX:MaxPermSize=384m -Xms1024m -Xmx1024m -
Djava.security.policy=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.policy -
Djava.security.egd=file:/dev/./urandom -Dweblogic.ProductionModeEnabled=true -da -
Dplatform.home=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3 -
Dwls.home=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server -
Dweblogic.home=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server -
Dcommon.components.home=/EQ02/CVQA/fs2/FMW_Home/oracle_common -Djrf.version=11.1.1 -
Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -
Ddomain.home=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA -
Djrockit.optfile=/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.jrf_11.1.1/jrocket_optf
ile.txt -
Doracle.server.config.dir=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/co
nfig/fmwconfig/servers/AdminServer -
Doracle.domain.config.dir=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/c
onfig/fmwconfig -
Digf.arisidbeans.carmlloc=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/c
onfig/fmwconfig/carml -
Digf.arisidstack.home=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/confi
g/fmwconfig/arisidprovider -
Doracle.security.jps.config=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/
config/fmwconfig/jps-config.xml -
Doracle.deployed.app.dir=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/s
ervers/AdminServer/tmp/_WL_user -Doracle.deployed.app.ext=/- -
Dweblogic.alternateTypesDirectory=/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.os
soiap_11.1.1,/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.oamprovider_11.1.1 -
Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dweblogic.jdbc.remoteEnabled=false -
Dportlet.oracle.home=/EQ02/CVQA/fs2/FMW_Home/oracle_common -
Dem.oracle.home=/EQ02/CVQA/fs2/FMW_Home/oracle_common -
Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -
Dwlw.logErrorsToConsole=false -
Dweblogic.ext.dirs=/EQ02/CVQA/fs2/FMW_Home/patch_wls1036/profiles/default/sysext_manifest_cla
sspath weblogic.Server >

NMProcess: Mar 07, 2017 1:09:32 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Starting WebLogic server with command line:

/EQ02/CVQA/fs2/EBSapps/comn/util/jdk64/jre/bin/java -Dweblogic.Name=AdminServer -
Djava.security.policy=null -
Djava.library.path=/EQ02/CVQA/fs2/FMW_Home/patch_wls1036/profiles/default/native:/EQ02/CVQA/
fs2/EBSapps/10.1.2/jdk/jre/lib/i386:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386/server:/EQ02/CV
QA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386/native_threads:/EQ02/CVQA/fs2/EBSapps/appl/cz/12.0.0/bin:/
EQ02/CVQA/fs2/EBSapps/10.1.2/lib32:/EQ02/CVQA/fs2/EBSapps/10.1.2/lib:/usr/X11R6/lib:/usr/openwi
n/lib:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i38
6/server:/EQ02/CVQA/fs2/EBSapps/10.1.2/jdk/jre/lib/i386/native_threads:/EQ02/CVQA/fs2/EBSapps/a
ppl/sht/12.0.0/lib:/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/native/linux/x86_64:/EQ02/CVQ
A/fs2/FMW_Home/wlserver_10.3/server/native/linux/x86_64/oci920_8:/usr/java/packages/lib/amd64:
/usr/lib64:/lib64:/lib:/usr/lib -
Djava.class.path=/EQ02/CVQA/fs2/FMW_Home/patch_wls1036/profiles/default/sys_manifest_classpat
h/weblogic_patch.jar:/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic_sp.jar:/EQ02/C
VQA/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.jar:/EQ02/CVQA/fs2/FMW_Home/modules/fe
atures/weblogic.server.modules_10.3.6.0.jar:/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/lib/w
ebservices.jar:/EQ02/CVQA/fs2/FMW_Home/modules/org.apache.ant_1.7.1/lib/ant-
all.jar:/EQ02/CVQA/fs2/FMW_Home/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-
contrib.jar:/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.jdbc_11.1.1/ojdbc6dms.jar:/
EQ02/CVQA/fs2/FMW_Home/oracle_common/webcenter/modules/oracle.portlet.server_11.1.1/oracle
-portlet-
api.jar:/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.jrf_11.1.1/jrf.jar:/EQ02/CVQA/fs
2/FMW_Home/wlserver_10.3/common/derby/lib/derbyclient.jar -
Dweblogic.system.BootIdentityFile=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain
_CVQA/servers/AdminServer/security/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -
XX:PermSize=128m -XX:MaxPermSize=384m -Xms1024m -Xmx1024m -
Djava.security.policy=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.policy -
Djava.security.egd=file:/dev/./urandom -Dweblogic.ProductionModeEnabled=true -da -
Dplatform.home=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3 -
Dwls.home=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server -
Dweblogic.home=/EQ02/CVQA/fs2/FMW_Home/wlserver_10.3/server -
Dcommon.components.home=/EQ02/CVQA/fs2/FMW_Home/oracle_common -Djrf.version=11.1.1 -
Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -
Ddomain.home=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA -
Djrockit.optfile=/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.jrf_11.1.1/jrocket_optf
ile.txt -
Doracle.server.config.dir=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/co
nfig/fmwconfig/servers/AdminServer -
Doracle.domain.config.dir=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/c
onfig/fmwconfig -
Digf.arisidbeans.carmlloc=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/c
onfig/fmwconfig/carml -
Digf.arisidstack.home=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/confi
g/fmwconfig/arisidprovider -
Doracle.security.jps.config=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/
config/fmwconfig/jps-config.xml -
Doracle.deployed.app.dir=/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/s
ervers/AdminServer/tmp/_WL_user -Doracle.deployed.app.ext=/- -
Dweblogic.alternateTypesDirectory=/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.os
soiap_11.1.1,/EQ02/CVQA/fs2/FMW_Home/oracle_common/modules/oracle.oamprovider_11.1.1 -
Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dweblogic.jdbc.remoteEnabled=false -
Dportlet.oracle.home=/EQ02/CVQA/fs2/FMW_Home/oracle_common -
Dem.oracle.home=/EQ02/CVQA/fs2/FMW_Home/oracle_common -
Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -
Dwlw.logErrorsToConsole=false -
Dweblogic.ext.dirs=/EQ02/CVQA/fs2/FMW_Home/patch_wls1036/profiles/default/sysext_manifest_cla
sspath weblogic.Server

NMProcess: <Mar 7, 2017 1:09:32 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Working

directory is '/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA'>

NMProcess: Mar 07, 2017 1:09:32 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Working directory is

'/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA'

NMProcess: <Mar 7, 2017 1:09:33 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Rotated server
output log to
"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/logs/
AdminServer.out00009">

NMProcess: Mar 07, 2017 1:09:33 AM weblogic.nodemanager.server.LogFileRotationUtil log

NMProcess: INFO: Rotated server output log to

"/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/logs/
AdminServer.out00009"

NMProcess: <Mar 7, 2017 1:09:33 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Server error log
also redirected to server log>

NMProcess: Mar 07, 2017 1:09:33 AM weblogic.nodemanager.server.LogFileRotationUtil log

NMProcess: INFO: Server error log also redirected to server log

NMProcess: <Mar 7, 2017 1:09:33 AM> <INFO> <EBS_domain_CVQA> <AdminServer> <Server output
log file is
'/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/logs/
AdminServer.out'>

NMProcess: Mar 07, 2017 1:09:33 AM weblogic.nodemanager.server.AbstractServerManager log

NMProcess: INFO: Server output log file is

'/EQ02/CVQA/fs2/FMW_Home/user_projects/domains/EBS_domain_CVQA/servers/AdminServer/logs/
AdminServer.out'

[EVENT] [END 2017/03/07 01:10:28] Start Admin Server

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [START 2017/03/07 01:10:29] Starting Middle Tier Services

Program : started @ Tue Mar 7 01:10:29 2017

*** Log File =
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/TXK_CTRL_force
startup_Tue_Mar_7_01_10_29_2017/txkADOPCutOverPhaseCtrlScript_Tue_Mar_7_01_10_29_2017.log

/bin/sh: module: line 1: syntax error: unexpected end of file

/bin/sh: error importing function definition for `BASH_FUNC_module'

LOG FILE :
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/TXK_CTRL_force
startup_Tue_Mar_7_01_10_29_2017/txkADOPCutOverPhaseCtrlScript_Tue_Mar_7_01_10_29_2017.log

Program : completed @ Tue Mar 7 01:14:11 2017

[EVENT] [END 2017/03/07 01:14:12] Starting Middle Tier Services

[EVENT] [END 2017/03/07 01:14:14] Performing Cutover Phase steps

[END 2017/03/07 01:14:15] cutover phase

[START 2017/03/07 01:14:15] Generating Post Cutover Reports

[EVENT] [START 2017/03/07 01:14:15] Generating AD_ZD_LOGS Report

[EVENT] Report: /EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cutover_20170307_005945/CVQA_cvapq01/adzdshowlog.ou
t

[EVENT] [END 2017/03/07 01:14:16] Generating AD_ZD_LOGS Report

[END 2017/03/07 01:14:16] Generating Post Cutover Reports

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

Broadcast message from [email protected] (Tue Mar 7 01:14:16 2017):

CVQA environment has changed.

All users must re-source the environment using below command:

source /EQ02/CVQA/EBSapps.env run|patch

[END 2017/03/07 01:14:16] adzdoptl.pl run

adop phase=cutover - Completed Successfully

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170307_005945.log

adop exiting with status = 0 (Success)

You have new mail in /var/spool/mail/applebs

applebs@cvapq01 patch$

applebs@cvapq01 patch$

applebs@cvapq01 patch$

applebs@cvapq01 patch$ adop phase=cleanup cleanup_mode=full

Enter the APPS password:

Enter the SYSTEM password:

Enter the WLSADMIN password:

Please wait. Validating credentials...

RUN file system context file:

/EQ02/CVQA/fs2/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml

PATCH file system context file:

/EQ02/CVQA/fs1/inst/apps/CVQA_cvapq01/appl/admin/CVQA_cvapq01.xml
Execute SYSTEM command : df /EQ02/CVQA/fs1

Worker count determination...

Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.

[EVENT] [START 2017/03/07 01:15:42] Performing database sanity checks

[WARNING] The following required database bug fixes <UPGRADE REQUIRED>

are missing from this node, <cvdbq01>.

Refer to My Oracle Support Knowledge Document 1594274.1 to identify the patch that delivers
this bug fix.

[EVENT] [END 2017/03/07 01:15:43] Finished performing database sanity checks

[START 2017/03/07 01:15:43] adzdoptl.pl run

ADOP Session ID: 39

Phase: cleanup

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170307_011514.log

[START 2017/03/07 01:15:44] cleanup phase

[EVENT] [START 2017/03/07 01:15:44] Performing Cleanup steps

[EVENT] [START 2017/03/07 01:15:44] Running stored CLEANUP actions

Calling: adpatch options=hotpatch,nocompiledb interactive=no console=no workers=4 restart=no

abandon=yes defaultsfile=/EQ02/CVQA/fs2/EBSapps/appl/admin/CVQA/adalldefaults.txt
patchtop=/EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/patch/115/driver logfile=cleanup.log
driver=ucleanup.drv

ADPATCH Log directory:

/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cleanup_20170307_011514/CVQA_cvapq01/log

sh: module: line 1: syntax error: unexpected end of file

sh: error importing function definition for `BASH_FUNC_module'

[EVENT] [END 2017/03/07 01:19:04] Running stored CLEANUP actions

[START 2017/03/07 01:19:04] Generating All DDL Report

[EVENT] Report: /EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/sql/ADZDALLDDLS.sql

 [EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cleanup_20170307_011514/CVQA_cvapq01

[END 2017/03/07 01:19:05] Generating All DDL Report

[EVENT] Calling cleanup in FULL mode

[WARNING] Cleanup may take a while. Please wait.

[EVENT] [END 2017/03/07 01:28:32] Performing Cleanup steps

[END 2017/03/07 01:28:32] cleanup phase

[EVENT] [START 2017/03/07 01:28:32] Generating AD_ZD_LOGS Report

[EVENT] Report: /EQ02/CVQA/fs2/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql

[EVENT] Output:
/EQ02/CVQA/fs_ne/EBSapps/log/adop/39/cleanup_20170307_011514/CVQA_cvapq01/adzdshowlog.ou
t

[EVENT] [END 2017/03/07 01:28:33] Generating AD_ZD_LOGS Report

[END 2017/03/07 01:28:33] adzdoptl.pl run

adop phase=cleanup - Completed Successfully

Log file: /EQ02/CVQA/fs_ne/EBSapps/log/adop/39/adop_20170307_011514.log

adop exiting with status = 0 (Success)

SQL> select * from ad_bugs where bug_number = '17797381';

no rows selected
SQL> select * from ad_bugs where bug_number = '24576605';

BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

1138128

24576605 07-MAR-17 R12

5 07-MAR-17 5 ont

BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

n US
 BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

1138151

24576605 07-MAR-17 R12

5 07-MAR-17 5 ont

BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

nD
 BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

1138174

24576605 07-MAR-17 R12

5 07-MAR-17 5 ont

BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

nE
 BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

1138197

24576605 07-MAR-17 R12

5 07-MAR-17 5 ont

BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

nF

BUG_ID APPLICATION_SHORT_NAME
---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

1138220

24576605 07-MAR-17 R12

5 07-MAR-17 5 ont

BUG_ID APPLICATION_SHORT_NAME

---------- --------------------------------------------------

BUG_NUMBER CREATION_DATE ARU_RELEASE_NAME

------------------------------ --------------- ------------------------------

CREATED_BY LAST_UPDATE_DAT LAST_UPDATED_BY TRACKABL

---------- --------------- --------------- --------

BASELINE_NAME

--------------------------------------------------------------------------------

G LANG

- ----

nS

Before Patch : Invalids...

SQL> select object_name, owner, object_type from all_objects where status like 'INVALID';
SQL> /

OBJECT_NAME OWNER OBJECT_TYPE

------------------------------ -------------------------------------------------

BU_VBOM_COMP APPS VIEW

OE_ITEMS_MV APPS MATERIALIZED VIEW

OZF_EARNING_SUMMARY_MV APPS MATERIALIZED VIEW

XXX_MENU_FUNC_LIST APPS_RO PROCEDURE

/42019701_ObjectFactory LOFTWARE JAVA CLASS

com/loftware/lps/LPSTransfer LOFTWARE JAVA CLASS

PO_APPRVL_ANALYZER_PKG JOSE.DAGUM PACKAGE BODY

7 rows selected.

1* select object_name, owner, object_type from all_objects where status like 'INVALID'

SQL> /

Before Patch : Invalids...

OBJECT_NAME OWNER OBJECT_TYPE

------------------------------ ------------------------------ -------------------

OMDANA_AMA_ORDER_ACK APPS TRIGGER

JAI_OE_OHA_ARIUD_T1 APPS TRIGGER

JAI_OE_OLA_ARIUD_T1 APPS TRIGGER

JAI_OE_OLA_ARIUD_TRG APPS TRIGGER

JAI_OE_OLA_BRIUD_T1 APPS TRIGGER

BU_VBOM_COMP APPS VIEW

OE_ODR_LINES_SN APPS MATERIALIZED VIEW

OZF_EARNING_SUMMARY_MV APPS MATERIALIZED VIEW

XXX_MENU_FUNC_LIST APPS_RO PROCEDURE

/42019701_ObjectFactory LOFTWARE JAVA CLASS

com/loftware/lps/LPSTransfer LOFTWARE JAVA CLASS

PO_APPRVL_ANALYZER_PKG JOSE.DAGUM PACKAGE BODY

12 rows selected.

SQL> alter TRIGGER OMDANA_AMA_ORDER_ACK compile;

Trigger altered.

SQL> select object_name, owner, object_type from all_objects where status like 'INVALID';

OBJECT_NAME OWNER OBJECT_TYPE

------------------------------ ------------------------------ -------------------

JAI_OE_OHA_ARIUD_T1 APPS TRIGGER

JAI_OE_OLA_ARIUD_T1 APPS TRIGGER

JAI_OE_OLA_ARIUD_TRG APPS TRIGGER

JAI_OE_OLA_BRIUD_T1 APPS TRIGGER

BU_VBOM_COMP APPS VIEW

OE_ODR_LINES_SN APPS MATERIALIZED VIEW

OZF_EARNING_SUMMARY_MV APPS MATERIALIZED VIEW

XXX_MENU_FUNC_LIST APPS_RO PROCEDURE

/42019701_ObjectFactory LOFTWARE JAVA CLASS

com/loftware/lps/LPSTransfer LOFTWARE JAVA CLASS

PO_APPRVL_ANALYZER_PKG JOSE.DAGUM PACKAGE BODY

11 rows selected.

SQL> select object_name, owner, object_type from all_objects where OBJECT_NAME

='OMDANA_AMA_ORDER_ACK';

OBJECT_NAME OWNER OBJECT_TYPE

------------------------------ ------------------------------ -------------------

OMDANA_AMA_ORDER_ACK APPS TRIGGER

SQL> select object_name, owner, object_type,status from all_objects where OBJECT_NAME

='OMDANA_AMA_ORDER_ACK';

OBJECT_NAME OWNER OBJECT_TYPE STATUS

------------------------------ ------------------------------ ------------------- -------

OMDANA_AMA_ORDER_ACK APPS TRIGGER VALID

SQL> alter TRIGGER JAI_OE_OHA_ARIUD_T1 compile;

Trigger altered.

SQL> alter TRIGGER JAI_OE_OLA_ARIUD_T1 compile;

Trigger altered.

SQL> alter TRIGGER JAI_OE_OLA_ARIUD_TRG compile;

Trigger altered.

SQL> alter TRIGGER JAI_OE_OLA_BRIUD_T1 compile;

Trigger altered.

SQL> commit;

Commit complete.

SQL> select object_name, owner, object_type from all_objects where status like 'INVALID';

OBJECT_NAME OWNER OBJECT_TYPE

------------------------------ ------------------------------ -------------------

BU_VBOM_COMP APPS VIEW

OE_ODR_LINES_SN APPS MATERIALIZED VIEW

OZF_EARNING_SUMMARY_MV APPS MATERIALIZED VIEW

XXX_MENU_FUNC_LIST APPS_RO PROCEDURE

/42019701_ObjectFactory LOFTWARE JAVA CLASS

com/loftware/lps/LPSTransfer LOFTWARE JAVA CLASS

PO_APPRVL_ANALYZER_PKG JOSE.DAGUM PACKAGE BODY

7 rows selected.