Concurrent Managers
Concurrent Managers
Concurrent Manager
Output-post-processor
XML Post Processor
XML-publisher
Concurrent Manager Queries
Concurrent Manager
cd $APPLCSF/$APPLLOG
Internal Manager
Standard Manager.
Internal monitors
Service Manager
Transaction Manager
Answer: It takes care of resolving the program incompatibilities and checks if a request in queue
can be run in parallel with the running request. If a program is identified as run alone, then it
prevents concurrent managers from starting other programs in the same conflict domain.
Answer : Standard Manager is the master concurrent manager. It is always running and can take
care of processing any concurrent request. If at all, no other manager is assigned to a program,
that program will be picked by standard manager.
Question 5:
Answer
2)a) If the request is incompatible/constraints defined ,Once the request time to run is arrived,it
status is set to pending/Standby.Now the conflict resolution manager takes care of the request and
finds out what are the incompatibilities and set the status pending normal when the
incompatibilities are cleared.
b) If there are no incompatibilities then Once the request time to run is arrived,it status is set to
pending/Normal
c) ALL the standard concurrent Manager and special manager continuously poll the
FND_CONCURRENT_REQUESTS table .The job of a concurrent manager is to execute concurrent
requests that are in Pending / Normal phase / status and that it is qualified to run according to its
specialization rules.
– Act independently
– Select only requests that: (a) match the manager specialization rules, (b) are Pending/Normal, (c)
have a requested start time <= sysdate
d) Once the request is processed, the FND_CONCURRENT_REQUESTS table is updated with the
status.
Question 6: Business user create the task that Concurrent request is taking a lot of time to
complete. What will be your approach for debugging it?
Answer
1) First find the status of the concurrent request. It may be scheduled later on or it could be
pending/standby mode or all the concurrent manager are occupied running other request. If it is
pending/standy, we need to find the incompatible program running and inform user.Many times the
users schedule the request to run at a later time.
2) Find out the database sid of the concurrent request and check it is waiting on any locks. We will
kill the blocking session to order to complete the job
3) We can run a trace on the request id to find the sql running and then generate the explain plan
for it. You can see if the sid is stuck on particular sql . If it is particular sql, then it is good to check
the statistics of the table involved . We can look for tuning option for that query
4) We can check the parameters with which the request is run. (For example, once a user came
saying the request is not printing the output. On Checking the possible things, it was realized that
he scheduled the request with print copies = 0.)
Question 7: What happens when the internal concurrent manager dies abruptly? Are all the
managers also killed immediately after it?
Answer
No ALL the standard manager keep running and executing the request. if the internal manager dies,
below queue control request are not performed
f) Sets the target number of processes for each service based on the current work shift.
Question 8: Does the internal manager run or schedule any request for itself?
Answer
No, the internal manager does not run or schedule any requests. It has nothing to do with
scheduling requests, or deciding which manager will run a particular request. Its function is only to
run ‘queue control’ requests
f) Sets the target number of processes for each service based on the current work shift.
We can increase the target processes of the concurrent manager in order to increase the
parallelism. This can be done using define concurrent manager form or through direct update from
sqlplus
Question 10 : If the internal manager goes down, do I need to kill all the managers before restarting
the internal manager?
Answer
No, if the internal manager goes down you need not kill all the managers. You can simply start the
internal manager using startmgr.
Question 11 : What are the problems u have faced while shutting down applications?
Answer
While shutting down application generally concurrent manager won’t go down because some or the
other request may be running. We will see what are the concurrent requests running by querying
fnd_concurrent_requests, fnd_concurrent_program_vl, v$session, v$process and v$sqltext.
If that request is only doing some select statement then we will kill those requests, otherwise we
will check what time it will take to complete by querying the previous runs of that request and
then we will decide what to do.
Answer: Internal Monitors are used specifically in PCP to allow for ICM failover to other available
middle tier nodes.
a) Place an Internal Monitor on any node where the ICM can start in case of a failure.
d) If multiple ICMs are started, only the first will stay active. The others will gracefully exit.
Answer:
Yes, you can delete any concurrent manager. For deleting, query for the manager in the defined
concurrent manager form and then delete the row.
Deleting the predefined concurrent managers is not recommended and it should never be done.
Deletion may cause instability in the system.
Question 14: How can you know which trace file is created for the particular request?
Answer
You can find out the same using the script given below. The trace will be located in the udump
location of the database server.
prompt
accept request prompt ‘Please enter the concurrent request id for the appropriate concurrent
program:’
prompt
set lines 80
set pages 22
fnd_concurrent_programs_v1 prog,
fnd_executables execname
and req.oracle_process_id=proc.spid(+)
and dest.name=’user_dump_dest’
and dbnm.name=’db_name’
and req.concurrent_program_id =
prog.concurrent_program_id
and req.program_application_id =
prog.application_id
and prog.application_id =
execname.application_id
and
prog.executable_id=execname.executable_id;
Answer
In case of parallel concurrent processing, all the managers are assigned a primary and a secondary
node. The managers are started in their primary node by default. In case of node failure or Oracle
instance failure, all the concurrent managers on that node are switched to their secondary nodes.
Once the primary node is available again the concurrent managers on the secondary nodes are
migrated back to the primary node. During the migration process, a manager may be spread across
both primary and secondary nodes.
In case of parallel concurrent processing, it may happen that in a node where parallel concurrent
processing is configured, the Oracle instance may or may not be running. The node which is not
running Oracle, the concurrent managers connects via Net8 to a node which is running Oracle.
The internal concurrent manager can run on any node, and can activate and deactivate concurrent
managers on all nodes. Since the internal concurrent manager must be active at all times, it needs
high fault tolerance. To provide this fault tolerance, parallel concurrent processing uses internal
monitor processes. The job of the internal monitor process is to constantly monitor the internal
manager and start it when it fails. Only one internal monitor process can be active on a single
node. You decide which nodes have an internal monitor process when you configure your system.
You can also assign each internal monitor process a primary and a secondary node to ensure fail
over protection. Internal monitor processes, like concurrent managers, can be assigned work shifts,
and are activated and deactivated by the internal concurrent manager.
Question 16: What are the circumstances in which you need to bounce the concurrent manager?
Answer: There can be many situation where you need to bounce the concurrent manager
b) When you modify the environment variables. Suppose you have changed the APPLTMP and
APPLPTMP variable.
c) When all the requests are pending and hanging and no processing happening
e) We have many global hangs in the system due to locks by several concurrent manager and other
processes
Answer:
The concurrent manager hangs due to many reasons. A few of them are:
– The internal manager was activated by someone other then owner of the application system
– You’ve shut down the internal manager, but actual has a number in it
– The database is hanging may be because the archive log files have filled
Answer: It can be done using profile options “Concurrent: Use ICM”. Set it to “Y” to enable Conflict
Resolution manager. To disable it, set the profile option to “N”.
b) A client makes a request for a specific transaction manager to run a program, and waits for the
results of that program
c) Product teams’ programs are linked directly into the transaction manager executables
Question 20: How the mechanism of viewing log and output files works from browser?
Answer
2. The browser receives the request and spawns the cgi program FNDWRR.exe
4. FNDWRR.exe constructs the service name for the file server on that node. And makes the tns call
to contact the listener for this service name.
5. The listener responds by spawning the local FNDFS executable, as defined in it’s listener.ora file.
Now, FNDFS and FNDWRR.exe are able to communicate directly now, using RPC calls.
6. FNDWRR.exe requests FNDFS to transfer the file that was selected by the user.
7. FNDFS transfers the file contents to a temporary file directory on the web server’s node.
8. The web server displays the file contents to the user.
Question 21: Why does the Concurrent manager put a concurrent program into a queue? Why
doesn’t the manager simply let the program run?
Answer: Because at any given point in time a concurrent manager can run no more than say 10
programs concurrently. This figure of 10 is configurable of course. First the manager puts a
submitted program into a queue, next the manager checks if there is a slot available (i.e. Less than
10 programs are currently running). If a slot is found available, the concurrent manager then runs
the program, or else it keeps the concurrent program in a queue with status Pending.
a) Make certain that there is at least one active manager with specialization rules that allow the
program.
b) If you have confirmed the previous point, then the problem may be a stale Worker Request View
– The view is regenerated when managers are created, or specialization rules are altered
Answer: Service Managers are spawned on the middle-tier nodes of a GSM enabled system in order
to act as an agent of the ICM. When the ICM sees that it needs An Service Manager to perform some
function, such as start a concurrent manager process, on a middle-tier node, it will make remote
procedure control (RPC) calls to the Apps listener on that node to start the Service manager. Once
the Service manager has been started and initialized, the ICM communicates directly to the SM
through RPC, giving it information to manage the services on that node. The SM is spawned from
the APPS TNS Listener.The APPS TNS Listener must be started on every middle-tier node in the
system, and started by the user that starts ICM (e.g. applmgr) .TNS Listener spawns Service
Manager to run as agent of ICM for the local node
The Service Manager is started by ICM on demand when needed. If no management actions are
needed on a node a Service Manager will not be started by ICM until necessary. When ICM exits its
Service Managers exit as well.
Answer: The ICM itself and each process spawned by the ICM has an entry in
FND_CONCURRENT_PROCESSES and holds a uniquely named DBMS lock.
The unique lock of the ICM has a format of FNDCPLK_ICM. This database session lock is the method
that the ICM ensures each PMON cycle that manager and service processes are still alive. If the ICM
can get the DBMS session lock of a process, the ICM will start a new process for that manager or
service.
Concurrent Processing now uses the Output Post Processor (OPP) to enforce post-processing
actions for concurrent requests.Post-processing actions are actions taken on
If a request is submitted with an XML Publisher template specified as a layout for the concurrent
request output, then after the concurrent manager finishes running
the concurrent program, it will contact the OPP to apply the XML Publisher template and create
the final output.
OPP runs as a service that can be managed through Oracle Applications Manager (OAM) from
the System Activity page (Navigation: Applications Dashboard > Applications
$APPLCSF/log/<SID>/FNDOPP####.txt OR
1,System Administrator > Concurrent > Manager > Administer
2,Search for ‘Output Post Processor’
3,Click the ‘Processes’ button .
4,Click the Manager Log button. This will open the ‘OPP’
Upload the OPP log file.
cd $APPLCSF/$APPLLOG
Solution:
1,System Administator > Concurrent > Manager > Administer
2,Query "Output Post Processor" -> Processes button
Get the sytem id of "Output Post Processor"
3,ps -ef|grep [system id]
4,kill -9 [system id] in Unix
5,System Administator > Concurrent > Manager > Administer
6,Query "Output Post Processor" -> Restart button
One or more post-processing actions failed. Consult the OPP service log for details.
Cause:
The concurrent manager process was able to successfully invoke the Output Post-Processor
(OPP) but encountered a timeout as the OPP takes longer than the value
assigned to complete the job.
Solution:
1,Increase the value of profile Concurrent: OPP Response Timeout . Bounce Apache and retest.
2,If the issue still exists, perform the following steps.
3,Increase the number of Output Post Processors as follows:
4,Increase the number of processes for Output Post Processor.
5,Additionally, ensure there is a setting of
oracle.apps.fnd.cp.opp.OPPServiceThread:2:0:max_threads=5 under Parameters.
****
We had an incident for XML reports not being Post Processed. I checked the logs for the reported
request id and one of the processes on amxp05cm01b was giving OutOfMemoryError errors. So I
have restarted OPP on that node. Can you please monitor the logs to check that the issue is not on-
going? We already have the max memory 2GB per process allocated, so if it does re-occur we need
to raise an SR with Oracle.
grep 93399877 *OPP*
FNDOPP334418.txt:[10/6/17 11:52:39 AM] [334418:RT93399877] Output file was found but is zero
sized - Deleted
Troubleshooting:
don’t this this program (Statement Generation Program – Request id’s 88960874 and 88960963 ) is
running through OPP as it was running from Standard Manager2.
I checked ARXSGPO template and looks like default output type defined as “PDF”.
I did see this program was submitted today since a week, as this was executed at first on Thursday
06/01/2017 @ 13:58. There was another concurrent request “Statement Generation Program (XML
Report Publisher)” was executed on May 26th.
Even though the Concurrent Program says XML the Statement Print should come out in PDF based on
below set up:
We have the soft link missing for ARXSGPO.rdf and this rdf was updated on May 28 th with CR #
42406 applied at May 28, 2017 10:03:04 AM CDT on OPRD(adpatch) (application: Oracle
Accounts Receivables). Just recreated the link to point the standard program to custom program. I
had taken the backup of the standard report before creating the soft link. I have submitted the
“Statement Generated Program” and its completed successfully and also spawns the child program
as well. I can see the output in PDF format. Please verify the same and let us know further.
XML Publisher
XML Publisher is a simple engine. It needs only 3 input values to generate report.
1.Data to display on report(in XML format)
2.Layout model/presentation(in XSL-FO format or RTF)
3.Final output type (PDF, Doc, HTML, etc)
Points to Note:
o XML publisher doesn’t care about how you are generating xml data. Only it needs is, your actual
data(text file, excel file, data in DB) must be converted in xml format(calling XML Data) as input to
the engine. To get this, you can write your own xml data or use Oracle Reports to generate xml data or
use any other third party tools to generate xml data from actual data.
o XSL-FO stands for Extensible Stylesheet Language Formatting Objects. Used to present XML Data
in readable and user friendly format. But normal user understands very little in this, so there is an
alternative i.e. user friendly RTF. You do presentation work on RTF file and give this as input to the
XML Publisher engine. XML Publisher converts RTF into XSL-FO.
Run CP
|
———–Oracle Reports(RDF)—–>Generates XML Data–>
| |—>XML Pub Engine–>CP Output(in PDF or etc)
–Finds template from XML Publihser Admin resp–>RTF file–>
Summary Steps:
1. Create Oracle Report(XXXX_TEST.rdf)
2. Define/Register Concurrent Program(XXXX_TEST)
3. Create Template Definition and Template(XXXX_TEST.rtf)
4. Run CP and Show XML Publisher report in PDF format(XXXX_TEST_OUTPUT.pdf)
Concurrent Manager
Manager status
Concurrent Queue
WHERE a.concurrent_queue_id=b.concurrent_queue_id
AND b.process_status_code='A'
ORDER BY b.process_status_code;
select fcp.node_name,
fcp.logfile_name
AND fcq.concurrent_queue_name = 'FNDCPOPP'
order by 2 desc;