CA Wily Introscope

Workstation User
Guide
Version 8.2

Date: 10-2009

Copyright 2009, CA. All rights reserved.

Wily Technology, the Wily Technology Logo, Introscope, and All Systems Green are registered
trademarks of CA.
Blame, Blame Game, ChangeDetector, Get Wily, Introscope BRT Adapter, Introscope
ChangeDetector, Introscope Environment Performance Agent, Introscope ErrorDetector, Introscope
LeakHunter, Introscope PowerPack, Introscope SNMP Adapter, Introscope SQL Agent, Introscope
Transaction Tracer, SmartStor, Web Services Manager, Whole Application, Wily Customer Experience
Manager, Wily Manager for CA SiteMinder, and Wily Portal Manager are trademarks of CA. Java is a
trademark of Sun Microsystems in the U.S. and other countries. All other names are the property of
their respective holders.
For help with Introscope or any other product from CA Wily Technology, contact Wily Technical
Support at 1-888-GET-WILY ext. 1 or [email protected].
If you are the registered support contact for your company, you can access the support Web site
directly at www.ca.com/wily/support.
We value your feedback
Please take this short online survey to help us improve the information we provide you. Link to the
survey at: http://tinyurl.com/6j6ugb
If you have other comments or suggestions about Wily documentation, please send us an e-mail at
[email protected].

6000 Shoreline Court, Suite 300

South San Francisco, CA 94080

US Toll Free 888 GET WILY ext. 1

US +1 630 505 6966
Fax +1 650 534 9340
Europe +44 (0)870 351 6752
Asia-Pacific +81 3 6868 2300
Japan Toll Free 0120 974 580
Latin America +55 11 5503 6167
www.ca.com/apm

CONTENTS

Table of Contents

Chapter 1

Introscope Workstation Overview .

10

The managed application and Introscope components

10

How the Workstation fits in an Introscope installation

11

Introscope and the Workstation .

.
.

.
.

.
.

.
.

Administering the Workstation

12

Starting the Workstation .

12

Executing Workstation functions from the command line

17

Ending your workstation session .

18

Configuring HTTP tunneling for the Workstation.

19

Configuring the Workstation to use SSL

20

Introscope Workstation elements

21

About the Workstation Console

21

About the Workstation Investigator .

22

About the Management Module Editor .

24

About the Dashboard Editor

24

About Data Viewers

25

Workstation help

27

29

User permissions

29

User preferences

29

Managing language settings

31

. 33

Managing users.

Chapter 2

Using the Workstation Console

About the Workstation Console .

.
.

.
.

.
.

34

Navigating among dashboards in the Console .

34

Dashboard drop-down list .

34

Navigating among dashboards.

35

Using hyperlinks to navigate

35

Contents iii

CA Wily Introscope

Creating dashboard favorites .

Chapter 3

36

Manipulating the contents of Data Viewers .

37

Displaying minimum/maximum metric values in a graph

37

Using tool tips to view metric names and values in a Data Viewer

38

Showing/hiding metric data in a graph .

40

Changing the scale of graph charts .

41

Moving metrics to front/back in graph .

43

Copying a Data Viewer to the clipboard

44

Exporting data from Data Viewers

44

Viewing data in the Console

44

Viewing historical data

45

Filtering by agent with the Console Lens .

48

Applying the Console Lens .

49

Clearing the Console Lens .

50

Console Lens and tab views in dashboards

50

. 53

Using the Workstation Investigator

About the Workstation Investigator .

54

About the Investigator tree .

55

Viewer pane .

57

Navigating in the Workstation Investigator .

58

About the Investigator tab views

59

General tab .

59

Overview tabs

60

Search tab

81

Traces tab .

82

Errors tab .

82

83

Viewing Blame information in the Investigator tree .

85

Viewing metrics for Frontends in the Investigator .

85

Viewing metrics for Backends in the Investigator .

87

Metric Count tab

Viewing data in the Investigator .

89

89

Working with alerts in the Investigator

92

Exporting data from the Investigator .

93

Viewing historical data

iv Contents

Workstation User Guide

Chapter 4

Introscope Sample Dashboards .

About the Introscope sample dashboards.

. 95

96

Dashboard alerts that show overall status.

97

The Overview dashboard

99

Alerts showing overall status .

99

Overview dashboard graphs

. 100

The Problem Analysis dashboard.

. 101

How alerts are defined using heuristic metrics .

. 102

. 103

105

Eliminating alerting on transient spikes

Chapter 5

Using the Introscope Transaction Tracer

About the Transaction Tracer .

. 106

Automatic transaction trace sampling .

. 106

Transaction Trace overhead

. 107

Transaction Tracer compatibility with agents from previous releases 107

Starting, stopping, and restarting a Transaction Trace .
Starting a Transaction Trace session

. 108

. 108

Stopping a Transaction Trace session .

. 110

Restarting a Transaction Trace session .

. 110

. 110

Turning off low-threshold execution time warnings

. 110

Reviewing agents targeted for tracing .

Transaction Trace session options .

Using the Transaction Trace Viewer.

. 111

. 112

Summary view .

. 114

Trace view.

. 115

About the Tree view .

. 120

Viewing aggregated data for multiple transactions .

. 120

Printing a Transaction Trace window

. 121

Querying stored events .

. 122

. 122

Querying historical events .

. 123

Saving and exporting Transaction Trace information .

. 126

. 126

Exporting selected Transaction Trace to a text file .

. 127

129

. 130

Query syntax .

Saving Transaction Trace data.

Chapter 6

Introscope Reporting .
Creating report templates .

.
.

.
.

.
.

.
.

.
.

.
.

.
.

.
.

.
.

.
.

.
.

Contents v

CA Wily Introscope

Adding report elements to reports

Appendix A

. 132

Defining properties in the Report Editor

. 134

Setting custom group definitions .

. 145

Time series bar charts

. 148

Working with report templates

. 152

Copying or deleting report templates

. 152

Generating reports from report templates.

. 152

Introscope sample report templates .

. 154

157

Introscope Metrics .

.
.

.
.

How Introscope measures application health

. 158

Common terms .

. 158

Types of metrics

. 160

Viewing metrics .

. 160

The five basic Introscope metrics

. 161

Average Response Time (ms) .

. 161

Concurrent Invocations .

. 163

Errors Per Interval .

. 165

Responses Per Interval .

. 166

Stall Count

.
.

. 168

Other common metrics .

. 169

Memory-related metrics .

. 169

Utilization metrics .

. 170

Socket metrics .

. 171

Thread pool metrics

. 172

Connection pool metrics .

. 172

Event metrics

. 174

Using perflog.txt

. 175

Other metrics
EJB .

vi Contents

. 175

. 175

Servlets

. 176

JDBC

. 176

JSP (Java Server Pages) .

. 177

RMI (Remote method invocations) .

. 179

XML (Extensible Markup Language) .

. 179

J2EE Connector .

. 180

JTA (Java Transaction API) .

. 181

Workstation User Guide

JNDI (Java Naming and Directory Interface) .

. 181

JMS (Java Messaging Service) .

. 182

Java Mail .

. 183

CORBA .

. 183

Struts .

. 184

Instance Counts .

. 184

Data about machines.

. 184

. 185

193

Enterprise Manager health and supportability metrics

Index .

Contents vii

CA Wily Introscope

viii Contents

CHAPTER

Introscope Workstation Overview

Introscope enables you to manage your applications performance. You use the
Introscope Workstation to view and manipulate data that is stored by the
Enterprise Manager.
This guide describes the Workstation components youll use on a daily basis to
monitor and manage your application, including the Workstation Console,
Investigator, Sample Dashboards, Transaction Tracer, and Reporting.
This chapter includes these topics:
Introscope and the Workstation

10

Administering the Workstation .

12

Introscope Workstation elements .

21

Managing users

29

Introscope Workstation Overview 9

CA Wily Introscope

Introscope and the Workstation

Introscope, through the ProbeBuilder, adds Introscope probes to a Java
Application. Using AutoProbe automates this process, with the ProbeBuilder
dynamically adding probes to the Java Application when the application starts.
The probes measure specific pieces of information about an application without
changing the applications business logic. An Introscope agent is installed on the
same machine as the instrumented application. After the probes have been
installed in the bytecode, the Java application is referred to as an instrumented
application. When the Java application with probes is running, it is called a
managed application.
This illustration shows how Introscope prepares a Java application to be
managed:

The managed application and Introscope components

As a managed application runs, probes relay collected data to the agent. The
agent then collects and summarizes the data and sends it to the Enterprise
Manager.
Data collected by the Enterprise Manager can be accessed through one or more
Workstations. You can use the Workstation to view performance data, and
configure the Enterprise Manager to perform such tasks as collecting information
for later analysis, and creating alerts.
As a managed application runs, Introscope agents collect performance data in
real time, and send the information to the Enterprise Manager. The Workstation
allows you to configure the Enterprise Manager, organize metrics, define actions
based on their values, and display the information you choose in the most
convenient format for you.

10 Introscope Workstation Overview

Workstation User Guide

How the Workstation fits in an Introscope installation

The Introscope Workstation tools help you do the following to better monitor
application performance:
Filter and view performance metrics for various elements of the system your

application runs on.

Drill down to uncover the root cause of system performance issues.
Create graphical displays of metrics.
Create reports of system performance data.

The Workstation, Java Web Start, and WebView

Many users find it easy to access the Workstation by using either Java Web Start
or WebView.
It is important to understand the difference between these two methods. Java
Web Start uses a command or browser to download and invoke a full Workstation
client, while WebView allows users to launch a simplified, in-browser, read-only
alternative to the full Workstation tools. The table below contrasts WebView and
Java Web Start.

Introscope and the Workstation 11

CA Wily Introscope

Method of
invocation

Java Web Start

WebView

Do one of these:

Use a URL like:

Use a command like java

http://WebViewHost:8080

-client -Xms64m -Xmx256m -jar

launcher.jar
Use a URL like http://
EMHost:8081/workstation
in which EMHost is the machine
where Enterprise Manager is
installed.

in which WebViewHost is the

name of the machine where
WebView is installed.

See Starting the Workstation on

page 12 for more information.
What
happens

The EM downloads and launches a

.jar file containing a full working
copy of the Workstation.

The WebView server returns

an HTML display of Console or
Investigator content.

See Designating the Java version for

Java Web Start on page 13 for more
information.
What the
user can do

Use full workstation functionality.

Browse metric data and

graphical metric data
displays.

For more information on WebView, see the Introscope WebView Guide.

Administering the Workstation

This section has information on starting and stopping the Workstation, and
configuring it for tunnelling and for SSL.

Starting the Workstation

To start Introscope Workstation:
1 Launch the Introscope Workstation using one of these methods:
On Windows, you can:

Run Introscope Workstation.exe.

Click Start > Introscope > Introscope Workstation

Using a browser with a URL like:

http://EMhost:8081/workstation
where EMhost is the hostname of the Enterprise Manager (EM).

12 Introscope Workstation Overview

Workstation User Guide

You can also use a more complex URL that specifies which page of the
Workstation to start at. See Launching the Workstation using specific
parameters on page 15.
Using the command linesee Executing Workstation functions from the

command line on page 17.

2 In the login dialog, enter:
the Host name or IP address

Note You can use the IP address instead of the host name only if both your
client machine and the host machine support the same IP protocol.
the Port number
your User Name
your Password

Tip

The Workstation will remember the last five login attempts, so if you
have entered host and user information previously, the Host, Port and
User ID fields will be drop-downs from which you can select the
credentials you want to use.

3 Click Connect, or to make the current host and user information the default for
future log-ins, click Set Defaults.
If authentication was successful, the Console opens. If authentication was
unsuccessful, a message notifies you of the failure and the Introscope
Workstation Login window reopens.
Note If a user tries to log in but does not have permissions defined in
domains.xml or server.xml, Workstation login fails.

Designating the Java version for Java Web Start

Using Java Web Start downloads a temporary copy of the Workstation client from
the EM to your machine. Machines using proxy authentication to connect to an
Enterprise Manager might not automatically download the correct JVM if it is
missing. If you encounter this problem, install the correct version of JVM
manually before attempting to use Java Web Start.
On the client system, Java Web Start will launch the workstation using a Java
version defined by these two files in the Enterprise Manager:
<iscroot>\product\enterprisemanager\plugins\com.wily.introscope

.workstation.webstart_8.0.0\WebContent\jnlp\workstation.jsp
<iscroot>\product\enterprisemanager\plugins\com.wily.introscope

.workstation.webstart_8.0.0\WebContent\jnlp\com.wily.introscop
e.workstation.feature_8.0.0.jsp

Administering the Workstation 13

CA Wily Introscope

In both of these files, you will find a j2se node with a version attribute to use in
determining the Java version to be used to launch the Workstation:
<j2se version="1.5*&1.5.0_15+ 1.6*&1.6.0_05+" ... />

Note In order to parse these instructions, the client system must have an
installed JVM of 1.5.0 or later. If it has JVM 1.4.2 or earlier, or no JVM at
all, it will not be able to read these instructions. In this case you should
download JVM 1.5.0_15 from the java.sun.com website, then proceed.
The attribute lists Java version ranges in preferred order from first to last. Each
range is separated by a single space. Java Web Start checks the client system for
all version ranges in the listed order and installs the first qualifying version that
it finds.
If Java Web Start does not find a pre-installed JVM that matches the listed version
ranges, it downloads the newest available version permitted by all of the version
ranges taken together, regardless of order.
For example: When the j2se version is set to "1.5*&1.5.0_15+

1.6*&1.6.0_05+" ...
Web Start will first check the client system for Java 1.5, update 15 or later.

1.5*
= version starts with 1.5
&
= and
1.5.0_15+ = version is 1.5 update 15 or later
If a valid update level of Java 1.5 is not found, Web Start will then check the

client system for Java 1.6, update 5 or later.

1.6*
= version starts with 1.6
&
= and
1.6.0_05+ = version is 1.6 update 5 or later
If a valid update level of Java 1.6 is not found, Web Start downloads the newest

available version permitted by any of the version ranges.

It would never download Java 1.7 since the version range does not permit it.

CA Wily has deliberately chosen 1.5.0_15 and 1.6.0_05 as minimum update

levels since these updates are the earliest to address critical security issues in
Java (see http://nvd.nist.gov/nvd.cfm?cvename=CVE-2008-1185). Users can
customize the ranges to allow earlier update levels at their own risk.
Note If you edit the version range list, be sure to edit it in both files:
workstation.jsp and
com.wily.introscope.workstation.feature_8.0.0.jsp.

14 Introscope Workstation Overview

Workstation User Guide

Launching the Workstation using specific parameters

You can launch Workstation using specific parameters that specify which
Workstation page you want to access. These parameters can be used in either:
a Java launch command issued on the command line
a URL that launches the Workstation using Java Web Start

For example, in the command line, the -page and -agent options would be used
like this:
java -client -Xms64m -Xmx256m -Dsun.java2d.noddraw=true -jar launcher.jar consoleLog -noExit -product com.wily.introscope.workstation.product name "Introscope Workstation" -install ".\\product\\workstation" configuration ".\\product\\workstation\\configuration" -page
investigator -agent "SuperDomain|localhost|WebLogic|WebLogic Agent"

In a URL, the same combination would be referred to like this:

http://localhost:8081/
workstation?page=investigator&agent=SuperDomain|localhost|WebLogic|Web
Logic%20Agent
Note the way each of the above examples handles the space character in the
agent name.
In the command line example, quotes are used around the entire agent name

because the name contains a space.

In the URL example, a space character is rendered as %20.

The table below lists other parameters you can use.

Options

Description

Suppresses the login screen and logs into Workstation

loginimmediate immediately using specified hostname and port number, or
default values.

Use the next three parameters to enter login information:

-loginhost
<hostname>

Specifies login host name; defaults to localhost if

unspecified.

-loginport
<portnumber>

Specifies login port number; defaults to 5001 if unspecified.

-loginresponse Specifies authentication values for username and password in a

comma-separated list.
<values>

Administering the Workstation 15

CA Wily Introscope

Options

Description

-page

The name of the Workstation screen to be launched. You must

include this parameter with every request to the Workstation
Command Line Interface.
Supported values:
investigator
historicalquery
console

-agent

The fully qualified agent name to display in the Investigator

window. Required if the page parameter is investigator.

-metric

The metric path to display in the Investigator window, for a

specified agent. You must specify an agent if you use this
parameter.

-start

The start time, in standard Java format of milliseconds, for a

historical time range in the Investigator window, or the start
time for a transaction tracer Historical Query, depending on the
value of the page parameter.

Note: The start/end or guid parameters are required if the

page parameter is historicalquery.

-end

The end time, in standard Java format of milliseconds, for a

historical time range in the Investigator window, or the end time
for a transaction tracer Historical Query, depending on the value
of the page parameter. The start/end or guid parameters are
required if the page parameter is historicalquery.
For example:

http://localhost:8081/
workstation?page=historicalquery&start=1135686483474&e
nd=1136686483474

-guid

The unique identifier for a transaction to display in the

transaction tracer Historical Query window. The start/end or
guid parameters are required if the page parameter is
historicalquery.
For example:

http://localhost:8081/
workstation?page=historicalquery&guid=aRx345

16 Introscope Workstation Overview

Workstation User Guide

Options

Description

Filters data to limit the dashboard display to data from the agent
agentSpecifier you specify. Can be used only when the page parameter =
console.
The argument to the AgentSpecifier parameter must contain the
agent name including the Enterprise Manager host name;
special characters, such as the | symbol which separates
elements of the agent name, must be escaped with backslashes.
Substitute the string %20 for spaces in agent names.
In this example, the dashboard will display only data from
WebLogic Agent:

http://localhost:8081/
workstation?page=console&agentSpecifier=machine1\|WebL
ogic\|WebLogic%20Agent&metric=GC%20Heap:Bytes%20In
%20Use

-dashboardName Specifies a dashboard to display. Can be used only when the

page parameter = console.
Substitute the string %20 for spaces in dashboard names.
In this example, the URL will jump to the dashboard called GC
Memory In Use:

http://localhost:8081/
workstation?page=console&dashboardName=GC%20Memor
y%20In%20Use&metric=GC%20Heap:Bytes%20In%20Use
Executing one of the URLs above (or launching a Workstation with an equivalent
Java command line) starts a Workstation instance and opens the appropriate
window. Subsequent URL requests open a new window in the existing
Workstation instance.

Connecting to alternate Enterprise Managers

You can start multiple Workstation application instances on different Enterprise
Manager hosts from a single browser, using the parameters specified in
Launching the Workstation using specific parameters on page 15. To connect to
an alternate or different EM host, change the loginHost parameter as
appropriate.

Executing Workstation functions from the command line

You can execute Workstation functions from a command line. This is useful if you
need to execute these functions from a script for the purpose of batching or
scheduling the functions.
Command Line Workstation is described at much greater length in the Introscope
Configuration Administration Guide.

Administering the Workstation 17

CA Wily Introscope

To execute Workstation functions from the command line:

1 Change to the <Introscope_Home> directory.
2 Execute the Workstation start command, using the examples below as models.
Here is an outline of the command:
java [optional arguments] -jar launcher.jar [Eclipse arguments]

Here is an example of a full Workstation start command:

java -client -Xms64m -Xmx256m -Dsun.java2d.noddraw=true -jar launcher.jar
-consoleLog -noExit -product com.wily.introscope.workstation.product name "Introscope Workstation" -install ".\\product\\workstation" configuration ".\\product\\workstation\\configuration"

Follow these guidelines:

On UNIX, change escaped backslashes to forward slashes.
If adding your own optional JVM arguments, insert them before the -jar

argument. The following arguments appear in the example.

-clientRuns the JVM in client mode

-Xmsinitial Java heap size

-Xmxmaximum java heap size for the application to use

-Dsun.java2d.noddraw=trueOptional. Helps resolve potential

difficulties between drivers and Java APIs.

Modifying the Eclipse arguments (everything from -consoleLog onward) is not

recommended except at the request of CA Wily support.
Additional parameters available for using Command Line Workstation are listed
in the table in Launching the Workstation using specific parameters on page 15.

Ending your workstation session

Logging out of the Workstation
Logging out of the Workstation ends the current session, but does not shut it
down, so that you can log in again from the Authentication dialog box. This is
useful if you want to log in with different connection parameters, such as a
different host, port, user name, or password.
When you log out of the Workstation, it saves the number of open Investigator
and Console windows, so the same configuration appears when you next log in.
To log out of the Workstation:
Select Workstation > Logout.

18 Introscope Workstation Overview

Workstation User Guide

Exiting the Workstation

Exiting the Workstation logs you out of the Workstation and stops the
Workstation process.
When you exit the Workstation, it saves the number of open Investigator and
Console windows, so the same configuration appears when you next log in.
To exit the Workstation:
Select Workstation > Exit Workstation.

Performing Workstation functions from the command line

Many Workstation functions can be performed from a command line using the
Introscope Command-Line Workstation. See the Introscope Configuration and
Administration Guide for more information.

Configuring HTTP tunneling for the Workstation

You can configure the Workstation to connect through a proxy server to the
Enterprise Manager. This is necessary for a forward-proxy server configuration
where the Workstation is running behind a firewall that only allows outbound
HTTP traffic routed through the proxy server.
Note Because tunneling imposes additional CPU and memory overhead on the
managed host and Enterprise Manager beyond that expected for a direct
socket connection, do not set up Workstation HTTP tunneling if a direct
socket connection to the Enterprise Manager is feasible.
Important HTTP/1.1 is required to enable Workstation HTTP tunneling.
To use Workstation tunneling:
Edit the HTTP Tunneling Proxy Server section of

IntroscopeWorkstation.properties to specify the tunneling connection:

a Uncomment the lines beginning with transport.http...
b Provide the host, port, username and password of the proxy server.
#################################
# HTTP Tunneling Proxy Server
#---------------------# These properties apply if the Workstation is tunneling over HTTP
# and must connect to the Enterprise Manager through a proxy server
(forward proxy).
# If the proxy server cannot be reached at the specified host and port,

Administering the Workstation 19

CA Wily Introscope

# the Workstation tries a direct HTTP tunneled connection to the

Enterprise Manager
# before failing the connection attempt.
#transport.http.proxy.host=
#transport.http.proxy.port=
# These properties apply if the proxy server requires authentication.
#transport.http.proxy.username=
#transport.http.proxy.password=

Configuring the Workstation to use SSL

The Workstation ordinarily uses HTTP to connect to the Enterprise Manager (EM).
You can configure connections through HTTPS/SSL, optionally using certificates.
To configure the Workstation to connect to the EM using SSL, you edit the
IntroscopeWorkstation.properties file for the following properties:
Property

Description

Example

truststore location path

Path to the location of a

truststore containing
trusted EM certificates.

transport.tcp.truststore=
C:\\Introscope\\config\\internal
\\server\\keystore

Note that on Windows, a

backslash must be
escaped with another
backslash.

transport.tcp.trustpasswo Password for the

certificate truststore
rd
workstation certificate location
path

transport.tcp.trustpassword=passw
ord

Path to the location of the See example above. Again, backslashes

trusted certificate for the must be escaped.
Workstation

transport.tcp.keypassword Keystore password

transport.tcp.keypassword=
password

transport.tcp.ciphersuites=
transport.tcp.ciphersuite List of cipher suites,
separated by commas. If SSL_DH_anon_WITH_RC4_128_MD5,
s
this property is blank,
SSL_RSA_WITH_NULL_MD5
Workstation will use the
default list.

Things to note:
Specify a truststore to configure the Workstation to authenticate the server

(EM). If no truststore is specified, the server is automatically trusted.

Specify a keystore only if the EM has been configured to require client

authentication.

20 Introscope Workstation Overview

Workstation User Guide

Introscope Workstation elements

You use the Workstation to view metric data in different forms. Authorized users
can perform administrative and configuration functions. The Workstation
presents information in these windows:
Consoleshows data in dashboards, which contain Data Viewers.
Investigatorpresents a tree view of agents, applications, resources, and

metrics.
Management Module Editorpresents a tree view of Management Modules and

elements.
Dashboard Editorenables users with write permission for a Domain (or

SuperDomain) to create and edit Data Viewers and other dashboard objects
such as imported images, shapes, lines, and text.
Data Viewervisual presentation of data based on the type.

About the Workstation Console

The Console is the default view when you start the Workstation, and contains
dashboards that show performance data in graphical views. Dashboards are basic
tools for viewing management data in Introscope.
The Introscope Sample Management Module provides a set of sample
dashboards. Authorized users can create custom dashboards using the
Dashboard Editorsee the Introscope Configuration and Administration Guide for
more information.

Introscope Workstation elements 21

CA Wily Introscope

You can have more than one Console window open at the same time. To open a
new Console window, select Workstation > New Console. This illustration
shows the Overview sample dashboard:

For more information, see Using the Workstation Console on page 33.

About the Workstation Investigator

You use the Investigator to view application and system status, to search, and to
browse metric data using a tree structure.
You can have more than one Investigator window open at the same time.
To open a new Investigator window:
Select Workstation > New Investigator.

You can also open an Investigator window from the Console by double-clicking on
some dashboard elements, depending on how the element was created. See
Using hyperlinks to navigate on page 35.

22 Introscope Workstation Overview

Workstation User Guide

The Investigator opens, showing data for your Java or .NET application.

This example shows data for

a Java application.

This example shows data for

a .NET application.

The Investigator contains these sections:

The Investigator tree provides information about each host and application

managed by the Enterprise Manager. The metrics that appear in the

Investigator tree are a function of the resources your applications use and the
data that your Introscope agents are configured to report.
The Viewer pane on the right side of the Investigator presents details, often

graphical, for the resource or metric in the tree. You can select View tabs to
open different views of data. The tabs that are available vary, depending on
the item selected in the tree. For some views, options might be available in the
bottom section of the Viewer pane to control the data displayed in the Viewer.
For more information, see Using the Workstation Investigator on page 53.

Introscope Workstation elements 23

CA Wily Introscope

About the Management Module Editor

You use the Management Module Editor to create or edit a Management Module,
which contains a set of Introscope monitoring configuration information.
Management Modules are listed for each domain, and contain objects, known as
elements, that contain and organize data with monitoring logicalerts, actions,
and dashboards.
Note If you have a full Wily Introscope license, you can create, edit, or delete
information in the Management Module Editor. If you do not have a full
license, you can only view information here.

The Management
Module Editor tree
lists the
Management
Modules deployed
to the Enterprise
Manager, by
domain, and the
elements in each
Management
Module.

The right side of the

Management Module Editor
presents the current
configuration settings for
the element selected in the
tree. An authorized user can
modify elements in the
Management Module Editor.

For more information about using the Management Module Editor to modify
elements, see the Introscope Configuration and Administration Guide.

About the Dashboard Editor

The Dashboard Editor provides tools for creating and laying out Data Viewers,
shapes, lines, text boxes, and connectors. Users with appropriate permissions
can create and edit dashboards and dashboard objects such as imported images,
shapes, lines, and textsee the Introscope Configuration and Administration Guide
for more information.

24 Introscope Workstation Overview

Workstation User Guide

About Data Viewers

Data Viewers in the Investigator Viewer pane or in a dashboard show data from
an Introscope-enabled application in a visual form. Data appears in a Data Viewer
based on the type of datafor example, metrics appear as graphs, and alerts
(formerly called traffic lights) appear as indicators. Data Viewers can display data
from a metric, a resource, or an element, such as an alert.

Data Viewer types

These are the default viewer types for the specified type of data.
Data type

Default Data
Viewer type

Can also be viewed as

Metric

Graph

Dial Meter, Bar Chart, Graphic Equalizer, String

Viewer, Text Viewer

Metric Grouping

Graph

Bar Chart, String Viewer

Alert

Alert indicator

Graph, Bar Chart, or String Viewer

Calculator

Graph

Dial Meter, Bar Chart, Graphic Equalizer, String

Viewer

Depending on the type of metric or element, Introscope can display the data in a
Data Viewer with the view display types shown here.
Graph

Graphs plot values over time. In real-time

views, the graph dynamically displays the most
recent time period that fits in the graph.
For historical views, you can determine what
time period is displayed.
If the graph displays an alert, caution and
danger thresholds appear as yellow and red
lines, respectively.

Introscope Workstation elements 25

CA Wily Introscope

Bar Chart

Bar charts show current data values as

horizontal bars.
The bar chart is the default view for Top N
Filtered Views.
If a bar chart is showing an alert, the bars are
either green, yellow or red to correspond to
alert status.
The bar chart is available for live data viewing
only (not available for viewing historical data).

Graphic
Equalizer

Graphic equalizers show the current value of

the data, as well as recent high levels.
A graphic equalizer can show data only for a
single metric.

Dial Meter

Dial meters show current data values as

positions on a half-round dial.
A dial meter can only show data for a single
metric.

String
Viewer

String viewers can show a value as a line of

text. String viewers allow some values to be
displayed in a relatively small space. You can
also use a string viewer for simple values that
do not change, such as Launch Time or IP
Address.

Text
Viewer

Text viewers show the text for data where new

values are added to old ones, or for text-type
datafor example, a system or exception log.

26 Introscope Workstation Overview

Workstation User Guide

Alert

Alert indicators show the current status of an

alert by lighting one of three colored symbols
that correspond to conditions defined in the
alert:
Red octagon = danger threshold was crossed
Yellow diamond = caution threshold was
crossed

Green disc = status normal

If the alert has no data, the alert indicator is a
gray disc.
An alert indicator with three color states can
also be shown as a single symbol.

Workstation help
Online documentation
To open Workstation Help:
1 From the Help menu, select Help and Documentation.
The top-level Help window appears.

Select one of the topic

areas here to see the
content in searchable
HTML.

Click this link to see a list of

books available in PDF
format.

Introscope Workstation elements 27

CA Wily Introscope

2 Select from the two available Help links.

Use the Search tab to
search for a topic
Use the Index tab to see
an alphabetical index
Click the open book icon to
contract the TOC to that level

Click a closed book icon to

expand a topic

Note On the UNIX platform, the Help system is hard-coded to use the Mozilla
browser. You must have Mozilla in your classpath for the links displayed
in the top-level Help window to be functional.

Troubleshooting
Difficulty bringing up Searchable Help is commonly due to a network problem of
some kind. Possible issues include DNS, domain suffixes, or VMWare
configurations. A workaround for all of these causes is to use the IP address
instead of the hostname of the target Enterprise Manager machine when you log
in to Workstation. For more information, see Knowledge Base article 1673.

PDF-format documentation
The same books viewable in online help are available in PDF format for download.
To download PDF-format documentation:
1 From the Help menu, select Help and Documentation.
The top-level Help window appears.
2 Click Individual Introscope Guides (PDF format).
3 View a book:
Click the name of an individual book to view it in a PDF-enabled browser.

or
Right-click the name of an individual book and save it to your computer.

28 Introscope Workstation Overview

Workstation User Guide

Managing users
User permissions
In Introscope, Workstation users are assigned user permissions. Each
Workstation user is assigned a user name, password, and certain permissions.
Permissions are granted at the Domain and Enterprise level. Some Workstation
functions require specific permissions. For example, to publish a MIB
(Management Information Base, a directory of information used by network
management protocols), a user must have publish_mib permission for the
server. Your Introscope administrator assigns these to you.
If you do not have sufficient permissions for a function, the function is disabled.
For more information about user permissions, see the Introscope Installation and
Upgrade Guide.

User preferences
You use Introscope user preferences to specify:
a home dashboard
whether to display Management Module names alongside dashboard names in

the Console
low-threshold execution-time warnings for Transaction Tracer.

Setting a home dashboard

Dashboards are pre-configured windows that present graphical views of current
or historical performance and availability metrics.
To change your home dashboard:
1 Select Workstation > User Preferences.
2 Select a dashboard by doing one of these:

Managing users 29

CA Wily Introscope

Select a dashboard from the dropdown list.

Click Choose, enter a search string to narrow the selection, and select from

the remaining list.

3 Click Apply.

Displaying a dashboards Management Module and Domain

You can use the same name for dashboards that are in different Management
Modules, and use the same name for Management Modules that are in different
Domains.
You can set User Preferences to display the name of the Management Module and
Domain that contain the dashboard.
To display the Management Module name next to the dashboard name:
1 Select Workstation > User Preferences.
2 Check Show Module and Domain name with Dashboard name.
3 Click Apply.
The Management Module and domain that contain the dashboard appear after the
dashboard name.
Note Domain information does not appear if you have access to only one
Domain.

30 Introscope Workstation Overview

Workstation User Guide

Turning off low-threshold execution time warnings

If you are running the Transaction Tracer and set the threshold execution time to
less than one secondto perform a deep analysis, for exampleyou might see
continual warnings. The warnings indicate increased overhead because of
increased traces, so you might want to turn them off in a production
environment.
To turn off the warnings about low-threshold execution time:
1 Select Workstation > User Preferences.
2 Click the Transaction Tracer tab.
3 Check the Dont warn when threshold is less than 1 second checkbox.
For more information about Transaction Tracing, see Using the Introscope
Transaction Tracer on page 105.

Managing language settings

When using the Workstation tools:
User dialogs reflect the regional language set in the Control Panel on your

computer.
You can set properties in Introscope reports to use a specific language setting

separate from the regional language set for your computer. See Defining
properties in the Report Editor on page 134.

Managing users 31

CA Wily Introscope

32 Introscope Workstation Overview

CHAPTER

Using the Workstation Console

This chapter describes how to use the Introscope Workstation Console. It includes
these topics:
About the Workstation Console

34

Navigating among dashboards in the Console .

34

Creating dashboard favorites .

36

Manipulating the contents of Data Viewers .

37

Viewing data in the Console

Filtering by agent with the Console Lens

44

48

Using the Workstation Console 33

CA Wily Introscope

About the Workstation Console

The Workstation Console displays metric information in dashboards. Dashboards
are pre-configured windows that present graphical views of current or historical
performance and availability metrics.
When you open the Console, it shows live performance and availability data. You
can view historical data by selecting a time rangesee Viewing historical data on
page 45.
Note You can also easily get to the Investigator from the Console by doubleclicking any element in the Console. A new Investigator window will open
with data on the element you selected.

Navigating among dashboards in the Console

You can select Console dashboards in several different ways:
Dashboard drop-down list
Forward and backward buttons
History list
Home button
Hyperlinks

Dashboard drop-down list

You can select dashboards from the drop-down list at the top of the Console page.
You can type all or part of the dashboard name, to narrow the selections in the
list.

34 Using the Workstation Console

Workstation User Guide

Navigating among dashboards

After you have viewed several dashboards, you can navigate among them:
You can move among previously viewed dashboards with the Forward
and Back arrow buttons...

...and use the drop-down list on the

Forward or Back arrow to select a
previously viewed dashboard.

If you have defined a home dashboard in your User Preferences, you can open it
by clicking the Home button.

Using hyperlinks to navigate

You can use hyperlinks to navigate between Introscope dashboards and the
Investigator:
Automatic hyperlinksIntroscope automatically links a Data Viewer to the

metric grouping it is based uponthe Links menu for the viewer contains a link
to the underlying metric grouping definition in the Management Module Editor.
Similarly, dashboards that contain Data Viewers based on the same metric
grouping are automatically linked, and you can navigate between them using
the Links menu.
Custom hyperlinksYou can define custom links for dashboard items, to link

to other dashboards or to web pages. You can define custom links if you have
dashboard editing permission.
Note Some out-of-the-box Console dashboardsfor example, EM Capacity
do not automatically contain links to underlying data. Edit these default
dashboards or create new dashboards with links. For information about
creating and editing custom links, see the Introscope Configuration and
Administration Guide.
To follow dashboard links:
1 Hover your cursor over a dashboard object that has a hyperlink.
The pointer changes to a hand.

Navigating among dashboards in the Console 35

CA Wily Introscope

2 Double-click the object to follow the link to its default target.

To

Do this

See a list of available links

Select a dashboard object and select

Properties > Links.

Right-click the dashboard object and

select Links from the context menu.

See the target of a hyperlink in a new
window

Press Shift and click the object

If no links are available for an object, the Links menu is disabled.

Creating dashboard favorites

To simplify access to dashboards that you use often, you can add them to the
Console Favorites menu.
To

Do this

Add a favorite
dashboard

1 Open a Console window.

2 Navigate to the dashboard to add to your Favorites.
3 Select Favorites > Add to Favorites.

Change the order of

Favorites

1 Open a Console window and select Favorites > Organize Favorites.

2 Select the dashboard to move in the list, click Move Up or Move Down, and
move the dashboard to the appropriate position in the list.
You can also click and drag items up or down in the list.
3 Click OK.

Delete a dashboard
favorite

1 Open a Console window and Select Favorites > Organize Favorites.

2 Select the dashboard to delete from the list, and click Delete.
3 Click OK.

Edit dashboard
favorites

1 Open a Console window and select Favorites > Organize Favorites.

In the list, any dashboards Favorites whose associated dashboards have been
renamed or deleted are indicated by an exclamation point icon.
2 Select the dashboard to edit, and click Edit.
3 Select a dashboard from the drop-down list and click OK (or click Choose,
select a dashboard, click Choose again, then click OK).

Note Favorite links are not retained when you rename or delete a favorite
dashboard. You need to update the link, or delete the old link and create
a new one.

36 Using the Workstation Console

Workstation User Guide

Manipulating the contents of Data Viewers

Data Viewers in the Investigator viewer pane or in a dashboard show data from
an instrumented application in a visual form. Data appears in a Data Viewer
based on the type of datafor example, metrics appear as graphs, and alerts
appear as colored indicators. Data Viewers can display data from a metric, a
resource, or an element, such as an alert.
In Data Viewers, you can:
Display minimum/maximum metric values in a graphsee page 37
Use tool tips to view metric names and valuessee page 38
Show or hide metric data in a graphsee page 40
Change the scale of graphssee page 41
Move metrics to the front or back in graphssee page 43
Export datasee page 44

Displaying minimum/maximum metric values in a graph

To show the minimum and maximum values of metrics and metric groupings in
a graph:
1 Click the graph in the Console to select it.
2 Show the minimum and maximum values in one of two ways:
Right-click the Data Viewer and select Show Minimum and Maximum.
Select Properties menu, and select Show Minimum and Maximum.

Manipulating the contents of Data Viewers 37

CA Wily Introscope

Minimum and maximum metric

values are shown here.

... but are not shown here.

Note This change remains in effect only while you view the current dashboard.
If you open a new Console or switch to a different dashboard, this setting
reverts to the default, which does not show minimum and maximum
metric values. To show minimum and maximum metric values by default
in a Graph, turn on this option while editing a dashboard with the
Dashboard Editor.

Using tool tips to view metric names and values in a Data Viewer
In a Data Viewer, you can hover your cursor over a point on a graph to open a
tool tip.
To open a tool tip:
Mouse over any element in the Workstation metrics tree or in a Data Viewer, such

as a point on a graph.

38 Using the Workstation Console

Workstation User Guide

The illustration below displays information about a particular data point in the
graph, showing:
Metric name
Exact value of the metric
Min/max values for the metric across the period represented by the data point.

(See How time range affects data points, below.)

The count of 15-second intervals represented by the data point. (See How time

range affects data points, below.)

The date and time for the data point in the graph.

Tool tips now show exact

values.

Pressing F2 while a tool tip is active

allows you to click on the hyperlinked
text. When you do this, an Investigator
window opens with the tree expanded
to the metric shown in the tool tip.

In Introscope 8.0, instead of rounding to a value using K for thousand or M for

million, tool tips now show exact values, as shown in the illustration above.
Note For information on tool tips used in the Transaction Trace window, see
Tool tips on page 117.

How time range affects data points

Each data point on a graph represents an equal division of the time covered by
the graph. If the time range is set to Live (as in the illustration above), each data
point represents a 15-second interval.
If the time range is set to another value, the interval represented by each data
point will be different. If the time range is set to two hours, for instance:
Each data point represents a two-minute interval, or eight 15-second intervals.

Manipulating the contents of Data Viewers 39

CA Wily Introscope

Since there are eight 15-second intervals in two minutes, the count of each

data point is 8.

Showing/hiding metric data in a graph

If you are viewing the data from multiple metrics in one graph, you can show or
hide individual metric data.
To show or hide a metric in a graph:
1 Display a graph in the dashboard in the Console.
2 You can:
Show the metric by clicking its check box.
Hide the metric by unchecking its check box.

All metrics appear in this graph

because all metric checkboxes are
selected.

Only two metrics are

selected to appear in
this graph.

Note Show/hide metric options are not available when you view graphs or bar
charts that are displaying sorted or filtered data.

40 Using the Workstation Console

Workstation User Guide

Changing the scale of graph charts

You can change the scale of graph charts while viewing live data in Introscope
Workstation, to provide a more readable view. You change the scale of a chart by
setting a minimum and maximum value for the charts data axis.
The chart scaling feature is available only for graph charts in Live mode. It is not
available in Historical mode or for any other viewer type such as bar chart, top
ten, or string viewer.
Note Scale changes that you make to a chart are temporarythe settings are
not saved with the dashboard. When you select a new dashboard or close
the Console window, Introscope discards the settings and returns to the
scale options that were applied when the dashboard was created.
To view the scale of a graph chart:
Click on a chart to select it, and then:
Select Viewer > Scale Options, or
Right-click the chart and select Scale Options from the context menu.

The Data Options dialog box opens.

Setting the
Auto Scale
Minimum and
Maximum
default values
provides a
more readable
view of charts
in Live mode.

To rescale using min and max values:

1 Click on a chart to select it, and then:
Select Viewer > Scale Options, or
Right-click the chart and select Scale Options from the context menu.

2 Enter the minimum and maximum values for the data axis of the graph.
3 Click OK.

Manipulating the contents of Data Viewers 41

CA Wily Introscope

For example, if the chart data values lie primarily between 350 and 550 but the
chart value axis shows 0-1000, it might be helpful to set the scale Min value to
300 and Max value to 600 for a better view of the relevant data:

To force minimum and maximum values:

1 Click on a chart to select it.
2 Select Viewer > Scale Options.
3 Select Pin at on both the Minimum and Maximum sides of the dialog, and enter
a value for the minimum and maximum points of the data access.
4 Click OK.
Setting Min and Max values for a chart showing live data is risky, however, if
there is a chance the data may exceed the values you set.
To avoid this problem, use the Auto Scale option to automatically set the graph
to change its scale according to the data it displays.
To rescale using Auto Scale:
1 Click on a chart to select it.
2 Select Viewer > Scale Options.
3 Select AutoScale on both the Minimum and Maximum sides of the dialog.
4 Click OK.

42 Using the Workstation Console

Workstation User Guide

The resulting charts data axis is reset based on the data in the chart, as shown
in the illustration below. This often results in sharper valleys and peaks in the
graph display:

You can also set the scaling options to Auto Expand. This option uses 0 as the
bottom of the data axis and automatically expands and scales the data axis to
display all data for the time range.
To rescale using Auto Expand:
1 Click on a chart to select it.
2 Select Viewer > Scale Options.
3 Choose Auto Expand on both the Minimum and Maximum side of the dialog.
4 Click OK.

Moving metrics to front/back in graph

When a graph contains multiple metrics, it is possible for data points to overlay
each other. You can use the Bring to Front or Send to Back options to choose
which metric appears at the top of the list of metrics.
Note The Bring to Front/Send to Back options are not available when viewing
graphs displaying sorted or filtered data.

Manipulating the contents of Data Viewers 43

CA Wily Introscope

To change the overlap order of metrics in a graph:

1 Open the Console and display a graph in a dashboard.
2 Right-click the label of the metric to change, and choose an option from the
menu:
Bring to Front (moves selected metric to the top of the metrics listed)
Send to Back (moves selected metric to the bottom of the metrics listed)

The metric moves to the chosen position.

Copying a Data Viewer to the clipboard

You can copy a snapshot of the data in a Data Viewer to the clipboard as a bitmapped image. You can then paste the image into an email or other document,
or any application that accepts bit-mapped images. This is a handy tool if, for
example, you want to show data in a Data Viewer to a colleague, or perhaps use
it in a presentation.
To copy a Data Viewer to the clipboard:
1 Open a Console and select a Data Viewer
2 Select Viewer > Copy to Clipboard as Image.
Note You cannot copy multiple Data Viewers.

Exporting data from Data Viewers

Introscope enables you to take a snapshot of current data in a Data Viewer and
export it to a comma-separated values (.csv) file. You can export data from all
Data Viewer types except the alert.
To export data from a Data Viewer:
1 In the Console, select a Data Viewer.
2 Select Viewer > Export Data.
3 In the Save dialog box, choose a location to save the .csv file and click Save.

Viewing data in the Console

You can view live data in the Console, or select a range of time to view historical
data.
Live data is data that reflects the most recent intervals in real time (assuming

the Enterprise Manager and the agent are running). Depending on the type of
data viewer and how the data viewer is configured, live data may represent
one or more of the most recent intervals, including the interval that just ended.

44 Using the Workstation Console

Workstation User Guide

The default view of data is Live:

Historical data is data for a certain time range, which you specify the beginning

and end of. Historical data is stored in SmartStor.

Viewing historical data

To view historical data, you select a time range. When you select a time range,
Introscope immediately shows the data for that range, sets the end time to the
current time, and bases the duration on your time range selection.
The time range controls can help you identify the time a problem occurred. For
example, you think the problem occurred within the last hour, so you set the time
range to an hour and look at the data from the current time backward. If you
dont see the problem within that hour range, you can use the controls to move
backward or forward to locate the time the problem occurred.
To view historical data:
1 Select the metric or dashboard for which you want to see historical data.
2 Select a time range for the historical view from the Time Range drop-down
menu.

You can select a time range from the list, or select

Custom Range to define a range (see Defining a
custom time range on page 47).

Introscope shows the data for that range, using the duration that you selected
from the Time Range drop-down menu and setting the end time to the current
time.
Note If your historical time range includes a year, a four-digit year is required.

Viewing data in the Console 45

CA Wily Introscope

In this example, the time range was selected at 4:06:45, with a duration of 8
minutesthe end time for the range is therefore set to 4:06:45, and the start
time is 3:59:30.
When you select
a time range...

...the time bar

shows the start
time, end time,
and duration...

...and data for

the range
appears.

Note When you use the time-range control to view historical data, the range
you select is applied to other metrics or dashboards in the same window,
and to any new windows that you open.
3 Now you can select a resolution to adjust the granularity of the view, by
increasing or decreasing the number of data points that appear.
Each pre-defined time range is associated with a default resolution. You normally
do not need to change this. Changing the resolution is useful when you need to
see a greater level of detail or granularity in the data than appears by default.

You can select from the list...

...or type a value into the Resolution
field. Enter numeric values, followed by
the durationseconds, minutes, hours,
or daysas shown here.

4 After selecting a time range you can adjust it, using the controls to scroll in
increments based on the time range you selected:

46 Using the Workstation Console

Workstation User Guide

Drag the slider on the time bar to change the

time range:
Click the arrows to move backward and forward in time:

The single arrows move backward or forward in small increments; the double
arrows move backward or forward in time increments that are about equal to
the time of the selected time range.
Click the Reset icon to reset the end time of the range to the current time:

Defining a custom time range

To define a custom time range to view historical data:
1 Select the metric or dashboard for which you want to see historical data.
2 Select Custom Range from the Time Range drop-down menu.
The Custom Range window opens, showing the current date (Today) highlighted
with an outline.

3 Select dates:
a Use the calendar controls to select the start and end dates and times.
b Use the menu controls at the top of the calendar to select the month and year,
choose the date on the calendar, and type in the time in the time field at the
bottom of the calendar.
c Click OK.
Introscope shows the data for the custom range.

Viewing data in the Console 47

CA Wily Introscope

Zooming into historical data in graphs

When you view historical data in a graph, you can zoom in on data by clicking the
mouse pointer on a graph position and dragging, to specify the time range:

Drag the pointer to

zoom in.

Introscope refreshes the data in the viewer based on the new query, and the time
range in the viewer shows the new range.
The global time range in the window and the Time Range control do not change
automatically when you zoom in on data. For example, if you zoom in on a tenminute period on a graph with the Time Range set to 1 hour, the graph shows the
ten-minute period but the control remains at 1 hour, and the time bar still shows
the hour range.
You can override the default zoom actions in these ways:
Set the global time range and the Time Range control to match the zoomed

view: select Viewer > Set Time Range From Zoomed Range, or click the Set
Time Range from Zoomed Range icon
.
Lock your selected resolution by clicking the Lock icon

This maintains your selected resolution as you select different time ranges by
zooming in on data.
Hold down the shift key while you zoom, to constrain zooming to the time axis.

Filtering by agent with the Console Lens

You use the Console Lens to filter metric data for the agents that are reporting
data. In a dashboard that shows data for more than one agent, you can use the
Console Lens to view data only for selected agents.
When you apply the Console Lens, that filtering remains in effect as you navigate
among dashboards and switch between a Live view and a Historical view. The lens
filter remains until you close the Console window, or log out of the Workstation,
or use the Clear Lens command.

48 Using the Workstation Console

Workstation User Guide

Applying the Console Lens

To apply the Console Lens:
1 Click the Lens button

or select Dashboard > Lens.

If the Console is in Live mode, the dialog box lists the currently connected agents.
If you are viewing a time range of historical data, the dialog box lists agents
connected for the selected historical range.

2 In the Select Agent dialog box, select a single agent, or select multiple agents
(click and drag, or CTRL/click) on which to filter.
Note You can begin typing an agent name, hostname, or process name in the
Search field. As you type, the agent list filters to match what you type.
3 Click Apply or press Enter.
The dashboard refreshes to show only data for the selected agent(s). The Lens
button shows a black arrow:
The arrow on the lens changes from light blue to black
when a lens is applied.

Unsupported widgets
Some dashboard widgets do not support the lensing feature:
Graphs powered by calculators
Graphs based on a Virtual Agent powered by a simple alert. This includes the

Top 10 Connected Agents graph on the Overview dashboard.

Filtering by agent with the Console Lens 49

CA Wily Introscope

Clearing the Console Lens

To clear the Console Lens:
1 Click Lens.
2 Clear the Lens by clicking the Clear button on the Apply Agent Lens dialog box.

Console Lens and tab views in dashboards

The effect a Console Lens has on an Investigator View in a dashboard depends
on the type of tree item with which the view is associated.
If the Investigator
item associated
with the view is:

and...

a domain

a single agent is
...the item association changes
selected in the lens... to a single agent selection. If the
view doesn't support agent
selection, an error message
appears.

an agent

a single agent is
...the item association changes
selected in the lens... to a single agent selection.

a metric

a single agent is
...the same metric on the
selected in the lens... selected agent becomes the
current selection. If that metric
does not exist an error message
appears.

a metric path

a single agent is
...the same metric path on the
selected in the lens... selected agent becomes the
current selection. If that path
doesn't exist, an error message
appears.

another item type

then

an error message appears.

If more than one agent is selected, an error message appears in the tab view.
If the lensed agent is a Virtual Agent, the view shows data for that agent, if it
supports that type of selection. You can determine what views are supported for
a given item type by selecting an item in the tree, and observing the view tabs
that are available.
A Virtual Agent is a group of physical agents that are configured to be a single
agent, enabling you to see an aggregated view of the metrics reported by several
agents.

50 Using the Workstation Console

Workstation User Guide

For information about Virtual Agents, see the Introscope Java Agent Guide and the
Introscope .NET Agent Guide.
For information about adding Investigator Views to a dashboard, see the
Introscope Configuration and Administration Guide.

Filtering by agent with the Console Lens 51

CA Wily Introscope

52 Using the Workstation Console

CHAPTER

Using the Workstation Investigator

This chapter describes how to use the Workstation Investigator to view

application data in the Introscope Investigator window. It includes these topics:
About the Workstation Investigator .

54

Navigating in the Workstation Investigator .

58

About the Investigator tab views .

59

Viewing Blame information in the Investigator tree .

85

Viewing data in the Investigator .

.
.

.
.

89

Working with alerts in the Investigator .

92

Exporting data from the Investigator

93

Using the Workstation Investigator 53

CA Wily Introscope

About the Workstation Investigator

The Investigator contains two main panes:
Investigator TreeThe left side of the Investigator is a tree structure that

organizes metric data reported by the agents that report to the Enterprise
Manager you are logged into. Metrics are organized hierarchically by host,
process, agent, and resource type. Selecting items in the tree causes
predefined views to be presented in the Viewer pane. Inactive metrics appear
grayed out in the tree.
ViewerThe Viewer pane on the right side of the Investigator presents details,

often graphical, for the resource or metric currently selected in the tree.
Depending on the item selected in the tree, tabs at the top of the Viewer pane
allow you to select one or more views, including the General, Overview, Errors,
Traces, and Search views. For some views, options might be available in the
bottom section of the Viewer pane that control the data displayed in the
Viewer.

54 Using the Workstation Investigator

Workstation User Guide

About the Investigator tree

This illustration shows the Investigator tree in a Java environment, as seen by a
user with read or write permission to the SuperDomain. In this example, the
SuperDomain contains no domains, and two agents.
Custom Metric Agents
SuperDomain
Custom Metric Host
Custom Metric
Process
Custom Metric
Agents
EM supportability
metrics
Host Machine
Process
Agent

Super Domain node

The SuperDomain node contains metrics for all agents that report to the
Enterprise Manager to which the Workstation is connected. Metrics are organized
in a Host|Process|Agent hierarchy.
The nodes immediately under the SuperDomain node are virtual and physical
hosts.
Custom Metric Host (Virtual)This node does not correspond to a physical host

machine. It is a virtual host that contains metrics that are not reported by a
specific, individual agent. For example, if you have configured calculators that
create custom metrics, or have configured aggregated agents, they typically
appear under the Custom Metric Host.
HostsOne node for each machine that hosts an agent. Each host node

contains a process node for the instance of the application being monitored,
which in turn contains an agent node. The agent node contains nodes that
correspond to application and system resources, which contain metrics.

About the Workstation Investigator 55

CA Wily Introscope

Note: The application resources that appear in the agent node differ based on
whether the agent type is Java or .NET.

The SuperDomain is that which includes all user-defined domains and agents. The
Enterprise Manager administrator can set up the EM to display child domains with
separate permissions.
This illustration shows two child domains, myDomain1 and myDomain2, listed
under the Domains node as well as under the SuperDomain node in the default
Custom Metric Process.

The metrics that appear in the Investigator tree are a function of the PBDs
(ProbeBuilder Directives) used to instrument the application, and the run-time
activity of the application itself.
A metric only appears in the tree when the agent starts reporting it. The metric
remains visible in the tree, even if the agent stops reporting it.
Note Metrics might have the same name and appear twice in the Investigator,
if the metrics have different metric types. As with all metrics, inactive
metrics in this situation are grayed out.

Supportability metrics
Supportability metrics give information about the state of the Enterprise Manager
and the machine it runs on. You can view them under the path
SuperDomain|Custom Metric Host|Custom Metric Agent|Enterprise Manager. The
Introscope Sizing Guide contains extensive information about the supportability
metrics.

56 Using the Workstation Investigator

Workstation User Guide

Domains node
If the agents that report to the Enterprise Manager are organized into domains,
the Investigator tree domain node contains sub-nodes for each domain. Each
domain node is structured in the same Host|Process|Agent hierarchy as the
SuperDomain, and might also contain a Custom Metric Agent for custom metrics.

Investigator tree and domain permissions

Contents of the Investigator tree are based on user domain permissions:
Users with SuperDomain permission (at least read permission) see all domains

for that Enterprise Manager in the Investigator tree.

Users with permissions for multiple domains see domain information for those

domains in the Investigator tree.

Users with permissions for only one domain do not see domain information in

the Investigator tree; they only see the folders for metrics and Management
Modules.

Viewer pane
The contents of the Viewer pane vary, depending on the type of the item selected
in the Investigator tree. For metrics, a view of the metric data appears. Each
metric type has a default type of view, referred to as a Data Viewer Type.

Changing the Data Viewer type

You can change the Data Viewer type of the information you are viewing in the
Investigator Preview pane.
To change Data Viewer type:
1 Select Properties > View As.
2 Select the viewer display type to change to.
Note Only the Data Viewer types that are compatible with the type of metric
data you are viewing are available in the menu for selection.

Manipulating the contents of Data Viewers in the Investigator

For information on how you can manipulate the contents of Data Viewers, see
Displaying minimum/maximum metric values in a graph on page 37
Using tool tips to view metric names and values in a Data Viewer on page 38
Showing/hiding metric data in a graph on page 40
Changing the scale of graph charts on page 41

About the Workstation Investigator 57

CA Wily Introscope

Moving metrics to front/back in graph on page 43

Top ten views

When you select certain resources in the Investigator, the General tab of the
Viewer pane shows the top ten matching metrics for the selected resource. Java
resources include servlets, JSP, EJBs, and JDBC; for .NET, resources include
ASP.NET, ADO.NET, and serviced components.
These metrics appear in a bar chart in the Investigator Preview pane.

You can also view the response times of the top-ten called components of a
selected Servlet, EJB, or JSP for Java, or ASP.NET, ADO.NET, and serviced
components for .NET.
If you see fewer than ten bars in the bar chart, it is because there are fewer than
ten monitored components under that resource. If the metrics dont contain data,
you might see the metric names in the Preview pane but no data bars.

Navigating in the Workstation Investigator

After you have viewed multiple items in the Investigator tree, you navigate
backwards and forwards among the views.
To open an Investigator:
Select Workstation > New Investigator.

To navigate forward and back:

Click the Forward or Back arrow buttons in the upper right corner of the

58 Using the Workstation Investigator

Workstation User Guide

Investigator to move forward or backward among previously viewed

Investigator tree items.
Select from the drop-down lists next to the Forward or Back buttons in the

upper right corner of the Investigator.

Tool tips identify metric paths and values in the Investigator tree and the Viewer
pane.
When you hover the cursor over a metric in the Investigator tree, or the metric
name in the legend area of a Data Viewer, a tool tip shows the fully qualified
metric name.
When you hover the cursor over a data point in a Data Viewer (a graph, graphic
equalizer, bar chart, or dial meter), the tool tip shows:
Fully qualified metric name
Value of the metric
Minimum and maximum values of the metric
the count of how many data points were reported in the selected time slice
timestamp of data value nearest the cursor

About the Investigator tab views

The views that appear in the right pane of the Investigator vary, depending on
the resource or metric selected in the Investigator tree. Depending on the type
of node selected, you see tabs for one or more of these views:
General tab
Overview tabs
Search tab
Traces tab
Errors tab

Note Investigator Views are available only in the Workstation, not WebView.

General tab
When you select a metric, the General tab shows a graphic view of the metric
either for live data, or for a selected historical period. See Viewing historical data
on page 89 for an explanation of how to select ranges of historical data to view.
For some nodes in the tree, the General tab shows the path to that node object
in the Investigator hierarchy. For example, when the Frontends node is selected,
the General tab shows this path:

About the Investigator tab views 59

CA Wily Introscope

*SuperDomain*|HostName|ProcessName|AgentName|Frontends

For some other nodes in the tree, the General tab shows the Top 10 view of the
selected node. For example, when the EJB node is selected, the General tab
shows the response times of the top ten called components of the selected EJB
node.

Overview tabs
The Investigator summarizes information in an Overview tab for:
the overall Applicationsee Application Overview on page 60
the health of the EMsee EM overview on page 68.
data from ASP .NET pagessee ASP. NET overview on page 69.
data from EJBssee EJB overview on page 70.
data from application front endssee Frontend overviews on page 71.
data from application backend systemssee Backend overview on page 72.
the garbage collection (GC) heapsee GC heap overview on page 73.
instance counts of Java classes instantiated on the JVMsee Instance Counts

on page 73.
data from JTA componentssee JTA overview on page 74.
data from the Leak Hunter add-onsee LeakHunter overview on page 75.
data from servletssee Servlet overview on page 76.
socket connectionssee Socket overview on page 77.
data from strutssee Struts overview on page 78.
data on running threadssee Threads overview on page 79.
data from XML componentssee XML overview on page 80.

Application Overview
The Application Overview is available when you select an agent in the
Investigator tree, and enables application monitoring and triage. It shows highlevel health indicators, and a log of related events and historical metric
information.
The Overview shows a row of lights for each application managed by the currently
selected agent. Introscope presents this data for each application it discovers
when a servlet executes, Introscope makes a call to getServletContextName()
of the ServletContext interface to determine the name of the application. After
the application starts, the Overview tab automatically updates to display a row of
lights for it.

60 Using the Workstation Investigator

Workstation User Guide

The illustration below shows the Overview tab for WebLogicAgent on MyServer22:

Double-clicking one of the alert indicators

under the User column links to the URL
metrics for the selected application.

This illustration shows one application -- called /pipeorgan -- managed by this

agent. For this application, you can view alerts showing the state of:
User

Indicates how satisfactory the end-users interactions with the

application are likely to be. Satisfaction is a function of response time,
waits, stalls, and errors.
Greennormal, satisfactory user interactions with the application.
Yellowan attempt to use the application is likely to yield
unsatisfactory results, for instance poor response time or errors.
Redindicates a serious availability issue and that an attempt to use
the application will probably fail.

VM

Indicates the health and availability of server resources, such as resource

pools and CPU.
Greennormal health of server resources.
Yellowresource limitations or outages
Redserious resource limitations or outages.

About the Investigator tab views 61

CA Wily Introscope

Backend
Indicates the worst health and availability across all backends accessed
Summary by the application. For example, if one of three backends has a serious
resource limitation or outage, the All Backends light is red. The purpose
of the All Backends light is to allow the user, with minimal scrolling, to
quickly assess whether any of the backends have problems that require
investigation.
Greennormal backend health and availability across all backends
accessed by the application.
Yellowat least one backend accessed by the application is
experiencing errors or stalls, or poorer than expected response times.
Redat least one backend accessed by the application is experiencing
serious resource limitations or outages.
Backends

Any lights to the right of the Backend Summary light correspond to the
individual backends. For information about how Introscope identifies
backends see Viewing metrics for Backends in the Investigator on
page 87.
Greennormal backend health and availability.
Yellowbackend errors or stalls, or poorer than expected response
times.
Redserious backend resource limitations or outages.

You can see a

live Application
Overview...
...or a historical
Overview, based
on a time range
you select.

62 Using the Workstation Investigator

Workstation User Guide

The lights refresh every 15 seconds. The rows are sorted first by colorrows with
red lights precede those with yellow, which precede rows with all greento
reduce scrolling needed to identify potential problems. Within a color category,
rows are alphabetized by application name.

Using alerts to drill down for more data

You can double-click an alert from the overview tab to display the underlying data
for that application tier. For example, if you double-click the User alert, the
Workstation will display the URLs node for that agent.

Alert metrics in the Investigator tree

Each alert color has a metric value:
Gray0, no data is available
Green1, OK
Yellow2, Caution
Red3, Danger

There are some special cases to be aware of.

During the first minute of baseline calculation, the baseliner always reports

that the metric is normal. The baseliner is learning during this time, but it will
not report problems, to reduce false positives.
Another special case is in the calculation of baselines for average response

time. If an application component is idle, and the average response time metric
has a count of zero, the baseliner ignores this value in its learning. It does not
learn that 0ms was normal for that time period. Instead, it assumes that the
calculated baseline was expected during that time.
The following table shows how metrics drive alert values in the Overview Tab.

User

Yellow

Red

Frontend errors are abnormal

Frontend response time is

Frontend errors are very abnormal

Server execute threads in use are

abnormal
Server execute threads in use

are abnormal (for WebLogic

Server only)
Stall count is abnormal

very abnormal (for WebLogic

Server only)
Stall count is very abnormal

About the Investigator tab views 63

CA Wily Introscope

Yellow
VM

Aggregate CPU utilization is

Backend
Summary

Backend response time is

Red

Aggregate CPU utilization is very

abnormal and greater than 30
abnormal and greater than 50
percent
percent
JDBC connection pool utilization JDBC connection pool utilization is
is abnormal
very abnormal

abnormal
Backend error count is
abnormal
Backend stalls are abnormal

Backend error count is very

abnormal
Backend stalls are very abnormal

You can view the alert metrics by selecting the User, VM, and
Backends|BackendName metrics, below the Heuristics node in the Investigator.
The underlying metrics that drive the alert metrics appear in the User, VM, and
Backends|BackendName folders in the tree.

Application Overview metrics for historical mode

Over a historical range, an alert color reflects the worst-case value of the
heuristic at any point in the historical range. For example, if at any time during
a historical range the User heuristic for an agent was yellow, but never red, the
Overview tab for that historical range is yellow.

Application Overview metrics for a Virtual Agent

For Virtual Agents, heuristics are evaluated on the basis of Virtual Agent metrics.
For this reason, the Overview tab for a Virtual Agent might indicate a different
value than for the physical agents in the Virtual Agent.

64 Using the Workstation Investigator

Workstation User Guide

For example, the Overview tab for a Virtual Agent could display a green User
alert, even though the Overview tab for one of the agents in that Virtual Agent
shows a yellow User alert.
Heuristic metrics are only generated if the metrics they analyze exist. So, for
example, if the Virtual Agent is configured not to include CPU, JMX, or WebSphere
PMI metrics, there is no VM folder and the VM alert remains gray.
For information about configuring Virtual Agents, see the Introscope Installation
and Upgrade Guide.

Whats Interesting events

The lower half of the Overview lists Whats Interesting events, which Introscope
generates automatically when the color of an alert changes to yellow or red. In
Live mode, the previous 20 minutes of events appear.

For each selected item, you can see:

Timestamptime at which alert changed to yellow or red
StateThe state of the alert, identified by color
Applicationthe application for which the alert displays status
Isolated tothe tier associated with the state change

About the Investigator tab views 65

CA Wily Introscope

Whats Interestinga description of what drove the state changefor

example:
The number of errors in /pipeorgans User tier is unusual. The current
value is 28, while the typical value is 4.

This information also appears in the Whats Interesting View tab, as shown.
Also notice the tool tip that appears when you mouse over one of the alerts in the
Whats Interesting table.

Heuristics and metric baselines

Introscope determines the color of an alert in the Overview tab evaluating current
metrics against a baseline for those metrics.
Baselines are calculated using a statistical algorithm that has been successfully
applied in domains such as sales forecasting and weather forecasting. For a given
metric, the baseliner algorithm determines the next expected value, as well as
the expected deviation from that value. If the actual deviation exceeds (2x), or
significantly exceeds (4x) of that expected deviation, the baseliner indicates a
moderate or severe violation, and an associated heuristic turns yellow or red.
Internally, the baseliner evaluates the slope of the time series, and determines
the expected value of the slope. Recent data is given more weight than older
data.
Note While Introscope polls for metric data every 15 seconds, the baseliner
logic runs only every 60 seconds. This means that during the 60-second
interval, Introscope will poll for heuristic data and report an unchanged
heuristic value which can only be updated at the end of the 60-second
interval.
The baseliner has a notion of periodic seasons, time intervals during which we
expect environmental conditions to repeat. During the first week that a baseliner
is active, current values are compared against measurements taken on previous
days, with weekdays and weekends distinguished from each other. Lets say that
the Enterprise Manager is started on Thursday at noon. During the first 24 hours
the baseliner compares current values against data from all 24 hours, with more
recent data more heavily weighted. Starting Friday at noon, current data is
compared against data measured during the same 30 minute period on previous
weekdays. So, on Tuesday at 3:15PM, current data is compared against data on
Thursday, Friday, and Monday between 3:00PM and 3:30PM.

66 Using the Workstation Investigator

Workstation User Guide

Weekend data is only compared against itself. On Saturday the baseliner learns
from scratch, and on Sunday current data is compared against data from
Saturday.
After the first week we switch from a daily season to a weekly season. So, in our
example, starting on Thursday at noon we begin comparing current values
against 30 minute periods from the same time in previous weeks. Over time, an
increasing amount of historical data improves the quality of the baseline data and
the analytics.
For information about the metrics that drive each alert, see Using Blame Tracers
on page 67.

Some benefits of seasonality

Scheduled downtime is not supported in baselines, but baseline seasonality
compensates for this in cases where the scheduled downtime occurs regularly.
For example, if scheduled downtime occurs from 2 a.m. to 3 a.m. on Sunday
morning, the baseliner learns to expect strange values during this time, but those
values are not expected at other times of the week.
Abnormal data might pollute the baseline temporarilythe baseliner could slowly
learn that abnormal data is typical. However, abnormal data would need to be
sustained for a long time, and in seasonal mode (after the first day) the baselines
are even more robust against this. The baseliner looks at expected values over
30 minute periods in previous seasons, so unless a problem persists for many
days or weeks the baseliner expects good, normal activity.

Using Blame Tracers

You can use Introscopes Blame Tracers to mark the frontends of your
applications. For more information, see the Introscope Installation and Upgrade
Guide.

About the Investigator tab views 67

CA Wily Introscope

EM overview
You can view a variety of metrics on the Enterprise Manager itself by selecting
the EM node under Custom Metric Agent:

68 Using the Workstation Investigator

Workstation User Guide

ASP. NET overview

In environments where Introscope is monitoring a .NET application, an ASP .NET
node on the Investigator tree allows you to monitor metrics for application
components.

About the Investigator tab views 69

CA Wily Introscope

EJB overview
The EJB (Enterprise Java Beans) overview shows statistics for Entity beans,
Session beans, and Message Driven beans:

70 Using the Workstation Investigator

Workstation User Guide

Frontend overviews
Overviews for Frontend nodes show graphed application metrics, and statistics
related to transactions in the application:

You can open a tool

tip by hovering your
cursor over a metric.

About the Investigator tab views 71

CA Wily Introscope

Backend overview
Overviews for Backend nodes show graph views of database metrics and a table
view of SQL below the node:

72 Using the Workstation Investigator

Workstation User Guide

GC heap overview
The garbage collection (GC) heap overview shows heap use:

Instance Counts
The Instance Counts overview tab shows the classes instantiated on the JVM.

About the Investigator tab views 73

CA Wily Introscope

JTA overview
The JTA overview tab data about JTA components:

74 Using the Workstation Investigator

Workstation User Guide

LeakHunter overview
The LeakHunter overview shows statistics graphically and in a table. Leak tabs
appear for nodes under LeakHunter, and show details of the leak and a graph of
the number of collections over time:

About the Investigator tab views 75

CA Wily Introscope

Servlet overview
The Servlet overview shows a table of servlets in the node. When you select a
servlet, the Investigator shows its statistics in a graph:

You can select one or

more servlets to view.

Select an individual servlet to see its Overview summary tab:

76 Using the Workstation Investigator

Workstation User Guide

Socket overview
The socket overview shows tables for client and server sockets, and socket
information for each port:

With the Socket node

in the Investigator tree
selected...
... you can see all the
ports with active sockets

Selecting a port in the Server table

at the top...
... displays its Client ports in the
Client table on the bottom

Selecting a port in the

Investigator tree displays
metric graphs about
events and load

About the Investigator tab views 77

CA Wily Introscope

Struts overview
The Struts Overview tab shows an overview of Struts components, with a display
of the average response time for all components.
Selecting one of the component nodes shows an overview of the metrics for that
node, as shown in the second screenshot.

78 Using the Workstation Investigator

Workstation User Guide

Threads overview
The Threads overview shows all active threads being processed through an
agent:

About the Investigator tab views 79

CA Wily Introscope

XML overview
The Overview tab for the XML node displays metrics for XML components.

Selecting the XML node on the tree

shows the running XML components
in the Overview tab.
Selecting an individual XML
components shows the metrics for
that component.

80 Using the Workstation Investigator

Workstation User Guide

Search tab
The Search tab is available when you select a node in the Investigator tree that
contains metrics. It enables you to quickly find metrics.

The node selected in the Investigator tree sets the scope of a search. You can
enter either a string or a regular expression in the Search field. If you enter a
regular expression, check the Use Regular Expression box. Click Go to run the
search.
Note Regular expressions cannot filter by agent, so it is not possible to search
for agent name.
The right pane lists the resources with metrics that match the search argument,
and the value for each. To display Min, Max, and Count columns, click the
corresponding box above the metric list.
If you click a metric in the list, a view appears in the bottom of the right pane.
If you click on a different node that contains metrics, the search argument used
in the previous search remains active, and is applied to the newly selected node.

About the Investigator tab views 81

CA Wily Introscope

Traces tab
The Traces tab, available when a resource or component is selected in the
Investigator tree, is similar to the Transaction Tracer (see Using the Introscope
Transaction Tracer on page 105). The Traces tab lists the recorded Transaction
Trace events for the selected resource or component.

Errors tab
The Errors tab, available when a resource or component is selected in the
Investigator tree, lists errors and error details for the selected item.
Note You must have ErrorDetector installed to see the Errors tab.

82 Using the Workstation Investigator

Workstation User Guide

The top half of the Errors tab lists the time, description, and type of each error.
The lower half of the tab shows detailed information for each component involved
in the error selected in the list above.

Metric Count tab

Many of the nodes in the Investigator tree now have a new Metric Count tab,
which displays a pie chart of the metric distribution for the node.

About the Investigator tab views 83

CA Wily Introscope

The illustration above shows the pie chart, with a table display of the same data
beneath it.
The pie chart displays a maximum of 50 slices. When there are more than 50
resources in the selected node:
The pie displays the resources reporting the 50 highest values.
In addition to the slices representing the 50 highest values, an additional slice

will be labelled All Other Metrics to show the proportion of metrics with data
outside the top 50 reported.
The status bar displays the message Displaying the top 50 resources.

Remaining resources grouped in "All Other Metrics".

Mousing over an area of the pie chart displays a tool tip with count and
percentage.
Long labels will be truncated, but when you select a slice of the chart, the fully
qualified name of the resource will appear in the table beneath the chart.
The Metric Count tab is available in both live and historical modes.

84 Using the Workstation Investigator

Workstation User Guide

Viewing Blame information in the Investigator tree

Blame Technology is the Wily Introscope term that describes the tracking of
component interactions and component resource usage in an instrumented
application.
The Investigator presents a simplified blame stack that enables you to triage a
problem to the tier levelthe application frontend or the backend. This feature is
referred to as boundary blame, and is enabled by default. When enabled, only the
frontend component and a backend system are represented as blamed
components in the Investigator tree, indicating whether a response time problem
is internal to the application server (slow servlet) or external (slow backend).
For information about how Introscope determines frontend and backends, and for
instructions for using blame-related tracers to explicitly mark frontends and
backends, see the discussion on Configuring Boundary Blame in the Introscope
Java Agent Guide and Introscope .NET Agent Guide.

Viewing metrics for Frontends in the Investigator

The Frontends node contains a node for each frontend, including those
automatically detected by Introscope, or marked explicitly as a frontend with
ProbeBuilding.
The Frontends node organizes frontends by type. Typically the Frontends node
includes at least an Apps node, under which specific applications are listed. If
your environment includes multiple types of frontends, the Frontends node
contains a subnode for each.

Viewing Blame information in the Investigator tree 85

CA Wily Introscope

These examples show the Frontends node in the Investigator for Java and .NET
agents:

Java

.NET

Frontend metrics
These metrics are listed for each frontend:
Average Response Time (ms)
Concurrent Invocations
Errors Per Interval
Responses Per Interval
Stall CountIf a called component or backend stalls after being invoked by a

front end, the stall is reflected in the Stall Count value for the front end, as
well as in the components or backends Stalled Count.
For information on configuring front-end metrics, see the Introscope Java Agent
Guide.

Heuristics
The Heuristics node shows the metric values related to the alerts displayed in the
Overview tab, when an agent is selected. For more information, see Using Blame
Tracers on page 67.

86 Using the Workstation Investigator

Workstation User Guide

URL metrics
The URLs node under a front end node shows these metrics for each URL group
that is configured for the front end:
Average Response Time (ms)
Concurrent Invocations
Errors Per Interval
Responses Per Interval
Stall Count

URLs that do not match a URL group definition are shown in the Default group. If
no URL groups are defined, all URLs belong to the Default group.

Called Backends
The Called Backends node contains metrics that reflect the activity and
performance of a backend for a particular URL group:
Average Response Time (ms)
Concurrent Invocations
Errors Per Interval
Responses Per Interval
Stall Count

Viewing metrics for Backends in the Investigator

The Backends node contains a node for each backend, including those
automatically detected by Introscope, or marked explicitly as a backend during
ProbeBuilding.

Backend database metrics

For each database backend, these metrics reflect the activity and performance of
the backend across all applications it serves:
Average Response Time (ms)
Concurrent Invocations
Errors Per Interval
Connection Count
Responses Per Interval
Stall Count

Viewing Blame information in the Investigator tree 87

CA Wily Introscope

Backend database format

This section defines the Introscope naming convention for database backends.
For this database The backend name
Oracle

Is a concatenation of the Oracle SID string, the database host

and port delimited by a hyphen, and the string (Oracle DB).
For example:

PRODORCL3 sfoprod6.globex.com-1521 (Oracle DB)

DB/2

Is a concatenation of the DBName string and the string (DB/2

DB). For example:

Inventory4 (DB/2 DB)
Microsoft SQL
Server

Can be a concatenation of the database name, instance name,

the database host and port delimited by a hyphen, and the
string (MS SQL Server DB), depending on the configuration
of the database driver.
If the driver has a database name and an instance name, the
backend name in Investigator would look like this:

PRODORCL3 (instance Mx22) on prod6.globex.com1521 (MS SQL Server DB)

If the driver has no database name, the backend name in
Investigator would look like this:

SQLServer on prod6.globex.com-1521 (MS SQL Server

DB)
If the driver has a database name and no instance
name, the backend name in Investigator would look like this:
PRODORCL3 on prod6.globex.com-1521 (MS SQL
Server DB)
If the driver has an instance name and no database name, the
backend name in Investigator would look like this:

(instance Mx22) on prod6.globex.com-1521 (MS SQL

Server DB)

Defaults and fallbacks

In cases where the database driver does not support querying for the database
name, the name of the database defaults to the JDBC URL, with colon characters
(:) replaced by percent characters (%). In some cases even this fallback value is
not available, so the database name defaults to the classname of the database
driver. Exact behavior depends on the vendor and version of the database driver.

88 Using the Workstation Investigator

Workstation User Guide

Viewing data in the Investigator

You can view live data in the Investigator, or select a range of time to view
historical data. The default view of data is Live:

Viewing live data

Viewing historical data

To view historical data, you select a time rangeusing a time range can help you
quickly identify the time a problem occurred. For example, if you think the
problem occurred within the last hour, you could set the time range to an hour
and look at the data from the current time backward. If you dont see the problem
within that hour range, you can use the controls to move backward or forward to
locate the time the problem occurred.
To view historical data:
1 Select the metric or dashboard for which you want to see historical data.
2 Select a time range for the historical view from the Time Range drop-down menu.

You can select a time range from the list, or select

Custom Range to define a range (see Defining a
custom time range on page 47).

Viewing data in the Investigator 89

CA Wily Introscope

Introscope shows the data for that range, using the duration that you selected
from the Time Range drop-down menu and setting the end time to the current
time.
In this example, the time range was selected at 2:07, with a duration of 20
minutesthe end time for the range is thus set to 2:07, and the start time is
1:47.
Time range

Time bar

Data for the range

Note When you use the time-range control to view historical data, the range
you select is applied to other metrics or dashboards in the same window,
and to any new windows that you open.
3 To select a Resolution to adjust the granularity of the view, increase or decrease
the number of data points that appear.
Each pre-defined time range is associated with a default resolution. You normally
will not need to change this. Changing the resolution is generally useful when you
need to see a greater level of detail or granularity in the data than is displayed
by default.

You can select from the list...

...or type a value into the Resolution
field. Enter numeric values, followed by
the durationseconds, minutes, hours,
or daysas shown here.

90 Using the Workstation Investigator

Workstation User Guide

4 After selecting a time range you can adjust it, using the controls to scroll in
increments based on the time range you selected:
Drag the slider on the time bar to change the time range.
Click the arrows

to move backward and forward in time.

The single arrows move backward or forward in small increments; the double
arrows move backward or forward in time increments that are about the time
of the selected time range.
Click the Reset icon

to reset the end time of the range to the current time.

Defining a custom time range

To define a custom time range to view historical data:
1 Select the metric or dashboard for which you want to see historical data.
2 Select Custom Range from the Time Range drop-down menu.
The Custom Range window opens, showing the current date (Today) highlighted
with an outline.

3 Use the calendar controls to select the start and end dates and times, and click
OK.
Introscope now shows the data for the custom range.

Using zoom on historical data in graphs

When you are viewing historical data in a graph, you can zoom in on data.
To zoom in on data in a chart:
Do one of these:
Click the mouse pointer on a graph position and drag to specify the time range.
Right-click on the graph and choose Zoom to fit data.

Introscope refreshes the data in the viewer based on the new query, and the time
range in the viewer shows the new range.

Viewing data in the Investigator 91

CA Wily Introscope

To zoom back out:

1 Right-click the zoomed-in chart.
2 Choose Zoom Out or Zoom All the Way Out.
The global time range in the window and the Time Range control do not change
automatically when you zoom in on data. For example, if you zoom in on a tenminute period on a graph with the Time Range set to 1 hour, the graph shows the
ten-minute period but the control remains at 1 hour, and the time bar still shows
the hour range.
To set the global time range and the Time Range control to match the zoomed
view:
Click the Set Times Range From Zoomed Range button:

Working with alerts in the Investigator

Alerts in the Investigator Preview pane can display a set of multiple indicators, or
a single alert indicator.
To change an alert view:
Display an alert in the Investigator Preview pane and select Properties > Alert

View.
Alert messages are triggered by an action associated with an alert status. These
alerts appear automatically. You can also view alert messages by selecting
Workstation > Show Alert Messages.

92 Using the Workstation Investigator

Workstation User Guide

If Introscope is configured to generate alert state metricsas described in the

Introscope Configuration and Administration Guide discussion about generating
alert state metricsthey appear under the Alerts node in the Custom Metric
Agent, as shown here:

Exporting data from the Investigator

The Investigator supports the same export options as the Console. For more
information, see Copying a Data Viewer to the clipboard on page 44.

Exporting data from the Investigator 93

CA Wily Introscope

94 Using the Workstation Investigator

CHAPTER

Introscope Sample Dashboards

Introscope dashboards combine and present application metrics in views that

Operations personnel can use to monitor the overall application environment.
Dashboards also deliver the in-depth performance information required by
Application Support personnel for rapid problem diagnosis and resolution of
production applications.
This chapter describes the sample dashboards delivered with Introscope and
provides a scenario for application monitoring, problem notification, and rapid
diagnosis. It describes how to interpret performance information shown in the
sample dashboards, and navigate among the dashboards.
This chapter includes these topics:
About the Introscope sample dashboards
The Overview dashboard

96

99

The Problem Analysis dashboard .

101

102

How alerts are defined using heuristic metrics

Introscope Sample Dashboards 95

CA Wily Introscope

About the Introscope sample dashboards

Introscope is shipped with a pre-built Management Module in the
SampleManagementModule.jar file, which the Introscope installer places in the
<Introscope home>/config/modules directory in a new installation, or in the
<Introscope home>/examples directory if the installation is an Introscope
upgrade.
The Introscope dashboards provide:
Efficient monitoringHigh-level application health and status views of large

numbers of applications
Rapid notificationAt-a-glance notification of problems in the production

application environment
Actionable informationEnables quick identification of what is wrong, what to

do, who to call

Minimal trainingPre-defined navigation between high-level and drill-down

performance information, reducing the learning curve.

Quick resolutionOperations and Application Support personnel collaborate

more effectively to identify and resolve problems

The sample dashboards described in this chapter are installed when a new
Introscope Enterprise Manager installation is installed. If you have upgraded from
a previous version of Introscope, the old sample dashboards are preserved and
the new dashboards are available in the Enterprise Managers examples
directory, in the Management Module file named
SampleManagementModule.jar.
You can Hot Deploy this management module to see the new dashboards in your
environment. For more information about the Hot Deploy feature, see the
Introscope Configuration and Administration Guide.
Note Users with SAP installations do not see Introscope sample dashboards.

96 Introscope Sample Dashboards

Workstation User Guide

Dashboard alerts that show overall status

Each sample dashboard includes alerts that show the overall state of the
environment, and how key performance indicators are affecting the environment:

These alerts appear on the Overview, Enterprise

Manager Capacity, and Problem Analysis sample
dashboards.

These are the overall status alerts.

This alert

Shows

Overall

How is the overall experience to the applications user?

Response Time How is the response time for the application?

Errors

Are application users experiencing application errors?

Stalls

Is the application experiencing stalls?

CPU

Is the CPU consumption for the application normal?

Thread Pools

Does the application have enough threads available in its thread

pool?

JDBC Pools

Does the application have enough JDBC connections in its

connection pool?

About the Introscope sample dashboards 97

CA Wily Introscope

When you open the Sample Management Module you see the Intro to
Introscope dashboard:

The Sample Management Module

contains these sample dashboards:
An Intro to Introscope
Overview
Problem Analysis
The EM Capacity dashboard is part of
the Supportability Management Module.

Double-clicking this
alert opens the
Overview dashboard.

The sample dashboards provide an example of how to organize Introscope

metrics into a meaningful set of views for Introscope users. The Overview
dashboard offers an at-a-glance view of the health of the entire environment that
is monitored by Introscope. The Problem Analysis and Thread Details provide
details to help you narrow the root cause of a performance problem.

98 Introscope Sample Dashboards

Workstation User Guide

The Overview dashboard

The Overview dashboard is designed for the Application Support team to monitor
the key performance indicators of their applications across the entire monitored
environment.

Graphs show average

response time of
monitored applications,
their throughput, the CPU
utilization, and the
connection state of the
agents.

These alerts appear on each sample dashboard, to show

the overall state of the environment.

Alerts showing overall status

Each sample dashboard includes alerts that show the overall state of the
environment, and how key performance indicators are affecting the environment:
This alert

Shows

Overall

How is the overall experience to the applications user?

Response Time

How is the response time for the application?

Errors

Are application users experiencing application errors?

Stalls

Is the application experiencing stalls?

CPU

Is the CPU consumption for the application normal?

Thread Pools

Does the application have enough threads available in its thread

pool?

JDBC Pools

Does the application have enough JDBC connections in its

connection pool?

The Overview dashboard 99

CA Wily Introscope

Overview dashboard graphs

The Overview dashboard includes these graphs:
This graph

Shows

Application
Average
Response Time
and Responses
per Interval

The aggregate Average Response Time of the monitored

applications, and their throughput (Responses per Interval).

Backend Average
Response Time
and Responses
per Interval

Average response time and throughput of connected backend

systems. Backend systems can be anything that the monitored
applications connect todatabases, LDAP servers, and mail
servers, for example.

An interval is 15 seconds. If Introscope reports 45 responses per

interval for an application, therefore, it translates to a
throughput of 3 hits per second.

Introscope automatically identifies connected systems and

monitors their performance. In many cases, poor response time
can be directly traced to one of its backend systems.
Key Application
Server CPU
Utilization

CPU utilization of the .NET and Java processes that Introscope is

monitoring.

Connected
Agents

Connection state of the Introscope agents. Introscope reports

the state of connected agents as metrics whose value is either 1
or 3:
1 for an agent indicates that agent is connected to the
Enterprise Manager.
3 indicates that an agent has disconnected from the Enterprise
Manager.

This graph does not indicate the overall CPU consumption on the
machineit is the CPU consumed by the .NET or Java process
itself. Introscope provides data about the CPU consumption of
the machine, and you can include them in your custom
dashboards.

The graph shows the top 10 connected agents. Because

disconnected agents have a larger value than connected agents,
disconnected agents are shown first.

100 Introscope Sample Dashboards

Workstation User Guide

The Problem Analysis dashboard

The Problem Analysis dashboard includes graphs that help you locate the cause
of a particular problem.

Applications Performance and

Load graphs show application
average response time and
responses per interval.

These alerts appear on each sample

dashboard, to show the overall state of
the environment.

Stalls graphs show stalls in all

application components, and
socket concurrency metrics to
help you find the cause of a
problem.

On the Problem Analysis dashboard, overview alerts show you the health of the
entire environment as you review the details of a particular problem.
The Problem Analysis dashboard includes these graphs:
This graph

Shows

Application
Average
Response Time

The aggregate response time of the monitored applications.

Responses per
Interval

The throughput of the monitored applications.

The Problem Analysis dashboard 101

CA Wily Introscope

This graph

Shows

Application Stalls

Shows stalls coming from all components of your application,

including backend systems.
Stalls are an important metric that can help you determine the
cause of many production application problems. Stalls occur
when a request has been made of a monitored application, but
the application has not responded within thirty seconds. Most
stalls in production environments occur because a backend
system has stopped responding to an applications requests.
Introscope often automatically identifies the backend systems to
which the application connects, and monitors those systems for
stalls. When Introscope is unable to find a backend system,
however, that system remains unmonitored.
When an unmonitored backend system stalls, secondary stalls
within the application might indicate that a stall is occurring, but
Introscope is unable to identify the cause. In this situation, the
Top Concurrent Socket Communication graph can help you
determine the cause of a problem.

Top Concurrent
Socket
Communications

Shows results of the Socket Concurrency metric.

The two types of socket concurrency metrics are readers and
writers. Reader metrics are the number of requests in the
application waiting for a backend system to respond with data
through a socket. Writer metrics are the number of requests in
the application waiting for a backend system to accept data
through a socket.
If a stall in an application is caused by a backend system that
Introscope does not identify, looking at a high level of concurrent
socket readers or writers can often identify the offending system.

How alerts are defined using heuristic metrics

Each alert on the sample dashboards is based on Introscopes automated
heuristic modeling of standard key performance indicators as described in
Application Overview on page 60. Every key performance indicator has a
matching heuristic metric.
The values for heuristic metrics are 1, 2 or 3:
A value of 1 indicates that the current state of the key performance indicator

appears normal.
For example, if the application's overall response time usually varies between
600ms and 1000ms and the current value is 835ms, the response-time
heuristic metric reports a 1.
A value of 2 this indicates that the current state of the heuristic's key

performance indicator is outside of normal.

For example, if the application's CPU is usually between 30% and 60% and the
current value is 75%, the heuristic value might be two.

102 Introscope Sample Dashboards

Workstation User Guide

A value of 3 indicates that the current state of the heuristic's key performance

indicates is outside of normal to a large degree.

For example, if an application normally has no stalls or occasionally has one
stall but suddenly, the application's database stops responding to requests.
The number of stalls might increase to a comparably high number such as ten.
In that situation, the stall heuristic for the application would report a value of
3.
By defining alerts in terms of the heuristic metrics rather than fixed thresholds,
the work of determining normal values for key performance indicators shifts from
the Introscope administrator to Introscope itself.

Eliminating alerting on transient spikes

A technique that is useful for defining alerts is the At least N of the last M periods
property, which defines the number of instances in which the status of Danger
triggers an alert. (See the Introscope Configuration and Administration Guide for
more information.)
In production environments, key performance indicators might spike for a short
period of time. For example, a CPU might spike over a 15-second period, then
return to normal in the next 15-second period.
It is undesirable for Introscope to alert on this type of spike. By telling Introscope
to alert only if a condition lasts for eight out of the last eight periods (each period
is 15 seconds, so two minutes out of the last two minutes), alerts are only
generated for conditions that are real problems, rather than random spikes.

How alerts are defined using heuristic metrics 103

CA Wily Introscope

104 Introscope Sample Dashboards

CHAPTER

Using the Introscope Transaction Tracer

Workstation users with appropriate permissions use Introscope Transaction

Tracer to trace the activity of transactions as they flow through a Java Virtual
Machine, or a Common Language Runtime (CLR) in a .NET environment, inside a
production application.
This section includes these topics:
About the Transaction Tracer .

106

Starting, stopping, and restarting a Transaction Trace .

108

Using the Transaction Trace Viewer .

112

Querying stored events .

122

Saving and exporting Transaction Trace information .

126

Using the Introscope Transaction Tracer 105

CA Wily Introscope

About the Transaction Tracer

Introscope Transaction Tracer reduces the time required to identify a problem
component in a transaction, enabling authorized users to trace the transaction
activity at the component level. Transaction Tracer can trace synchronous
transactions that cross boundaries in the homogeneous application server
environments that support this capability:
WebLogic Server 8.0 and later
WebSphere 6.x.

In other environments, transactions can be traced within the boundaries of a

single Virtual Machine (VM) or Common Language Runtime (CLR).
You view the results of a cross-process transaction trace query in the Trace View
tab of the Transaction Trace Viewer.
Introscope saves Transaction Trace session data in the Transaction Events
Database for a specified amount of time, and periodically aged out to reduce
overhead.
You can configure the Introscope agent to capture Transaction Trace data based
on the values of servlet or ASP.NET variables such as HTTP request headers,
request parameters, session attributes, session ID, username, URLs and URL
Query strings. In addition, Introscope agents automatically sample transactions;
see Automatic transaction trace sampling, below.
Note Metric Shutoff state does not affect Transaction Trace data. If a managed
agent is shut off, that agent does not report Transaction Trace data. If
the agent is shut off while a Transaction Trace session is in progress, the
agent does report the data collected before the shutoff request.

Automatic transaction trace sampling

By default, Introscope agents sample transaction behavior by tracing each
normalized unique URL in an application once per hour. You can view and analyze
sampled traces from a selected historical time range:
in the Introscope Workstation and Webview
in the Traces tab in the Investigator

You can also configure automatic trace sampling even if no URL groups are
configured by specifying the number of transactions to sample during a time
interval; the default value is one transaction every two minutes. For more
information, see the Introscope Java Agent Guide.

106 Using the Introscope Transaction Tracer

Workstation User Guide

Transaction trace sampling is enabled by default. You can disable the behavior,
change the sampling period, or de-randomize the timing of sampling as
appropriate. For more information, see the discussion of Controlling Automatic
Transaction Tracing Behavior in the Introscope Java Agent Guide and Introscope
.NET Agent Guide as appropriate.

Transaction Trace overhead

A Transaction Trace session affects overhead from the time it starts until all
transactions in process at the end of the session complete. You can specify the
execution threshold at the millisecond level, but doing so increases the load on
the system.
These Transaction Tracer features reduce the likelihood of trace sessions
imposing unacceptable overhead:
Transaction Trace Session TimeoutA Transaction Trace session times out

after a user-defined period so that the Admin user cannot accidentally leave
the Transaction Tracer on and negatively affect performance for a sustained
period. At the end of the timeout period, the agent stops tracing new
transactions and completes tracing for transactions in progress.
Anti-Flooding LogicTo prevent excessive overhead, agent anti-flooding logic

limits the number of transactions traced per 15 second interval to 200. After
this limit is exceeded, the agent logs that the anti-flood threshold was
exceeded, and does not report Transaction Trace data to the Enterprise
Manager until that 15-second period has expired. After the 15-second period
expires, the anti-flooding logic resumes reporting.
The Introscope Sizing and Performance Guide has more information about
controlling transaction trace overhead.

Transaction Tracer compatibility with agents from previous releases

Introscope version 8.1 with Transaction Tracer enabled is compatible with agents
from versions before 8.1, with these caveats:
When you use Transaction Tracer with agents from supported versions before

5.3.1, you can filter on threshold execution time only.

When you use Transaction Tracer with agents from version 5.3.2 and later, you

can filter on parameters and threshold execution time.

With 6.0 and later agents, Transaction Tracer can filter by errors, in addition

to parameters and threshold execution time.

About the Transaction Tracer 107

CA Wily Introscope

Starting, stopping, and restarting a Transaction

Trace
To run a Transaction Trace session, you specify the agents whose transactions
you want to trace, and how long to capture the data. You can specify filter options
to limit tracing to transactions that:
exceed a threshold execution time you define
match parameter values such as User ID, request headers information, etc.
have errors, if Introscope ErrorDetector is installed

When the Transaction Trace Session starts, Introscope captures transaction trace
data that is specified in the agent profile, for each transaction. The transactions
that match the filter criteria appear in the Transaction Trace Viewer window, and
are saved in the Transaction Events database.
Note You can start Transaction Trace using a CLW (Command-Line
Workstation) command. For information about the command and its
syntax, see the Introscope Configuration and Administration Guide.

Starting a Transaction Trace session

To start a Transaction Trace session:
1 Select Workstation > New Transaction Trace Session.
The New Transaction Trace Session window opens.

You can specify a

filter for the trace,
and define its
parameters.

2 In the Trace transactions section of the window, specify the threshold execution
time. Select milliseconds or seconds from the drop-down list.
Note Sub-second thresholds can have a negative impact on performance.

108 Using the Introscope Transaction Tracer

Workstation User Guide

3 To specify a transaction filter, click the checkbox to the left of the filter type list
in the Trace transactions section, and select a type from the list:
User IDenter an operator and a parameter value.
Session IDenter an operator and a parameter value.
URL, or URL Queryenter an operator and a parameter value.
Request Headerenter a data type name, a condition, and a value.
Request Parameterenter a data type name, an operator, and a parameter

value.
Session Attributeenter a data type name, an operator, and a parameter

value.
Note Data is only available for use in filters if the Introscope agent is
configured to capture it. See the discussion about configuring
Transaction Trace options in the Introscope Java Agent Guide and
Introscope .NET Agent Guide, as appropriate for your environment.
These are the filter conditions:
Filter
Condition

Condition Effect

equals

Transactions in which the parameter value matches the string

specified are traced.

does not equal

Transactions in which the parameter value does not match the

specified string are traced.

Note: Transactions that do not include the parameter to which

the filter applies are also traced.
contains

Transactions in which the parameter value contains the specified

string are traced.

does not
contain

Transactions in which the parameter value does not contain the

specified string are traced.

Note: Transactions that do not include the parameter to which

the filter applies are also traced.
starts with

Transactions in which the parameter value starts with the specified

string are traced.

ends with

Transactions in which the parameter value ends with the specified

string are traced.

exists

Transactions that include the parameter to which the filter applies

are traced, regardless of the parameter value.

does not exist

Transactions that do not include the parameter to which the filter

applies are traced.

4 In the Trace Agents section, enter the length of the Transaction Trace session.

Starting, stopping, and restarting a Transaction Trace 109

CA Wily Introscope

5 In the Trace Agents section, select one or more agents for which to trace
transactions:
To trace all agents that support Transaction Tracing, click Trace all

supported Agents. This option traces supported agents that are currently
connected, and any that connect during the Trace session.
To trace selected agents, click Trace selected Agent(s) and select agents

from the list (CTRL + click to select multiple agents).

6 Click OK to start the Transaction Trace session.
Transaction Trace results appear in the Transaction Trace Viewer window. For
more information see Using the Transaction Trace Viewer on page 112.

Stopping a Transaction Trace session

To stop a Transaction Trace session:
Click Stop, or
Select Trace > Stop Tracing Session.

Restarting a Transaction Trace session

Restarting the Transaction Trace session resets the timeout to the user-defined
time period and continues to trace Transactions in the targeted agents using the
same threshold criteria.
You can restart a Transaction Trace session:
after a session has timed out.
to restart a session you have stopped.
to restart an in-progress session.

To restart a Transaction Trace session:

Click Restart, or
Select Trace > Restart Tracing Session.

Transaction Trace session options

Turning off low-threshold execution time warnings
If you are running the Transaction Tracer and set the threshold execution time to
less than one secondto perform a deep analysis, for exampleyou might see
continual warnings. The warnings indicate increased overhead because of
increased traces, so you might want to turn them off in a production
environment.

110 Using the Introscope Transaction Tracer

Workstation User Guide

To turn off the warnings about low-threshold execution time:

1 Select Workstation > User Preferences.
2 Click the Transaction Tracer tab.
3 Check the Dont warn when threshold is less than 1 second checkbox.
4 Click Apply.

Reviewing agents targeted for tracing

To review the agents targeted for tracing:
1 Select Trace > Show Traced Agents.
The Tracing Agent(s) dialog box appears.

To sort the Agent

information by column, click
on any column header.

2 When you are finished viewing the Tracing Agent(s) information, click OK.

Transaction Trace session options 111

CA Wily Introscope

Using the Transaction Trace Viewer

The Transaction Trace Viewer shows trace information for transactions that meet
the criteria you specified for the trace session.
The transaction
table shows
traced
transactions.
Select a
transaction,
then click the
tabs to see
different views.

With this transaction selected,

and the Trace
View tab active...

Summary View

... each
component is
shown in an
upside down
wedding cake
display.
The Component
Details pane
shows
information about
the selected
transaction
element.

Tree view tab

The table in the top pane of the Transaction Trace viewer lists transactions that
were traced during the session. You can sort the rows by column by clicking on
the column header. New transactions are inserted into the table in sorted order.

112 Using the Introscope Transaction Tracer

Workstation User Guide

The Transaction Table contains this information:

This

Indicates

Type

The type of information in the trace row, either transaction trace

(T), or error (E)
Error data only appears if ErrorDetector is running.
If an asterisk appears after the type symbol, it means that some

of the components in the transaction were truncated, or

clamped. See Clamped transactions on page 118.
Domain

Domain to which the traced agent is mapped

Host

Host on which the traced agent is running

Process

Agent Process name

Agent

Agent Name

Timestamp

Start time, in the agent machines system clock, of the invocation

of the root component

Duration

Wall clock execution time of the root component

Description

The URL that was invoked to initiate this transaction, or the

Introscope path to the component that initiated the transaction.

UserID

The ID of the logged-in user that is running the transaction (if it is

configured and available)

The Transaction Tracer window includes three tabsSummary, Trace, and Tree
Views. The first time you select a transaction in the transaction table, the
Summary View opens. When you select a transaction that has been opened
before, it opens in the most recently selected view.
This information appears for the currently selected transaction in each tab:
the fully qualified agent name
start time, in the agent machines system clock, of the invocation of the root

component
execution time of the root component in milliseconds

Using the Transaction Trace Viewer 113

CA Wily Introscope

Summary view
Summary View shows metrics for the components in the selected transaction.
Metrics include the path, number of calls, the length of the call in milliseconds,
and the minimum, average, and maximum call times.
You can double-click a metric to
open it in the Investigator.

The Transaction Trace status bar shows:

the number of transactions that were collected in the session
the filter criteria for the transaction trace session
the remaining time before the current session times out

114 Using the Introscope Transaction Tracer

Workstation User Guide

Trace view
Trace View shows the selected transaction in graphical format:
Use the Zoom slider to
magnify a transaction
and see its components.

Transaction
components

Component Details
for the selected
component

The Trace View shows:

each component in the transaction as a bar
the percentage of total transaction execution time for each component
the calling relationships between componentsthe bars for components are

displayed from top to bottom in calling order.

transaction sequence over timethe placement of components from left to

right indicates sequence. Relative wall clock time in milliseconds appears

across the top of the Transaction Snapshot.
errors within transactions (if ErrorDetector is installed): red slices in the

Transaction Snapshot represent errors within transactions.

In the Trace View you can:
Hover your mouse pointer over a component to open a tool tip, as shown

above. See Tool tips on page 117.

Using the Transaction Trace Viewer 115

CA Wily Introscope

Right-click a component to open the Investigator and view component metrics.

Select a component in the Trace View to open the Transaction Component

Details pane.

Transaction component details

The component details of the Trace View shows this information:
TypeHigh-level component (for example, EJB, Servlet, JSP in Java, and ASPX

in .NET)
NameName of the component
PathFull resource name of component
DurationExecution time (in milliseconds) of the selected component
Timestamp (relative)Start time, in the agent machines system clock, of the

invocation of the selected component

% of total transaction timePercentage of total transaction time taken by

selected component
PropertiesAny optional properties reported by the component (for example,

URL, URL Query, Dynamic SQL), or defined for collection in the Introscope
agent profile (User ID, Request Header, Request Parameter or Session
Attribute). You can select the text of any field in the Properties details and copy
it using the keyboard commands CTRL+C.
Property

Description

User ID (Servlet, JSP, ASPX)

User ID of the user invoking the HTTP servlet

request.

URL (Servlet, JSP, ASPX)

URL passed through to the servlet or JSP, not

including the query string (text after the ?
delimiter in the URL

URL Query (Servlet, JSP, ASPX)

Portion of the URL that specifies query

parameters in the HTTP request (text after
the ? delimiter in the URL)

Session ID (Servlet, JSP, ASPX)

The HTTP session ID associated with the

servlet request, if any.

Dynamic SQL (Dynamic JDBC or

ADO.NET Statements, when SQL
Agent is installed)

Generalized dynamic SQL statement, as it

would be seen in the aggregate form in the
SQL Agent

Callable SQL (Callable JDBC or

ADO.NET statements, when SQL
Agent is installed)

Callable SQL (with the ? still present)

116 Using the Introscope Transaction Tracer

Workstation User Guide

Property

Description

Prepared SQL (Prepared JDBC or

ADO.NET statements, when SQL
Agent is installed)

Prepared SQL (with the ? still present)

Method (Blamed Tracers; everything

but servlets, JSPs and JDBC
statements for Java, ASPX and
ADO.NET for .NET)

Name of the traced method

Tool tips
Hovering your cursor over any of the individual components, or layers, of the
graphical depiction of a transaction provides details about the component in a
tool tip. The illustration below shows a tool tip produced by mousing over the
EJB|Session component.

The tool tip displays:

Path
Duration
Timestamp (relative)
% of total transaction time

See Transaction component details on page 116 for definitions of this

information.

Using the Transaction Trace Viewer 117

CA Wily Introscope

Clamped transactions
To prevent unusual transaction trace results from consuming too many cycles, a
clamp on transaction trace components is set by default at 5000. (This setting,
introscope.agent.transactiontrace.componentCountClamp, is specified
in IntroscopeAgent.profile. For more information about working with the
properties in this file, see the Introscope Java Agent Guide or Introscope .NET
Agent Guide.)
For traces producing clamped componentsthose exceeding the CountClamp
traces will be marked with an asterisk, as in the first row of the screenshot below:

Things to notice:
The first row of traces is selected.
The Type symbol is marked with an asterisk, signifying that some of the

components in the transaction were truncated, or clamped.

A tool tip indicates how many components were truncated. In the example

above, 15 of the components of the selected trace exceeded the number

specified in the
introscope.agent.transactiontrace.componentCountClamp property.
The components which were not truncated appear in the Summary View tab

below.
To see a tool tip with more information about a trace:
1 Select one of the traces in the table.
2 Mouse over the selected trace.

118 Using the Introscope Transaction Tracer

Workstation User Guide

The tool tip displays trace type and number of truncated, or clamped,
components.
To sort the traces by type:
Click the heading of the Type column in the table.

Searching for clamped transactions

You can search for clamped transactions by issuing a historical event query.
Following the instructions for querying historical transactions in Querying
historical events on page 123, use a string like this in your query:
componentsNotShown:[1 TO 9999]

This will ensure that traces that had clamped transactions will be returned by the
query.
Note Since the historical event viewer search uses Lucene syntax, note:
the word TO in the string is case sensitive.
the search syntax is lexigraphical.

Using the Transaction Trace Viewer 119

CA Wily Introscope

About the Tree view

Tree view is a hierarchical view of the transactions components:

In the illustration above, details of the transaction selected in the list pane are
shown in the Tree View tab. The Tree View shows a hierarchical breakdown of the
components of the transaction. In the illustration, notice that three methods
contribute to the selected transaction. Of the three, the third method,
runRequestCycle, is decorated with a red light, and took 100% of the 1453ms
it took the transaction to run. With that method selected, the tab displays
additional information about the method in the Component Details pane.
Trace components that do not contribute a significant amount of time to the
transaction are color-coded with a green icon. To remove these components and
view only the key transaction components, choose Trace > Transaction Filter.

Viewing aggregated data for multiple transactions

In Transaction Tracer, you can select multiple transactions to see a
representation of all components in the traces.

120 Using the Introscope Transaction Tracer

Workstation User Guide

To view aggregated data:

1 Open a list of transactions by running a transaction trace and viewing them (see
Using the Transaction Trace Viewer on page 112), or querying for them (see
Querying stored events on page 122.
2 Select multiple transactions.
You can Ctrl/click to select
non-contiguous transactions,
or Shift/click to select
contiguous ones.
sn

3 Open the Summary or Tree view to see the transaction data aggregated.
Transaction Tracer shows the aggregated data in the tableyou might need to

scroll down to see all the data.

Transaction Tracer
shows the number of
aggregated
transactions and
lists all data for all.
The Tree View shows the aggregated data:

In the Tree view, Transaction Tracer adds

a node called Root if the selected
transactions dont share a common root
node.

Printing a Transaction Trace window

To print the Transaction Trace window:
1 Select Workstation > Print Window.
The Page Setup window opens. Defaults are letter size, portrait orientation.
2 Click OK to proceed, or change options then click OK.

Printing a Transaction Trace window 121

CA Wily Introscope

The Print window appears.

3 Select printing options, then click OK.
Note Printing a page range is not supported (everything prints on one page).
The contents of the entire Transaction Trace window prints, scaled to fit on one
page.

Querying stored events

Transaction Trace session results are automatically stored in the Transaction
Event Database. Transaction events include Transaction Traces and errors,
including stalls (if you have installed Introscope Error Detector.) The Transaction
Event Database contains transaction traces that were automatically sampled by
Introscope, as described in Automatic transaction trace sampling on page 106,
as well as the results of Transaction Traces sessions that you run yourself.
The Transaction Event database supports these types of queries:
historical events (basic)see Querying historical events on page 123
similar events (to selection)
correlated events (to selection)

Note Be sure that you run some Transaction Trace sessions before you use the
historical query, so that there is data to query.

Query syntax
The sections below describe how to use the Historical Query facility to query
stored errors. The query facility:
Is case-insensitivefor query strings or values for query options.
Supports the asterisk (*) wildcard characterEnter a fragment of a search

term followed by the asterisk. (You may not start a search term with the
asterisk character). For instance, to look for errors associated with a
component whose name includes the string Shopping, use the query string
Shopping*.
Supports Boolean operatorsSearch terms can use boolean logic, such as

AND, OR, NOT. and () groupings.

Supports exclusion conditionsUse +JDBC -CICS to look for transactions

with JDBC but not CICS.

Supports query optionsUse the options described in Query options and

syntax on page 124 to limit your query error events that occurred in a
particular timeframe, or are associated with particular users, or elements of
the hosting environment (as identified by domain, agent, host, or process).

122 Using the Introscope Transaction Tracer

Workstation User Guide

Querying historical events

To query historical transaction events:
1 Select Workstation > Query Historical Events.
The Historical Query Viewer opens.

2 In the Query field, enter a combination of:

the query option type: to include all transaction events that match the

specified type.
a query stringto search for errors that contain or match a string. If you dont

enter a query string, all errors events are returned.

query optionsto limit your search based on event parameters, as defined in

Query options and syntax on page 124.

3 Use the Time Range option to filter your query based on a time range, if
appropriatesee Viewing historical data on page 45 for an explanation of how to
use the Time Range option.
If you dont select a time range, the query uses the default of All and does not
apply a filter.
4 Click Go.
Transactions that match the query are displayed in the Historical Query window
the format is similar to the Transaction Trace Viewer. For more information see
Using the Transaction Trace Viewer on page 112.
Note Only 500 events can be viewed. If more than 500 events match the
query, the oldest 500 are shown.

Querying stored events 123

CA Wily Introscope

Query options and syntax

Queries use Lucene regular expression syntax to locate and substitute text
strings. For information about Lucene syntax, see http://lucene.apache.org/java/
docs/queryparsersyntax.html.
Field

Description

agent

Limits the search to events reported by a agent:ControlledRangeAge

particular agent.
nt

domain

Limits search to events related to

component(s) in a given domain.

domain:AcmeWest

fullAgent

Limits search to events reported by

specific agent(s), as specified by its full
path:

fullAgent:AcmeWest|
Custom Metric Host|
ControlledRange
Agent

domain|process|host|agent.

Example

host

Limits search to events that occurred on

a particular host.

host:Wmiddle01

process

Limits search to errors related to

component(s) in a given application.

process:Custom Metric
Host

root

Limits search to events associated with

specific component(s), as specified by
metric path.

root:servlets|accountSer
vlet

type

Specifies the type of event to include in

query results.
errorsnapshotLimits search to error
events.
normalReturns transaction events
captured in user-initiated transaction
traces.
sampledReturns transaction events
that were captured as a result of
Introscopes default transaction
sampling.
whatsinterestingReturns Whats
Interesting Events, which are
generated when Application Overview
heuristic values change. For more
information see Whats Interesting

type:errorsnapshot

events on page 65.

124 Using the Introscope Transaction Tracer

type:normal
type:sampled
type:whatsinteresting

Workstation User Guide

Field

Description

Example

url

Limits search to events associated with

the specified transaction URL path
prefix.

url:/bWar/burgerServlet

The path prefix is the portion of the URL

that follows the hostname. In this URL:
http://burger1.com/bWar/
burgerServlet?
ViewItem&category=11776&item=55562
630&rd=1
the path prefix is:
/bWar/burgerServlet
urlParams

Limits search to events associated with

the specified transaction URL
parameters.

urlParams:category=734*

URL parameters follow a question mark

(?) in the URL. In this URL:
http://ubuy.com/ws/shoppingServlet?
category=734&item=3772&tc=photo
the URL parameter portion is:
?category=734&item=3772&tc=photo

Note urlParams cannot start with a

wildcard character.
user

Limits search to events for transactions user:jdoe

associated with the specified Username.

Using special characters

If the following special characters are part of your query, Lucene syntax allows
you to escape them with a backslash (\) character:
+ - && || ! ( ) { } [ ] ^ " ~ * ? : \

For example, to search for (1+1):2, use the query:

\(1\+1\)\:2

Querying for similar events

In Introscope you can query for events that are similar to a selected event. For
example, similar events might be events that all contain the same components
(Servlet > EJB > SQL) with varying response times. Introscope considers events
similar if 60% of the strings within them (component names, SQL tables names,
and so forth) overlap.

Querying stored events 125

CA Wily Introscope

Note Even if a transaction type event is selected, both transactions and errors
might be returned in the results (errors are only be returned if
ErrorDetector is installed).
To query for similar events:
With a window of query results open, select a table row, then select Trace >

Similar Events.
Introscope lists similar events in the Historical Query window.

Querying for correlated events

In Introscope you can query for events that are correlatedthose that are part
of the same larger transaction. For example, a browser response time event is
correlated with a servlet transaction event.
Note Even if a transaction type event is selected, both transactions and errors
might be returned in the results.
To query for correlated events:
With a window of query results open, select a table row, then select Trace >

Correlated Events.
Introscope lists correlated events in the Historical Query window.

Saving and exporting Transaction Trace information

In Introscope:
You can save Transaction Trace data as an XML file that can be opened later in

a Transaction Trace window.

You can export Transaction Trace data as a text file for review in a text editing

program.

Saving Transaction Trace data

To save Transaction Trace data to an XML file:
1 In the Transaction Trace Viewer, select the Transaction Traces to save:
CTRL + click to select multiple Transaction Traces.
Edit > Select All to select all Transaction Traces in the window.

2 Click Save As...

3 You can open the file now, or select a location to save the file into, enter a
filename, and click Save.

126 Using the Introscope Transaction Tracer

Workstation User Guide

Opening Saved Transaction Tracer XML Data

You can open and view saved Transaction Trace data in a new Transaction Trace
window. These files can be shared through e-mail or stored on a shared network
drive to enable users to collaborate on problem analysis.
When opening saved Transaction Trace data:
you cannot restart the Transaction Trace session being viewed.
links from Transaction Trace components to their metric paths are unavailable

if the metric paths arent live in the Enterprise Manager to which the
Workstation is connected.
To open saved Transaction Trace data in an XML file:
1 Select Workstation > Query Historical Events
2 Select Trace > Open Saved Events (XML).
3 Select the XML file from the browser window, and click Open.
The data in the XML file appears in a new Historical Query window.
Note When you view saved historical events in an XML file, correlated events
will be displayed, but will not be shown as correlated. To see correlation
for historical events in a transaction trace, view an active trace (see
Querying for correlated events on page 126).
Now you can:
export a Transaction Trace as a text file
select Transaction Traces within the data and save them as a new XML file.

Exporting selected Transaction Trace to a text file

To export selected Transaction Traces to a text file:
1 In the Transaction Trace Viewer, select the Transaction Traces to export:
CTRL + click to select multiple Transaction Traces
Edit > Select All to select all Transaction Traces in the window.

2 Select Trace > Export.

3 Select a location to save the file, and name the file (default name is <root
component type>_<root component name>.txt.), and click OK.
Sample Transaction Trace XML File

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<TransactionTracerSession EndDate="2005-03-15T17:28:13.953-08:00"
Version="0.1" Duration="32" StartDate="2005-03-15T17:28:13.921-08:00"
User="Admin">

Saving and exporting Transaction Trace information 127

CA Wily Introscope

<TransactionTrace Duration="32" Domain="SuperDomain" EndDate="2005-0315T17:28:13.953-08:00" AgentName="WebLogic Agent" Host="rnadimpalli-dt3"

StartDate="2005-03-15T17:28:13.921-08:00" Process="WebLogic">
<CalledComponent MetricPath="Servlets|ActionServlet"
ComponentName="ActionServlet" Duration="32" ComponentType="Servlets"
RelativeTimestamp="0">
<CalledComponents>
<CalledComponent MetricPath="JSP|__register"
ComponentName="__register" Duration="16" ComponentType="JSP"
RelativeTimestamp="16">
<CalledComponents>
<CalledComponent MetricPath="JSP
TagLib|HtmlTag|doStartTag" ComponentName="doStartTag" Duration="0"
ComponentType="JSP TagLib" RelativeTimestamp="16">
<Parameters>
<Parameter Value="doStartTag" Name="Method"/>
</Parameters>
</CalledComponent>
<CalledComponent MetricPath="JSP
TagLib|BaseTag|doStartTag" ComponentName="doStartTag" Duration="0"
ComponentType="JSP TagLib" RelativeTimestamp="16">
<Parameters>
<Parameter Value="doStartTag" Name="Method"/>
</Parameters>
</CalledComponent>
<CalledComponent MetricPath="JSP
TagLib|MessageTag|doStartTag" ComponentName="doStartTag" Duration="0"
ComponentType="JSP TagLib" RelativeTimestamp="16">
<Parameters>
<Parameter Value="doStartTag" Name="Method"/>
</Parameters>
</CalledComponent>
<CalledComponent MetricPath="JSP
TagLib|MessageTag|doStartTag" ComponentName="doStartTag" Duration="0"
ComponentType="JSP TagLib" RelativeTimestamp="16">
<Parameters>
<Parameter Value="doStartTag" Name="Method"/>
</Parameters>
</CalledComponent>
</TransactionTrace>
</TransactionTracerSession>

128 Using the Introscope Transaction Tracer

CHAPTER

Introscope Reporting

Reporting provides critical information for a variety of functions within an

enterprise. For example, reports enable business managers to assess
applications impacts on the business; they enable capacity planners to determine
resource consumption; and they give Service Level Agreement administrators an
understanding of whether goals are being met.
Introscope includes report templates for creating reports quickly, and enables
you to create your own templates with custom graphs and tables.
This chapter describes Introscope reporting. It includes these topics:
Creating report templates .

130

Working with report templates .

152

Introscope Reporting 129

CA Wily Introscope

Creating report templates

A report template defines which metric data to track, the time range of the
reported metric data, and how to present the data in graphical and tabular form.
After you save a report template, any user can generate a report at any time.
To create a report template:
1 In the Management Module Editor, select Elements > New Report Template.
Note The New Report Template menu item is disabled if you do not have write
permission.
The New Report Template dialog box opens.

2 Specify the initial elements for the report.

a Type the Name for the new report template.
b Select Force Uniqueness to ensure that the report name is unique.
If you select this option and you then enter a name that is not unique,
Introscope adds a number to the name to make it unique.
Note The appended number appears after the report template is created,
when you view it in the Management Module Editor. If you dont select
Force Uniqueness and an identical report template name exists,
Introscope displays an error message and does not create the report.
c Select a Management Module from the drop-down list box to choose the
Management Module that will contain the report.
d Optional: Instead of selecting an existing Management Module to contain the
report, click Choose, then click New Management Module and assign a name
to the new Management Module. The illustration below shows these options.

e Click OK.

130 Introscope Reporting

Workstation User Guide

For more information about creating Management Modules, see the Introscope
Administration and Configuration Guide.
The new report template is added to the Management Module Editor, and the
settings pane opens.

Your report template appears in

the Management Module Editor...
...and the settings
pane opens, for you
to define the data
that the report
contains when it is
generated.

1 In the settings pane, select the Active check box if you are ready to activate the
report template.
When you generate an Active report template it appears in the list of report
templates in the Console, Investigator, and Management Module Editor. See
Generating reports from report templates on page 152.
Tip

Its a good idea to leave a new report inactivated after you create it, so
that you can test-generate the report without having it appear in the list.
After you test the report and it is ready for use, click Active to make it
available.

2 Click Open Template Editor to define report data.

Use the tool bar to add

elements to your report.

In the Report
Editor you specify
the purpose of
the report, when
and how long it
runs, and how the
results look.

Creating report templates 131

CA Wily Introscope

Next steps
Now you can:
Add report elements, such as charts, to the reportsee Adding report

elements to reports, below.

Define report propertiessee Defining properties in the Report Editor on

page 134.

Adding report elements to reports

You can add graphical elements such as charts and graphs, based on metrics or
metric groupings, to your report.
To add a graphical report element to a report:
1 If the report template editor is not already open, open it:
a With the Management Module Editor open, select the report in the pane on the
left.
b Click Open Template Editor.
2 Right-click the Report listed in the upper left pane, and choose Add.
A list of available elements appears.

3 Select one of the element types.

A new set of tabs appears.
In the following steps, you configure settings for the report element. To save your
work as you go, click Apply at the bottom of the edit window.
4 Configure text settings for your new report element using the Text tab.
a Specify the title to appear with the report element.
By default, Use Metric Grouping Name as Title is selected. If you choose
this, the element will take the name of the metric grouping whose data it
displays. (You associate the element with a metric grouping in step 5d on
page 133.)
You can also choose Enter Title and type a new title to appear with the report
element.

132 Introscope Reporting

Workstation User Guide

b Optional: Enter a description for the report element. This will appear in a tool
tip with the element.
5 Configure Data Properties for the report element using the Data Properties tab.
a Set time range.
The time range is defined by a Start Time and End Time. The report element
will display data bound by these times.
The Template Default Time Range is set in the default report properties (see
step 3 on page 136 to set the default time range). You can choose to accept
the default time range, or click Override Template Default Time Range.
To set the time range, click the calendar icon

by the Start Time field.

A calendar dialog appears, with the current date (Today) circled. Use the
calendar dialog to set the date, and edit the clock time in the text field after
the dialog is closed.

Repeat to set the End Time.

b Set the report duration using the Duration field.
Note If you have specified a Start Time and End Time, leave the Duration field
blank.
c Use the Unit dropdown to match the numbers you entered in the Duration
field.
d Select a metric grouping to associate with the report element. Click the dropdown next to the Metric Grouping label.

Creating report templates 133

CA Wily Introscope

A list of available metric groupings appears.

Select one of the available metric groupings.

e Optional: Filter the metrics associated with a metric grouping, or define a new
metric grouping.
To filter the metrics associated with a metric grouping, click Choose and enter
a regular expression.
To create a new metric grouping, click Choose, click New Metric Grouping,
and use the dialog to create a new metric grouping based on a management
module. For information about defining metric groupings, see the Introscope
Configuration and Administration Guide.
f Set values for element attributes in the table of element attributes:

6 Set the display properties for the report element in the Display Properties tab.
For information about display properties, see Defining properties in the Report
Editor, below.
7 When you have finished setting all properties for the report element, click Ok.

Defining properties in the Report Editor

Each element in the reportgraphs, tables, bar charts, and pie chartshas
properties that you can edit by selecting a properties tab. When you select the
Report Element (the top element in the list, which is labeled with the report title)
you see tabs that enable you to specify default properties:
Cover Pagethese properties apply to the selected element only: a title for the

report, a logo to include on the cover page if appropriate, and a description of

the report.
Default Data Propertiesspecify defaults for the whole report: time range of

134 Introscope Reporting

Workstation User Guide

the data (start and end time), the reporting period (for example, 15 seconds
or 1 minute), and a specification of the metric data to report.
Report Propertiesspecify formatting properties that apply to this report only

(whether to show the title page and table of contents), and properties that
apply to the whole report (time zone and language).
Default Display Propertiesdefine the default appearance of graphs and tables

for the whole report.

Note Changes to the properties in the Default Data and Default Display tabs
affect all elements in the report. Individual element customizations will
not be affected by the changes in default properties.
To define properties in the Report Editor:
1 Click the Cover Page tab to specify the purpose of the report.

2 Enter the information that will appear on the reports cover page:
To add

Do this

Report Title

Type a title for the generated report; the title appears on the
title page with the table of contents.

Logo

Click Choose to browse for your logo or other graphic file.

Any graphic chosen here appears in the upper left corner of
the title page. Supported formats are .jpg, .gif or .png.

Report Introduction

Type text that describes the contents of the generated

report. The introduction appears on the title page above the
table of contents.

Creating report templates 135

CA Wily Introscope

3 Click the Default Data Properties tab to specify the default time and data
parameters for all elements.

4 You can accept the default data properties, or set new ones:
For

Do this

Start Time and

End Time

When you specify a time range, you can specify a specific start
date and end date, or specify a time period such as 24 hours.
You can specify a time range for the report in one of these ways:
Type a specific start and end date and time, or click the
calendar icon to select start and end dates.
Leave the Start Time blank and use the Duration and Unit
parameters to specify how long the report runs.
Leave the End Time blank and use the Duration and Unit
parameters to specify how long the report runs.
Type Now for the End Time and use the Duration and Unit
parameters to specify how far back in the immediate history
to report on.

Note: When you type a specific start or end date and time, use
the format mm/dd/yy hh:mm (or dd/mm/yy hh:mm,
depending on the machines regional settings) and then
specify AM or PMfor example, you would type 12/15/
06 10:00 AM for English Regional.
Duration

Type a number to specify how long the report runs. This number
works in conjunction with the Unit valuefor example, you
might type 24 for the duration if the Unit is hours.

Note: See the explanation of Start Time and End Time for a
description of how the Duration and Unit parameters
work in conjunction with Start Time and End Time.
Unit

136 Introscope Reporting

Select a time unit from the drop-down list. Options are minutes,
hours, days, or weeks.

Workstation User Guide

For

Do this

Default Period

Click the field to activate the drop-down menu, then select a

default reporting interval for the report. You can choose to
aggregate all data over the interval, or choose a specific
reporting intervalfor example, 15 seconds, 15 minutes, a day,
or a week. If you choose a specific interval, the data is averaged
over the specified interval.
The default period value is Auto; this chooses the period
automatically, based on the selected Start and End Time range.

Default Agent
Override
Expression

Type the default expression to use if you want to override other

agent expressions:
If you are entering data properties for the report element, and
therefore for the overall report, all elements in the template
use this expression. The value you enter here overrides the
metric grouping or Management Module settings.
If you are entering data properties for an individual element,
the value you enter here overrides values entered for the toplevel element, as well as the metric grouping or Management
Module settings.
This field is optional. If you leave it blank, Introscope reports on
the agents based on the metric grouping setting. If the metric
grouping is set to inherit the agent expression from the
Management Module, Introscope reports on the agents based in
the Management Module.

Note: When you generate a report you can specify an agent

expression that overrides the template agent
expression. See Generating reports from report
templates on page 152.
Start Time of
Reference Data

Enter a date and time if you want to overlay a graph with metric
data from the same metric grouping, but from a different time
range.
When you use an overlay, Introscope identifies the metric data
that is plotted on the graph, and overlays it with data from the
same metric grouping, but from you specified time range. The
length of the period is the same as that of the base metric
grouping.
To specify a start time for the reference data, you can:
Type a date and time, using the format mm/dd/yy hh:mm (or
dd/mm/yy hh:mm, depending on the machines regional
settings) and then specify AM or PMfor example, you would
type 12/15/06 10:00 AM for English Regional.
Click the calendar icon to select a start date. When you use the
calendar to select a start date, Introscope sets the time to the
current timeto change the time, type over it.

Creating report templates 137

CA Wily Introscope

5 Click the Report Properties tab to specify settings for the reports formatting,
time zone, and language:

6 Enter the settings for the report:

To...

Do this

Show title page

Click On to generate a title page for the report.

Include table of
contents

Click On to create a table of contents on the title page.

Add report signature

Type a signature to appear at the bottom of the title page.

138 Introscope Reporting

Workstation User Guide

To...

Do this

Time zone

Click the row to open the list of time zones and choose a time
zone. The default is Use Time Zone of Client. The report uses
the selected time zone for the Report Date, and Start and End
dates.

Language

Click the row to open the list of languages. Choose a language

to format the reports date and time according to its standard.
For example, the Italian date/time standard is 9-mar-2008
15.50; the Japanese standard is 2008/03/09 15:50.
The language settings also determine the font used to display
the report in PDF files. To display Asian Language text
properly in PDF files, be sure to set the language
appropriately.

Note: In reports set to a non-English language, some

English words will still appear where they represent
labels, the internationalization of which is not
supported.
The default is Use Client Locale, which bases the date and
time formatting on the language used on the client machine.

Note: Producing reports in Asian languages requires that

some additional components were installed on your
Workstation during Introscope Installation. See the
Introscope Installation and Upgrade Guide discussion
Configuring the Workstation for Asian-Language
Reports for information.

7 Click the Default Display Properties tab.

You can accept the default properties, or set new ones to determine how the
graphs and tables in the report look after the report is generated.
This tab, like the other Default tabs, enable you to set default property values for
all elements in the report. For example, by setting Row Limit to 10, you ensure
that all tables in the report have a maximum of 10 rows. You can, however,
override this value for a particular table element in the report by selecting the
element, clicking the Display Properties tab, then entering a new Row Limit
property.
Note Use the scroll bar on the right of the Default Display Properties tab to see
all the properties.
8 Click the Display Properties tab to set the default display properties.
In reports, Average Min, Average Max, Mean, Absolute Min and Absolute Max are
defined as follows:
Average Min
The unweighted average of the minimum values of all periods.

Creating report templates 139

CA Wily Introscope

Average Max
The unweighted average of the maximum values of all periods.
Mean
A weighted average, calculated as follows:
(tv1 + tv2 + tvn...) / dp
where tv is the total of all values for a period, and dp is the total count of
data points for all periods. This gives greater weight to periods with more
data points.
Absolute Max
The actual largest or highest single value across all periods.
Absolute Min
The actual smallest or lowest single value across all periods.
The table below contains additional information on display properties and the
steps necessary to configure them.
Note In this step, it is possible to set display property attributes Sort Rows,
Sort By, and Value Format only for the report element types Metric
Data Table and Bar Chart. These attributes cannot be set for report
element types Metric Data Bar Chart and Metric Data Graph.
For

Do this

Aggregate Data by
Group

If on, combines data across metrics by summing or averaging

all metrics in a group (based on the Aggregate Using
property).
When metrics are grouped, only the groups summary values
appear in a report, instead of the individual metric-level
values. The aggregated summary rows are presented like
metric-level rows in a table or a plot in a chart, but their labels
show the group name instead of the individual metric name.
The group name becomes the label for the data item,
replacing the Item Label regular expression.
Use the Group Definition regular expression property to
determine the groupsee Setting custom group definitions on
page 145.

Aggregate Using

If Aggregate Data by Group is on, set this property to Sum or

Average, to specify how grouped metrics appear in a report.

Fill Time Markers

If on, the time between the Marker Start and Marker End time
is highlighted in the report.

Fill Y Axis Markers

If on, the area between the Y Axis Marker Start and End
values is highlighted in the report

140 Introscope Reporting

Workstation User Guide

For

Do this

Group Definition

When either Aggregate Data by Group or Subtotal by Group is

on, use this property to define the group. You can select a
group from the drop-down list, or create a custom regular
expression.
The group options from the menu are:
Fully qualified metric name
Agent location
Agent location - Metric Name
Agent name
Host
Metric Category
Metric Category: Metric Name
Metric Name
Servlet Name
Selecting one of these options inserts the appropriate regular
expression.
To create a group using a custom regular expression, see
Time series bar charts on page 148.

Item Label

Select a label for the item to appear in the legend:

Fully qualified metric name
Agent location
Agent location - Metric Name
Agent name
Host
Metric Category
Metric Category: Metric Name
Metric Name
Servlet Name

Selecting an option inserts the appropriate regular

expression.
You can use variables or regular expressions to create labels.
See Setting custom group definitions on page 145.
List Agents

This setting allows you to choose whether to display a list of

the agents whose metrics are being displayed.
On (default)the list of the agents will be displayed
Offthe list of the agents will not be displayed

Min/Max Bars

Plots the minimum and maximum values in each period for

any given metric. You specify how you want the minimum and
maximum bars to appear:
Show None (shows only the mean value)
Show Max Only
Show Min Only
Show Min and Max

Red Line Value

Specify the Y axis value where a red line is drawn to represent

an alert trigger value, with a Red Line Label if you specify one.

Creating report templates 141

CA Wily Introscope

For

Do this

Red Line Label

Type a label for the red line.

Row Limit

Specify a value to filter to show only values above or below

the limit, depending on whether Sort Rows is set to ascending
or descending.

Show Average Lines

If On, shows the averages of the metrics in the graph.

Show Fractions of a
Second

If On, shows the fractional parts of a second, up to six decimal

places to the right. For example:
03:22 .5123456 for 3 minutes, 22 seconds and 123456 ms.
00:00.25 for 250 ms.
3.13s for 3130 ms.

Show Legend

If On, a legend is included for the selected graph. The legend

shows which metrics correspond to each plot in the graph
according to the color of the plot and, if Show Shapes is on,
according to the shape used to mark each data point.

Show Shapes

If On, Introscope draws shapes at each point, in addition to

plotting the line between points. For graphs with many
metrics or with a high density of data, showing the shapes
might obscure the data, but if you omit the shapes the only
way to correlate plots with the legend entries is by using
color.
If a plot consists of only one data point in the given time
range, it does not appear in the graph unless shapes are On.
In particular if you set the period to Aggregate All it plots a
single value in the chart, but if shapes are off nothing
appears. You need at least two data points for a line to be
plotted.

Show Volume

If On, the number of metric data points within each period is

plotted as a bar in the report. If more than one metric appears
on the chart, the volume bars overlay each other.

Sort By

Select how to sort the columns:

Metric/Group Label
Mean
Average Min
Average Max
Absolute Min
Absolute Max
Count
Sum

Sort Rows

Select Ascending or Descending to sort the rows.

142 Introscope Reporting

Workstation User Guide

For

Do this

Subtotal Data by
Group

In tables, you can set the Subtotal Data by Group to sort the
items by group and then subtotal themwhen Aggregate
Data by Group is on, the Subtotal Data by Group attribute has
no effect.
Use Group Definition to define how metrics are divided into
groups, to provide a label for the group.

Note: Data in tables is always summarized across the entire

time range. The Value column is labeled Sum or
Mean, depending on the Aggregate Using setting.
Choosing Sum adds up every metric value.
Summary Row Label

Type text to appear as the label for the summary row.

Table Columns

Select a value to specify which columns appear in the report:

Show All Columns includes Mean (or Sum, depending on
how the Aggregate Using property is set), Average Min,
Average Max, Absolute Min, Absolute Max, and Count
Show Mean, Min, Max, Count
Show Mean, Count
Show Text Value of Metric Only results in a single column
labeled Value, which shows the metric unformatted. This is
most often used for String metrics that would otherwise
appear as zero.

Note: For a text string value to be reported, the time range

for data must be a Live Range of the last 8 minutes.
Value Format

Select a value to use for the table value display format, and
for the Y axis format (except for pie charts):
General
Use M(illions) and B(illions)
Memory Value Format (MB, GB, KB)
Percent (%)
Percent x 100 (%)
Show two decimal places
Millisecond as HH:MM:SS (shows milliseconds in hours,
minutes, and seconds) use for metrics whose values are
milliseconds
Microsecond as HH:MM:SS (shows microseconds in hours,
minutes, and seconds) use for metrics whose values are
microseconds
Millisecond as d, h, m, s (shows milliseconds in days, hours,
minutes, and secondsfor example, 3h 22m 36s)

X Axis Label

Type a label to appear along the X axis of the graph.

Creating report templates 143

CA Wily Introscope

For

Do this

X Axis Marker Start

Time, X Axis Start
Marker Label, X Axis
Marker End Time, and
X Axis End Marker
Label

You can use these attributes to bracket a period within a

report chart, and to label the start and end points for that
occurrence.
Start/end date/time formats are expressed, for example, as:
3/31/99 11:30 AM
You can also use the calendar widget which appears when you
put your cursor in the Start Time or End Time field.
Labels are text strings.
The specified period will appear bounded by vertical lines in
the report chart, with labels.

X Axis Marker Start

Offset in Seconds and
X Axis Marker End
Offset in Seconds

These settings provide an alternative to setting absolute date

values for the start and end markers. The values are an
offset, in seconds, from the start of the graph to where the
marker appears.
Offsets are useful when a reports date range is relative to the
reports start and end date and are not absolute time ranges
from Now - 1 hour to Now, for example.
For an X Axis marker to appear, you must set either the date
or the offset. If both are set, the date is used; if neither is set,
no marker appears.

X Axis Time Format

Click the row to choose from a list of time formats

Y Axis Format

Click the row to choose from a list of formatsfor example,

Memory Value Format (MB, GB, KB), or Percentage (%).

Y Axis Label

Type a label to appear along the Y axis of the graph.

Y Axis Marker Start

Value, Y Axis Marker
Start Label, Y Axis
Marker End Value,
and Y Axis Marker
End Label

Use the Marker Start and End Values to bracket values on the
Y Axis, and label those values. See the corresponding note on
X Axis Date/Time formats and labels on page 144.

Y Axis Upperbound
and Y Axis
Lowerbound

Type values in these fields to specify values on which to

report. You would use these properties if, for example, you
have a metric that might fall far outside the range of values
say, 50 seconds as opposed to 1 second. If you specified the
Upperbound property in this situation as 0.8 and the
Lowerbound property as 0.2, the report would only report
between those values.

Yellow Line Value

Specify the Y axis value where a yellow line is drawn to

represent an alert trigger value, with a Yellow Line Label if
you specify one.

Yellow Line Label

Type a label for the yellow linefor example, Response time

is slow.

144 Introscope Reporting

Workstation User Guide

Setting custom group definitions

You can use group definition to define grouping for these elements:
Bar Charts

Bar Charts are a simple way to show summary data. The values in a
bar chart are the same as you would see in a table, but you can
additionally use Group Definition to group the bars.
You use the Group Definition property to group bars in the chart and
define the label that appears underneath each group of bars. By
default it is the agent.
To disable grouping, enter a literal value for the group definition and
that will appear as a single label underneath the chart.
Use the Item Label property to define what appears in the legend.

Pie Charts

Pie Charts are useful for showing relative values of summary or

grouped data, defined by the Group Definition property to divide
metrics into groups.
Set the Aggregate Data By Group property to on.
Use the Item Label property to define what appears in the legend.

Aggregating
Data

When you use the Aggregate Data into Groups property, Introscope
combines the metrics in a group by summing or averaging,
depending on how the Aggregate Using property is set.
The aggregated data becomes a new data item and appears as a
single row in a table, or a plot in a chart. The group name becomes
the label for the data item, and the Item Label property no longer
applies.

Subtotalling

You use Group Definition to define how metrics are divided into
groups, to provide a label for the group, and to subtotal rows. The
Subtotal Data by Group property is similar to aggregation.
In tables, both properties combine rows, but in subtotalling the
individual metric rows appear; with Aggregate Data by Group turned
on, only the subtotal rows appear.
In tables, you can set the Subtotal Data by Group to sort the items
by group and then subtotal themwhen Aggregate Data by Group is
on, the Subtotal Data by Group attribute has no effect.

Note: Data in tables is always summarized across the entire time

range. The Value column is labeled Sum or Mean, depending
on the Aggregate Using setting. Choosing Sum adds up
every metric value for every data point in the entire time
range.

You can use variables and regular expressions to:

extract a common part of a metric string and thus define a group.
format data item labels.

Creating report templates 145

CA Wily Introscope

Using variables
Use these variables to extract parts of the fully qualified metric string.
Variable

Substitution

$host

host part of an agent

$proc

process part of an agent

$agentname

agentname part of an agent. Compare with $agent.

$agent

full agent spec: host, process, agent

$metric

The part of the metric identifier to the right of the colon (:).

$path

The part of the metric identifier to the left of the colon (:).

$path[n]

The indexed segment of the path (base 1). If out of range, return
empty string

$path[-m]

Path segment m counting from the end.

$path[m:n]

The part of the path from segment m up to and including segment n.

If either value is negative, then the segment is counted from the
end.

$domain

Domain; for example *SuperDomain*

$regex

Defines the beginning of a regular expression string. See Using

regular expressions on page 146.

For example:

Using the example above:

This string using variables and plain
text...

is displayed:

$host - $path[-1]

damien.ca.com - ActionServlet

$agentname servlet $path[-1]

WebSphere Servlet ActionServlet

Servlet $metric

Servlet Average Response Time

Using regular expressions

You can also use regular expressions to define grouping.

146 Introscope Reporting

Workstation User Guide

Regular expressions use these patterns:

Variable

Description

$regex['pattern']

The part of the full metric URL which

matches the given regular expression.
Note:
If the regex has a group, then only
extract that group.
Otherwise extract whatever is
matched.
If nothing matched, return the full
metric. This will be needed to
represent old settings.

$regex['pattern','replacement'] Replace the part of the full metric URL

which matches the given regular

expression pattern with the given
substitution pattern. Any capturing
groups in pattern can be inserted into
replacement using $ variables, where $1
is the first group, $2 is the second, etc.

For the full qualified metric

*SuperDomain*|foo.company.com|WebSphere|WebSphere|Servlets|ActionServle
t:Average Response Time

This string using variables and plain text...

$regex['(\w*).company.com'] servlets

will display as:

foo servlets

Using regular expressions to match a range of metrics

Consider an example where this regular expression is used as the item name:
\|Servlets\|.*:Average Response Time.*

Lets say that this matches five different servlets on each of two agents. If you
show these metrics on a chart with default settings you will see 5 * 2 = 10 plots
on the chart.
You can group the metrics by Servlet or by agent. The default is by agent,
because the default group definition is: (.*?\|.*?\|.*?)\|
If you set Aggregate Data by Group to on, you will see only two plotsone for
each application server that is the aggregation of all servlets on that application
server.

Creating report templates 147

CA Wily Introscope

If you change the group definition to be a regular expression matching the servlet
name, the metrics for a particular servlet on both application servers will be
aggregated into a single plot, giving you 5 plots, one for each servlet.
In this case the group definition might be: Servlets\|(.*): to match the exact
Servlet name part of the metric.
A complete guide to the supported regular expression syntax is located at Suns
Java API Pattern class pagehttp://java.sun.com/j2se/1.5.0/docs/api/java/util/
regex/Pattern.html.

Time series bar charts

In Introscope 8.0, you can configure report bar charts to show time series data.

bar chart example

148 Introscope Reporting

Workstation User Guide

To specify a custom time range for the report:

1 Select the report template.
a If you want to create a new report template, see Creating report templates on
page 130.
b If the report template already exists, open the Management Module Editor,
expand the tree structure, and select the report template to edit.
2 Ensure the Active check box is selected:

With the report template

selected in the tree
structure...

... ensure the Active check

box is checked.

3 Click Open Template Editor.

The Edit Report dialog opens. For more information about the tabs in this dialog
and what you can do with it, see Defining properties in the Report Editor on
page 134.
4 Click the Default Data Properties tab.
5 With the Report selected in the left pane:
a right-click the title of the report template in the left pane, and choose Add.

Creating report templates 149

CA Wily Introscope

b Choose Metric Data Bar Chart from the available elements.

The Metric Data Bar Chart element you added will appear in the list of report
elements under the Report.
6 Click the Data Properties tab to define properties for the chart.
7 Set the time range:
a Select Override default time range.
b Enter start and end date and time values.
c Ensure the Duration and Unit settings agree with Start and End Time values.
These do not automatically reset based on the Start and End Times.
Ensure the Duration and Unit
settings...
... agree with the Start
and End Time settings.

Note Setting the time range to a relatively small period will cause graphic
display elements in the chart to overlap and reduce readability.
8 Select a metric grouping to associate with the report element.
a Click the drop-down next to the Metric Grouping label.

150 Introscope Reporting

Workstation User Guide

A list of available metric groupings appears.

b Select one of the available metric groupings.

c Optional: Filter the metrics associated with a metric grouping, or define a new
metric grouping.
To filter the metrics associated with a metric grouping, click Choose and enter
a regular expression.
To create a new metric grouping, click Choose, click New Metric Grouping,
and use the dialog to create a new metric grouping based on a management
module. For information about defining metric groupings, see the Introscope
Configuration and Administration Guide.
9 Set the Period for the chart. This sets the interval on the Y axis. For example,
setting the Period to 5 minutes will produce a series of bars each representing 5
minutes, as in the example on page 148.
10 Click the Display Properties tab to set display properties.
11 Set the Item Label:
a Click in the pane to the right of Item Label.
The pane becomes a dropdown.
b Choose Fully Qualified Metric Name.

After selection, Fully Qualified Metric Name is displayed as:

$agent|$path:$metric.
12 Set Aggregate Data by Group to On.
13 Set Group Definition:
a Click in the pane to the right of Group Definition.
The pane becomes a dropdown.
b Choose Fully Qualified Metric Name. After selection, Fully Qualified Metric
Name will be displayed as $agent|$path:$metric.

Creating report templates 151

CA Wily Introscope

14 Apply your changes, then click Ok.

Applying and reverting changes

To apply changes to a report:
Click the Apply button.

The Apply button saves your changes to a report without closing the report,
allowing you to continue working.
To save changes and close the report:
Click the Ok button.

To revert changes to a report:

Click the Revert button.

The Revert button returns the report to:

the state it was in after you last clicked Apply, or
if you havent clicked Apply, to the state it was in when you opened it.

Working with report templates

Copying or deleting report templates
To copy or delete report templates:
1 Right-click the template.
2 Select Copy Report Template <name>, or Delete Report Template <name>.

Generating reports from report templates

To generate reports, the report template must be active and the Enterprise
Manager must be running. Introscope produces reports in these formats:
PDF
HTML
XML (no pages)
XML (Embedded Pages)
Multi-Sheet Excel (*.xls)
Single Sheet Excel (*.xls)
Word (*.rtf)
Comma Separated (*.csv)
Text (*.txt)

152 Introscope Reporting

Workstation User Guide

Jasper Report (*.jrprint)

Note Any user with read permission can generate a report from a report
template.
To generate a report from a report template:
1 Select an active report template in one of these ways:
In the Management Module Editor, right-click on a report template and select

Generate Report from Report Template <name> from the menu.

In the Management Module, Investigator, or Console, select Workstation >

Generate Report.
The Choose Report Template dialog box opens.

2 Select a report template from the list and click Choose to open the Generate
Report dialog box:

Specify the reports run parameters.

To override the template agent
expression, specify a different
agent expression here.
After you click Generate Preview,
the report preview appears here.

3 Specify the reports start and end dates.

Note Time ranges for the report are calculated according to the time zone of
the Workstation generating the report. The day starts and ends at
midnight.

Working with report templates 153

CA Wily Introscope

4 If you want to override the template agent expression, specify a different agent
expression or click Select to choose an expression.
5 Click Generate Preview.
The Preview pane shows the report results.
Use the Preview buttons to
save the report, print it,
navigate through it, and
size it.

6 Now you can use the Preview buttons to manipulate the report output:
Click Save

to open the Save dialog box. Specify a location and file name,
and choose a format in which to save the report.

Click Print

to open the Print dialog box and specify a printer.

Click the navigation arrows

to move forward and backward

through the report, or type a page number in the page number field.
Click the page views
Click zoom

to choose how the report appears.

to choose the view magnification.

Introscope sample report templates

Introscope includes sample report templates that are based on the sample
dashboards and Management Module that are included with Introscope. You can
customize these sample report templates and edit them to match corresponding
business needs.

154 Introscope Reporting

Workstation User Guide

Application Capacity Planning report

The Application Capacity
Planning report includes the
graphs listed in this table of
contents. The report shows
trends in J2EE Application
server resource utilization
over a period of time. The
default is one day, for a
three-month period.

Production Application Health report

The Production Application

Health report includes the graphs
listed in this table of contents.
The report shows overall
application health. It reports on
the performance of EJBs, JSPs,
servlets, SQL statements,
available JDBC connections, and
idle threads over the last 7 days.

Working with report templates 155

CA Wily Introscope

QA/Test Application Performance report

The QA/Test Application

Performance report includes the
graphs listed in this table of
contents. The report shows all
the characteristics of the
application from a performance
point of view in a QA or test
environment. These include a
component performance view
as well as resources view.

156 Introscope Reporting

APPENDIX

Introscope Metrics

Introscope, and its extensions and add-ons, displays application performance

data collected from remote and local systems as metrics. This appendix is a guide
to understanding these metrics.
This chapter includes these topics:
How Introscope measures application health .

158

The five basic Introscope metrics .

161

Other common metrics .

169

Introscope Metrics 157

CA Wily Introscope

How Introscope measures application health

Introscope measures application health by measuring the performance of
individual methods as they are executed by various application components.
Probes inserted into application component bytecode report data to agents, which
in turn report data to the Introscope Enterprise Manager (EM). Other subsystems,
like JMX and PMI, also report data which is collected by agents.
The EM complies this data into metricsapplication health as measured at many
points in the application subsystemsand displays the metrics in Introscope
Workstation or WebView. You can also export the metrics to an external
database.

Common terms
To understand Introscope metrics, you should understand how Introscope uses
some common terms.
A complete Introscope Glossary is available in the Introscope Overview Guide.
backend
An external system, such as a database, a mail server, a transaction
processing system (such as CICS or Tuxedo), or a messaging system (such as
WebSphere MQ).

158 Introscope Metrics

Workstation User Guide

concurrency and concurrent invocations

Concurrent methods are methods that started during an interval without
finishing during the same interval. Since you want methods to complete
quickly, an unusually high number of concurrent invocations is undesirable.
errors
Errors generated by the application or system being monitored.
frontend
The component of an application that first handles an incoming transaction. It
may be a Servlet, a JSP, an EJB or some other component.
harvest
The process in which Introscope gathers data from Collectors.
interval
A user-defined time slice used to define and average metrics. In Introscope
this is usually 7.5 seconds, though the way some of the monitored systems
capture data sometimes necessitates a different interval.
response
Response always refers to method execution. Repsonse is measured as:

count, referring to the number of transactions finished during that interval.

time, referring to the time it took to execute a method, in milliseconds.

Responses Per Interval is the standard Introscope throughput metric.

response time
The time it took to execute a method. May be measured as:

average response time (ms)The average time, in milliseconds, it took to

execute the method during the interval.

response time, min and maxThe lowest and highest response times during
the interval.

rate
The number of method executions per second or time interval.
stall
An instance where a methods invocation time has exceeded a threshold
defined by an administrator.

How Introscope measures application health 159

CA Wily Introscope

Types of metrics
Count metrics
Count is an integer. It may represent, for example:
The number of data points which were averaged to compute a metric.
The number of events since a certain point in time.
The number of threads in use.

Examples of count metrics are concurrent invocations and stall count.

Heuristic metrics
Heuristic metrics are used to evaluate and report status. They are integers, but
the integers are symbols of status and do not measure anything. For example, a
dashboard alert may be based on a heuristic metric with these values:
0 = green = normal
1 = yellow = caution
2 = red = danger
Note These values are only examples. Your system may be configured with
different values.

Percentage metrics
Percentages are used to measure resource use against the maximum available
resources. Examples are CPU utilization and GC Heap in use.

String data
In addition to measurements and status, Introscope collects information that
identifies monitored applications and systems. Examples of this type of data are
system component names such as the name of a database, JVM versions, or IP
address.

Viewing metrics
Introscope provides two tools to view the metrics that Introscope gathers
Workstation and WebView. To run these tools, see the Introscope Workstation
Guide and the Introscope WebView Guide.
The illustrations in the sections below show how Workstation displays metrics.

160 Introscope Metrics

Workstation User Guide

The five basic Introscope metrics

Most instrumented methods report five metrics, as shown in the illustration
below.

The metrics listed under a

parent node...

... aggregate the metrics

under the child nodes.

The illustration shows a portion of the Workstation Investigator treethe EJB

node, with an Entity subnode, and an entity bean, ConcreteCustomer_f03aa728.
Under the bean we can see two of its methods, each of which reports the same
five metrics. In this configuration, the metrics at the higher level (in this example,
the entity bean level) aggregate those of the same name below.
You can see a similar arrangement under many of the nodes in the Investigator
tree, particularly nodes whichlike EJBcorrespond to high-level J2EE APIs or
their .NET equivalent.

Average Response Time (ms)

Response Time is the time it takes for a request to complete; this provides a basic
measurement of application response speed. Therefore:
Low response times are desirable.
High response times suggest a problem.

The Average Response Time metric averages the response times of all requests
that were completed during an interval.

The five basic Introscope metrics 161

CA Wily Introscope

Note The count for Average Response Time is identical to the value of
Responses Per Interval.
Mouse over a data point to see
more information about it.

The value of the data point,

8919 ms, is the average
response time of the
requests completed during
the interval.

Four requests were completed

during the interval selected.
This is the count.

The illustration above shows an Average Response Time graph for an EJB session,
as displayed in Introscope Workstation.
In addition to value and count, each data point has min and max data.
Min is the lowest single value of the requests represented in the countin this

example, the request that took the least time to be completed.

Max is the highest single value of the requests represented in the countin

this case, the request that took the most time to be completed.

Triaging using Average Response Time

You can use trends in Average Response Time, coupled with changes in other
metrics, to identify and diagnose problems. (See the index to find information on
the other metrics mentioned in this section.)

162 Introscope Metrics

Workstation User Guide

Consistent problems
Consistently high Average Response Times may indicate the following problems:
Accompanying metric

Possible cause

Low Available Thread count

Inefficient code
Overuse of external system
Slow backend
Too many layers

Periodic problems
Periodically high Average Response Timesas shown by a graph which
periodically spikes, then returns to normalmay indicate the following problems:
Accompanying metric

Possible cause

Low Available Thread count

Frequent GC leaks
Load-related backend bottleneck

Low CPU utilization

Internal chokepoint

Progressive problems
A steady increase in Average Response Time over a long period may indicate the
following problems:
Accompanying metric

Possible cause

Low Respones Per Interval

Resource leakmemory

Concurrent Invocations
Invocations are requests handled by the application and its various parts;
concurrent invocations are the requests being handled at a given time.
Introscope calculates the Concurrent Invocations metric by counting the number
of requests which were not completed during a particular interval.
A low Concurrent Invocations count is desirable.
A high Concurrent Invocations count suggets a problem.

The five basic Introscope metrics 163

CA Wily Introscope

The value of 1 means that one

request was completed during
the selected interval.

The count of 6 means that six

requests were in flight and were
not completed during the interval.

Notice the difference betwee value and count. Requests that were not completed
during the selected interval will likely be completed during subsequent intervals.
Those which are not completed before the end of a specified threshold are called
stalls (see Stall Count on page 168).

Triaging using Concurrent Invocations

You can use trends in Concurrent Invocations, coupled with changes in other
metrics, to identify and diagnose problems. (See the index to find information on
the other metrics mentioned in this section.)

Consistent problems
Consistently high Concurrent Invocation values may indicate the following
problems:
Accompanying metric

Possible cause

n/a

Overuse of external system

Slow backend

164 Introscope Metrics

Workstation User Guide

Accompanying metric

Possible cause

Low responses per interval

Inefficient code
Too many layers

Periodic problems
Periodically high Concurrent Invocation valuesas shown by a graph which
periodically spikes, then returns to normalmay indicate the following problems:
Accompanying metric

Possible cause

Low available connections

Frequently collected garbage leaks

Low available thread count

Internal chokepoint

n/a

Load-related backend bottleneck

Progressive problems
A steady increase in Concurrent Invocations over a long period may indicate the
following problems:
Accompanying metric

Possible cause

Low responses per interval

Rsource leak - threads

Errors Per Interval

Errors are the number of exceptions reported reported by JVM and HTTP error
codes. Examples of errors include:
a 404 Page Not Found status reported by the HTTP server
a SQL exception
a Java exception

The five basic Introscope metrics 165

CA Wily Introscope

Obviously, a low error count is desirable.

The metric is a simple count of errors reported during the interval. The illustration
above shows one data point selected with a value of 11, meaning 11 errors were
reported during that timeslice. Since this is a simple count metric, the value and
Max value will always be the same.
The metric path beneath the graph identifies the application reporting the
exception. To find more information about the errors shown in a graph, check the
logs for that application.

Error snapshots
For systems with ErrorDetector enabled, errors also generate error snapshots
detailed information about what was happening when an error occuredwhich
are stored in the Perst database. A large number of errors will generate a large
amount of documentary information, and preventing this is another reason to
minimize errors.

Responses Per Interval

Repsonses Per Interval reflects number of invocations finished that interval; it is
a measure of data throughput and thus of application performance. Generally:
A high number is desirable.
A low number is undesirable.

166 Introscope Metrics

Workstation User Guide

Of course, an unexpected spike in responses could indicate overuse of the

external system, such as a denial of service attack on a website.

The metric is a simple count of requests completed during an interval.

In the illustration above, the tool tip shows the value of the selected data point.
Since this is a simple count metric, the value and the Max value of the metric will
always be the same.
Note The value of the Responses Per Interval metric is always the same as the
count for the Average Response Time metric.

Triaging with Repsonses Per Interval

You can use trends in Repsonses Per Interval, coupled with changes in other
metrics, to identify and diagnose problems. (See the index to find information on
the other metrics mentioned in this section.)

Consistent problems
Consistently high Responses Per Interval values may indicate:
Over-usage of external system

The five basic Introscope metrics 167

CA Wily Introscope

Stall Count
Stalled requests are those which have not completed within a specified time
threshold. If a request is counted as stalled, that does not mean it is hung and
will never be completed, but that its execution exceeded the stall threshold.
A low count is desirable.
A high count is undesirable.

The default stall threshold is 30 seconds.

Information on stall events is stored in the Perst database.

How stall count is measured

Occasionally, a transaction trace will show several requests that were not
completed during the specified time threshold, i.e. stalls, but the Investigator will
display a different number as the stall count.
This is because stall count is recorded as a point value (at a point in time during
an interval) and not as a range value (for a time period). This means that while
there could be several stall values representing long transactions that are
completed during an interval, only the count available during a single moment is
used as the data point.

Triaging with Stall Count

You can use trends in Stall Count, coupled with changes in other metrics, to
identify and diagnose problems. (See the index to find information on the other
metrics mentioned in this section.)

Consistent problems
Consistently high Stall Count values may indicate:
Slow backend system

Periodic problems
Periodically high Stall Count values may indicate:
Load-related backend bottleneck

Progressive problems
A steady increase in Stall Count values over a long period may indicate:
Accompanying metric

Possible cause

Low available threads

Resource leak - threads

168 Introscope Metrics

Workstation User Guide

Other common metrics

In addition to the five metrics that commonly appear with instrumented methods,
you can see other common metrics at various places in the Investigator tree.

Memory-related metrics
Several metrics report memory-related data using bytes as a unit of measure.

GC Heap metrics
Garbage Collection is the process of freeing memory taken up by objects no
longer in use; once memory is freed up it is useable by other objects.
GC Heap|Bytes In Use
GC Heap|Bytes In Use reports the amount of memory being currently used by
objects.
GC Heap|Bytes Total
GC Heap|Bytes Total reports the total amount of memory allocated by the JVM.

File system, Sockets, UDP

Like Responses Per Interval (page 166), these are measures of data throughput.
They are measured in Bytes Per Second:
File system
File output rate (bytes per second)
File input rate (bytes per second)

UDP (User Datagram Protocol)

Output bandwidth (bytes per second)
Input bandwidth (bytes per second)

Sockets (total as well as host/port specific information)

Output bandwidth (bytes per second)
Input bandwidth (bytes per second)

A large number of port-related metrics indicates socket rate metrics should be

turned off, because this is possibly a metric explosion problem.
For other socket metrics, see Socket metrics on page 171.

Other common metrics 169

CA Wily Introscope

Utilization metrics
Utilization metrics measure the percentage of available resources being used. The
most common is CPU Utilization.

CPU Utilization
CPU utilization is measured by Introscopes platform monitor, and measures the
amount of CPUs being used. There are two different measurements:
CPU:Utilization % (process)
Percentage of the total computing power of the Introscope host, but limited to
the percentage utilized by the JVM process that Introscope monitors.
CPU:Utilization % (aggregate)
Utilization of an individual processor.
The illustration below shows CPU utilization metrics for an 8 processor host. One
of the data points is selected.

170 Introscope Metrics

Workstation User Guide

Socket metrics
Socket metrics are reported by port by type:
Client sockets
Server sockets

They are displayed at the following location in the Investigator tree:

Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)(*SuperDomain*)| Sockets | [Client|Server] |
Enterprise Manager | Port

Accepts Per Interval

The number of Accepts per interval.
Closes Per Interval
The number of sockets per interval that were closed.
Concurrent Readers
The number of threads being read by this port, per interval.
Concurrent Writers
The number of threads being written using this port, per interval.
Opens Per Interval
The number of sockets per interval that were opened.
Input Bandwidth (Bytes Per Second)
Input bandwidth for the port, measured in bytes per second.
Output Bandwidth (Bytes Per Second)
Output bandwidth for the port, measured in bytes per second.

Other common metrics 171

CA Wily Introscope

Thread pool metrics

The Threads metric shows the number of active, or currently in use, instrumented
threads created from classes that have had probes added by Introscope. The
metrics are typically gathered from JMX (on Java applications) or PMI (on
WebSphere applications).
The metrics are divided into:
I/O threads
Worker threads

For both of these types, you can view the following metrics:
Active Threads
Number of active threads.
Available Threads
The total number of threads available.
Maximum Idle Threads
The maximum number of threads that can be idle.
Minimum Idle Threads
The minimum number of threads that can be idle.
Threads in Use
The number of threads in use.
Thread Creates
The number of created threads during the interval.
Thread Destroys
The number of destroyed threads during the interval.
OpenSessionsCurrentCount
The number of currently open sessions.

Connection pool metrics

Connection pool metrics are typically gathered from JMX (on Java applications)
or PMI (on WebSphere applications). The metrics are typically divided into:
Count metrics for individual connection types

172 Introscope Metrics

Workstation User Guide

Percent metrics
Time metrics

The illustration below shows all three kinds of connection pool metrics configured
for a WebSphere application.

Connection pool count metrics

Counts of various kinds of connections, depending on what has been configured
for the application. These usually include:
PoolSize
The number of total connections in the connection pool.
FreePoolSize
The number of free connections in the connection pool.
avgUseTime
Average Use Time.
avgWaitTime
Average wait time.
concurrentWaiters

Other common metrics 173

CA Wily Introscope

Number of waiting threads.

faults
Number of faults.
jdbcOperationTimer
numAllocates
numConnectionHandles
numCreates
numDestroys
numManagedConnections
numReturns
prepStmtCacheDiscards

Connection Pool percent metrics

PercentMaxed
The percentage of connections in the connection pool that are maxed out.
PercentUsed
The percentage of connections in the connection pool that are active.

Event metrics
Event metrics are recordced by Introscope in specific situations. They include:
stalls (see Stall Count on page 168)
system logs

This Metric type monitors the application system out and system error output.
It is typically turned off. See System logs on page 175.
exception

Captures throwing/catching exceptions. Provides the ability to trace all

locations where exceptions are thrown and caught.
Note Exception catching should be turned off in production as it can cause
significant performance degradation.

174 Introscope Metrics

Workstation User Guide

System logs
Standard error
Prints the stderr log in text format.
Standard output
Prints the stdout log in text format.

Using perflog.txt
The Enterprise Manager records performance time for system events in a
performance log file, <Introscope_Home>/logs/perflog.txt. As an
alternative to the metrics displayed in the Investigator, this file may contain
useful information. For information on reading and understanding this file, see
the Introscope Sizing and Performance Guide.

Other metrics
Depending on your system architecture, the following metrics may also appear in
the Introscope Workstation Investigator tree.

EJB
Where Enterprise Java Beans (EJBs) are part of your architecture, the following
metrics appear under one or more of the following sub-nodes:
EJB entity
EJB session
EJB message driven

For each EJB under these three types, Enterprise Manager reports the five basic
Introscope metrics:
Average Response Time (ms)
Concurrent Invocations
Errors Per Interval
Responses Per Interval
Stall Count

For information on these metrics, see The five basic Introscope metrics on
page 161.

Other metrics 175

CA Wily Introscope

Servlets
The Servlets node commonly displays the five basic Introscope metrics for each
of the servlets invoked by the application being monitored by Enterprise
Manager:
Average Response Time (ms)
Concurrent Invocations
Errors Per Interval
Responses Per Interval
Stall Count

For information on these metrics, see The five basic Introscope metrics on
page 161.

JDBC
The JDBC node commonly displays these metrics for JDBC calls invoked by the
application being monitored by Enterprise Manager:
Average Result Processing Time (ms)
The average number of milliseconds it takes for a round-trip query, averaged
over an interval.
Queries Per Second
The number of times this query was issued during each interval.
For example, you can configure the agent to monitor the performance of
individual SQL series using the JDBC protocol. The metric path would be
something like:
*SuperDomain* | <Host_Name> | <Process_Name> | <Agent_Name> | JDBC | SQL
| [Dynamic|Static] | Query | <SQL_Query>

The illustration below shows how these metrics are reported for each query
executed:

176 Introscope Metrics

Workstation User Guide

Connection Count
Number of JDBC connections per interval.
Commit Count
Cumulative count of commits since agent start.
Commits Per Second
Average number of commits per second over the 15-second interval.

JSP (Java Server Pages)

Average Response Time (ms)
Average response time of the _jspService() methods of all the JSPs
executing in the JVM. The Response Times of all the individual JSPs are
averaged to calculate this value.
Responses Per Interval
Number of completed invocations of the _jspService methods of all the JSPs
executing in the JVM in the past 15 second time period.
Average Response Time (ms) by class name
Average response time in milliseconds of the JSP identified by the class name.
Each invocation of the _jspService() method is timed and averaged to arrive
at this value.
Responses Per Interval
Number of completed invocations of the _jspService() method of the JSP
identified by the class name in the most recent 15 second interval.

Other metrics 177

CA Wily Introscope

Responses Per Second

Rate at which the _jspService() methods of all the JSPs executing in the JVM
are being completed.
Responses Per Second by class name
Rate at which invocations of the _jspService() method of JSP identified by
a particular class name are being completed.
Stalled Methods by class name and by method name
The number of JSPs that are taking longer than a defined threshhold to
complete the execution of the _jspService() method.
Concurrent Invocations
The number of threads executing the _jspService() method.

JSP tag libraries (JSP TagLib)

Tag libraries are collections of custom tags used in JSP pages to invoke custom
actions. A custom action is any action not included in the set of six standard
actions provided for in the JSP specification. Examples of tasks invoked by
custom actions are form control, accessing external systems like databases and
email, and flow control.
The following metrics are available for JSP tag libraries:
Average Method Invocation Time (ms)
Method Invocations Per Interval
Average Method Invocation Time (ms) by class name and method name
Method Invocations Per Interval by class name
Method Invocations Per Interval by class name and method name
Method Invocations Per Second
Method Invocations Per Second by class name
Method Invocations Per Second by class name and method name
Concurrent Method Invocations
Concurrent Method Invocations by class name
Concurrent Method Invocations by class name and method name
Stalled Methods over 30 seconds by class name and method name

178 Introscope Metrics

Workstation User Guide

Average Method Invocation Time (ms)

JSP IO TagLibrary
Warning Count
Exception Count

RMI (Remote method invocations)

Remote method invocations are invocations of methods of distributed Java
objectsthat is, Java objects which may exist on more than one host.
The following metrics are available for both RMI clients and RMI servers.
Average Method Invocation Time (ms)
Method Invocations Per Interval
Average Method Invocation Time (ms) by class name
Method Invocations Per Interval by class name
Method Invocations Per Second
Method Invocations Per Second by class name
Stalled Methods over 30 seconds
Concurrent Method Invocations
Concurrent Method Invocations by class name

XML (Extensible Markup Language)

SAX
SAX:Average Method Invocation Time (ms)
SAX:Method Invocations Per Interval
SAX:Average Method Invocation Time (ms) by class name
SAX:Method Invocations Per Interval by class name
SAX:Method Invocations Per Second
SAX:Method Invocations Per Second by class name
SAX:Stalled Methods over 30 seconds by class name and method name
SAX:Concurrent Method Invocations

Other metrics 179

CA Wily Introscope

SAX:Concurrent Method Invocations by class name

XSLT
XSLT:Average Method Invocation Time (ms)
XSLT:Method Invocations Per Interval
XSLT:Average Method Invocation Time (ms) by class name
XSLT:Method Invocations Per Interval by class name
XSLT:Method Invocations Per Second
XSLT:Method Invocations Per Second by class name
XSLT:Stalled Methods over 30 seconds by class name and method name
XSLT:Concurrent Method Invocations
XSLT:Concurrent Method Invocations by class name

JAXM
JAXM|Listener:Average Method Invocation Time (ms)
JAXM|Listener:Method Invocations Per Interval
JAXM|Listener:Average Method Invocation Time (ms) by class name
JAXM|Listener:Method Invocations Per Interval by class name
JAXM|Listener:Method Invocations Per Second
JAXM|Listener:Method Invocations Per Second by class name
JAXM|Listener: Stalled Methods over 30 seconds by class name and method
name
JAXM|Listener:Concurrent Method Invocations
JAXM|Listener:Concurrent Method Invocations by class name

J2EE Connector
Average Method Invocation Time (ms)
Method Invocations Per Interval
Average Method Invocation Time (ms) by class name
Method Invocations Per Interval

180 Introscope Metrics

Workstation User Guide

Method Invocations Per Second

Method Invocations Per Second by class name
Stalled Method count over 30 seconds by class name and method name
Concurrent Method Invocations
Concurrent Method Invocations by class name

JTA (Java Transaction API)

Average Method Invocation Time (ms)
Method Invocations Per Interval
Average Method Invocation Time (ms) by class name
Method Invocations Per Interval by class name
Method Invocations Per Second
Method Invocations Per Second by class name
Stalled Methods over 30 seconds by class name and method name
Concurrent Method Invocations

JNDI (Java Naming and Directory Interface)

JNDI Lookup
Lookup:Context Average Method Invocation Time (ms)
Lookup:Context Method Invocations Per Interval
Lookup:Context Average Method Invocation Time (ms) by class name
Lookup:Context Method Invocations Per Interval by class name
Lookup:Context Method Invocations Per Second
Lookup:Context Method Invocations Per Second by class name
Lookup:Context Stalled Methods over 30 seconds by class name and method
name
Lookup:Context Concurrent Method Invocations
Lookup:Context Concurrent Method Invocations by class name

Other metrics 181

CA Wily Introscope

JNDI lookupLink
lookupLink:Context Average Method Invocation Time (ms)
lookupLink:Context Method Invocations Per Interval
lookupLink:Context Average Method Invocation Time (ms) by class name
lookupLink:Context Method Invocations Per Interval by class name
lookupLink:Context Method Invocations Per Second
lookupLink:Context Method Invocations Per Second by class name
lookupLink:Context Stalled Methods over 30 seconds by class name and method
name
lookupLink:Context Concurrent Method Invocations
lookupLink:Context Concurrent Method Invocations by class name

JNDI search
Search:Context Average Method Invocation Time (ms)
Search:Context Method Invocations Per Interval
Search:Context Average Method Invocation Time (ms) by class name
Search:Context Method Invocations Per Interval by class name
Search:Context Method Invocations Per Second
Search:Context Method Invocations Per Second by class name
Search:Context Stalled Methods over 30 seconds by class name and method
name
Search:Context Concurrent Method Invocations
Search:Context Concurrent Method Invocations by class name

JNDI called metrics

File system I/O

JMS (Java Messaging Service)

The following metrics each appear under one of these four sub-nodes:
message listener

182 Introscope Metrics

Workstation User Guide

message consumer
topic publisher
queue sender

Average Method Invocation Time (ms)

Method Invocations Per Interval
Average Method Invocation Time (ms) by class name
Method Invocations Per Interval by class name
Method Invocations Per Second
Method Invocations Per Second by class name
Stalled Methods over 30 seconds by class name and method name
Concurrent Method Invocations
Concurrent Method Invocations by class name

Java Mail
The metrics each appear under one of these two sub-nodes:
Java Mail (Send)
Java Mail (sendMessage)

Transport:Average Method Invocation Time (ms)

Transport:Method Invocations Per Interval
Transport:Average Method Invocation Time (ms) by class name
Transport:Method Invocations Per Interval by class name
Transport:Method Invocations Per Second
Transport:Method Invocations Per Second by class name
Transport:Stalled Methods over 30 seconds by class name and method name
Transport:Concurrent Method Invocations
Transport:Concurrent Method Invocations by class name

CORBA
Average Method Invocation Time (ms)

Other metrics 183

CA Wily Introscope

Method Invocations Per Interval

Average Method Invocation Time (ms) by class name
Method Invocations Per Interval by class name
Method Invocations Per Second
Stalled methods in any class over 30 seconds
Concurrent Method Invocations
Concurrent Method Invocations by class name

Struts
Average Method Invocation Time (ms)
Method Invocations Per Interval
Average Method Invocation Time (ms) by class name and method name
Method Invocations Per Interval by class name
Method Invocations Per Second
Method Invocations Per Second by class name
Stalled Methods over 30 seconds by class name and method name
Concurrent Method Invocations
Concurrent Method Invocations by class name

Instance Counts
Instance counts metrics measure the number of object instances of a particular
class on the heap.
Approximate Instance Count by package and class name

Data about machines

The following data are reported for the machine hosting the Enterprise Manager
as well as each machine with instrumented methods.
Java Version
Virtual machine
Launch time
Process ID

184 Introscope Metrics

Workstation User Guide

Host IP address
Host operating system
Host wall clock time

Enterprise Manager health and supportability metrics

Health and supportability metrics display information about the Enterprise
Manager rather than the application it is monitoring. They appear in the
Investigator tree, under:

Custom Metric Host (Virtual)

Custom Metric Process (Virtual)
Custom Metric Agent (Virtual)(SuperDomain)
Beneath this level, health and supportability metrics are arranged in the following
hierarchy. Definitions for some of these metrics follow the list.
Agents

<Host_Name>

<Agent_Type> (for example, DatabaseAgent

<Agent_Name>

ConnectionStatus

Historical Metric Count

Is Clamped

Metric Count

Raw Metric Count

Enterprise Manager

Host

Name

Overall Capacity (%)

Port

CPU

EM CPU Used (%)

Configuration

Agent Clusters Metric Load

Number of Agent Clusters

Number of Metric Groupings

Connections

Metrics Queued (%)

Number of Agents

Data about machines 185

CA Wily Introscope

Number of Applications

Number of Historical Metrics

Number of Metrics

Number of Metrics Handled

Number of Workstations

Data Store

SmartStor

Metrics Appended To Query Per Interval

Metrics Converted From Spool to Query Per Interval

SmartStor Disk Usage (mb)

MetaData

Agents with Data

Agents without Data

Metrics with Data

Partial Metrics with Data

Partial Metrics without Data

Write Duration (ms)

Tasks

Converting Spool To Data

Data Append

Reperiodizing

Transactions

Number of Dropped Per Interval

Number of Inserts Per Interval

Number of Queries Per Interval

Number of Traces in Database

Number of Traces in Insert Queue

TT Database Disk usage (mb)

Total Data Insertion Duration Per Interval (ms)

Total Index Insertion Duration Per Interval (ms)

Total Query Duration Per Interval (ms)

Volume Space Free

Baseline Volume Free (mb)

Log Volume Free (mb)

Smartstor Archive Volume Free (mb)

Traces Volume Free (mb)

186 Introscope Metrics

Workstation User Guide

Database

Metric Data Points Sent per Interval

Queued Metric Data Points

GC Heap

GC Duration (ms)

In Use (mb)

In Use Post GC (mb)

Total (mb)

Health

CPU Capacity (%)

GC Capacity (%)

Harvet Capacity (%)

Heap Capacity (%)

Incoming Data Capacity (%)

SmartStor Capacity (%)

Internal

Number of Connection Tickets

Number of metric Data Queries per Interval

Number of Queued Async Data Queries

Number of Registered Async Data Queries

Number of Registered Async MG Queries

Number of Registered Async Path Queries

Number of Transaction Trace Action Sessions

Number of Transaction Trace Session Clients

Number of Virtual Metrics

AlertID

Alerts

<Management_Module_Name>

Agent Connection Status

Number of Evaluated Metrics

Backend Heuristics

CPU Heuristic

Console SUmmery Alert

Frontend Errors Heuristic

Frontend Heuristics

Frontend Response Time Heuristic

Data about machines 187

CA Wily Introscope

JDBC Heuristic

JVM Heuristics

Thread Pool Heuristic

GC Heap

Collectors

Collection Count Per Interval

GC Duration (ms)

Pools

Messaging

Active Incoming Threads

Active Outgoing Threads

Corrupted Messages Per Interval

Post Offices

<Collector_Name>

<Post_Office_Name>

Number of Mailboxes

Queued Messages

Query

Cache Queries Duration (ms)

Cache Queries Per Interval

Smartstor Queries Duration (ms)

Smartstor Queries Per Interval

Threads

<Thread_name>

Blocked Count

Blocked Time (ms)

CPU Time (ms)

User Time (ms)

Wait Count

Wait Time(ms)

Problems

Management Modules

Warning Count

Tasks

Harvest Duration (ms)

Smartstor Duration (ms)

188 Introscope Metrics

Workstation User Guide

Harvest metrics
Harvest Capacity
The Harvest Capacity metric displays the percent of time needed for the data
harvest in a 15-second time slice. For example, if the data harvest takes 15
seconds, the metric value would be 100. The Investigator displays this metric at
the location
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom Metric
Agent (Virtual)(*SuperDomain*)| Enterprise Manager | Health | Harvest
Capacity (ms)

Harvest Duration
The Harvest Duration metric shows the time in milliseconds (during a 15-second
time slice) spent harvesting data. It is generally a good indicator in determining
whether or not the Enterprise Manager is keeping up with the current workload.
You can find this metric at the following location in the Investigator tree.
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom Metric
Agent (Virtual)(*SuperDomain*)| Enterprise Manager | Tasks | Harvest
Duration (ms)

For more information on this metric, see the Introscope Sizing and Performance
Guide.

Incoming Data Capacity (%)

The capacity of the Enterprise Manager to handle incoming data. The metric is
calculated by multiplying the total metric capcity by 2. For example, if 150,000
metrics are in queue waiting to be processed, and the Enterprise Manager has a
capacity to handle 300,000 metrics, Incoming Data Capacity is 25%.
You can find this metric at the following location in the Investigator tree.
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom Metric
Agent (Virtual)(*SuperDomain*)| Enterprise Manager | Health | Incoming
Data Capcity (%)

For more information on this metric, see the Introscope Sizing and Performance
Guide.

Number of Collector Metrics

The Number of Collector Metrics metric shows the total number of metrics
currently being tracked in the cluster. You can find the Number of Collector
Metrics metric here in the Investigator tree:

Data about machines 189

CA Wily Introscope

Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom Metric
Agent (Virtual)(*SuperDomain*)| Enterprise Manager | MOM | Number of
Collector Metrics.

For more information on this metric, see the Introscope Sizing and Performance
Guide.

Collector Metrics Received Per Interval metric

The Collector Metrics Received Per Interval metric is an extremely simple way of
gauging how much load metric data queries are placing on the cluster. This metric
is the total sum of Collector metric data points that the MOM has received each
15-second time period, including data queries. You can find the Collector Metrics
Received Per Interval metric here in the Investigator tree:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom Metric
Agent (Virtual)(*SuperDomain*)| Enterprise Manager | MOM | Collector
Metrics Received Per Interval

A large Collector Metrics Received Per Interval metric value, coupled with
degradation of the cluster, indicates that the MOM has been asked to read too
much metric data from the Collectors.
For more information on this metric, see the Introscope Sizing and Performance
Guide.

Converting Spool to Data metric

The Converting Spool to Data metric tracks whether or not the spool to data
conversion task is running. You can find this metric at the following location in
the Investigator tree:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom Metric
Agent (Virtual)(*SuperDomain*)| Enterprise Manager | Data Store |
SmartStor | Tasks | Converting Spool to Data

If this metric stays at a value of 1 for more than 10 minutes per hour, this
indicates that reorganizing the SmartStor spool file is taking too long. For more
information on this metric, see the Introscope Sizing and Performance Guide.

Overall Capacity (%) metric

The Enterprise Manager Overall Capacity (%) metric estimates the percentage of
the Enterprise Managers capacity that is consumed. You can find it at this
location in the Investigator tree:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)(*SuperDomain*)| Enterprise Manager: Overall
Capacity (%)

190 Introscope Metrics

Workstation User Guide

For more information on this metric, see the Introscope Sizing and Performance
Guide.

SmartStor Capacity (%) metric

The SmartStor Capacity (%) metric displays the percent of time needed for the
SmartStor write process in a 15-second time slice, where 15 seconds equals
100%.
You can find it at this location in the Investigator tree:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)(*SuperDomain*)| Enterprise Manager | Health |
SmartStor Capacity (%)

For more information on this metric and on SmartStor, see the Introscope Sizing
and Performance Guide.

Heap Capacity (%) metric

The Heap Capacity (%) metric is determined by what percentage of heap the JVM
is currently using (based on the GC Heap: In Use Post GC (mb) metric).
For more information on this metric, see the Introscope Sizing and Performance
Guide.

Write Duration (ms) metric

The Write Duration (ms) metric diplays the duration, in milliseconds, of the
SmartStor write process. This is the integer version of the SmartStor Capacity
metric (see above). You can find it in this location in the Investigator tree:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)(*SuperDomain*)| Data Store | SmartStor |
MetaData | Write Duration (ms)

Number of Agents metric

This metric displays the number of currently connected agents. It is located in:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)(*SuperDomain*)| Enterprise Manager |
Connections

Number of Metrics metrics

This metric displays the total metric load on the Enterprise Manager. It is located
in:

Data about machines 191

CA Wily Introscope

Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom

Metric Agent (Virtual)(*SuperDomain*)| Enterprise Manager |
Connections

Historical Metric Count metric

The Historical Metric Count metric shows the total number of metrics from an
agent that are live or recently active. It is located in the Investigator tree in:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)| Agent | Historical Metric Count

Number of Historical Metrics metric

The Number of Historical Metrics metric displays the total number of metrics an
Enterprise Manager is tracking across all agents. You can find this metric at the
following location in the Investigator tree:
Custom Metric Host (Virtual)| Custom Metric Process (Virtual)| Custom
Metric Agent (Virtual)(*SuperDomain*)| Enterprise Manager |
Connections | Number of Historical Metrics.

For more information on this metric, see the Introscope Sizing and Performance
Guide.

192 Introscope Metrics

I ND E X

Index

A
Absolute Max 139
Absolute Min 139
aggregation 161
alerts
changing Alert View options 92
defined with heuristic metrics 102
how to eliminate on transient spikes 103
in sample dashboards 97
analyzing transactions
Transaction Trace Summary View 114
Transaction Trace Tree View 120
authentication
Workstation user permissions 24, 29
Auto Expand 43
Auto Scale 42
Average Max 139
Average Min 139
Average Response Time 161
average response time 159

B
backend 158
backend metrics 72
backslash 125
bar chart data views 26
baselines 66

C
cipher suites 20
clamped 118
cluster
metric for total number of metrics currently
tracked in 189
concurrent 159
concurrent invocations 159, 163

connecting to Enterprise Manager host 12

multiple hosts 17
using proxy authentication 13
connection pool 172
Console
about 21, 34
dashboards 34
exporting data 44
navigation 34
viewing data 44
min/max values in graph 37
Console Lens
and dashboard views 50
applying 49
clearing 50
to filter by agent 48
converting spool to data metric, defined 190
CORBA 183
correlated transactions (or events) 126, 127
count 160
CPU utilization 170

D
dashboards
about the Dashboard Editor 24
alerts in sample dashboards 97
displaying Management Module names
in the Console 34
navigating
from Console to Investigator 34
selecting 34
using hyperlinks 35
Overview dashboard 99
graphs in 100
Problem Analysis dashboard 101
sample Introscope 95

30

Index 193

CA Wily Introscope

saving links to 36
data
diplay
in graphs 40
display 39
changing viewer type 57
in Console 44
in Investigator 54
min/max values 42
exporting from Investigator 93
filtering 48
granularity 46
historical 45
custom time range 47
time range 39
weighting historical data 66
data properties
in reports 134
data properties in reports 136
Data Viewers
about 25
alert 25, 27
defining using Heuristics 102
Heuristics as basic for 66
Heuristics as basis for 64
in Overview dashboard 99
in the Overview tab 61
in Whats Interesting table 65
lines in charts
red 141
yellow 144
showing overall status 97
threshholds 31
values 63
bar chart 26
bar charts, metric data 148
changing Alert View options 92
changing type 57
copying to the clipboard 44
dial meter 26
editing 37
exporting data from the Console 44
graph 25
graphic equalizer 26
string viewer 26
text view 26
tool tips 38, 59
types 25

194 Index

dial meter data views 26

display properties
in reports 135
display properties in reports 138, 139
documentation
in PDF format 28
in searchable Help format 27
domain permissions 57
domains 56
drill down 63

E
EJB 175
EJB metrics 70
elements 132
as members of Management Modules 24
error snapshot 166
ErrorDetector 166
errors 159, 165
escaping special characters 125
events
errors 82
error data in a Transaction Trace 113
querying for 122
historical 119
historical, querying 123
stalls 102
transactions 82, 112
Whats Interesting 65
exceptions 174
exporting
data from the Console 44
Transaction Trace information 126

F
favorites 36
filters
cannot filter by agent 81
for metrics in Colsole 48
for Transaction Traces 109
with Console Lens 49
formating reports 154
frontend 159
frontend metrics 71, 85

G
garbage collection (GC) metrics
GC Heap 169

73

Workstation User Guide

graph data views 25

graphic equalizer data views 26
graphs
Auto Expand 43
Auto Scale 42
displaying min/max values in 37
min and max values 41
moving metrics to front or back in 43
scale 41
showing and hiding metric data in 40
group definitions in reports 145

H
heuristic metrics 160
heuristics
and alerts 102
Heuristics and metric baselines 66
Heuristics node 86
historical baselines for metrics 66
historical data
custom time range 47, 91
selecting a time range 45, 89
viewing 45, 89
viewing in the Console 44
viewing in the Investigator 89
zooming in on data 48
historical events
querying 123
querying saved events 127
historical query 122
home dashboard
dashboards
setting a home dashboard 29
host 184
HTTP tunneling 19
hyperlinks in dashboards 35

I
instance counts 184
instance counts metrics 73
instantiated Java classes 73
instrumented application 10
interval 159
Introscope components 10
Investigator
about 22, 54
navigating in 58
opening 58

viewing data in
IP address 28

89

J
J2EE connector 180
Java mail 183
Java versions
for Java Web Start 13
Java Web Start 13
compared with WebView
JDBC 176
JMS 182
JMX 172
JTA 181
JTA metrics 74
JVM 18

11

L
language settings 31, 139
LeakHunter metrics 75
live data, viewing 44
log out 18
Lucene 124

M
magnifying transaction views 115
Management Module
about the Management Module Editor
displaying in a dashboard 30
sample dashboards 96
Mean 139
metric
Average Response Time (ms) 161
Concurrent Invocations 163
Connection pool 172
CPU Utilization 170
Errors Per Interval 165
File System I/O 169
GC Heap 169
Responses Per Interval 166
sockets 169
Sockets I/O 169
Stall Count 168
Stalls 174
Threads 172
UDP I/O 169
metric baselines 66
metric count 83

24

Index 195

CA Wily Introscope

metric data
in the Console 34
showing and hiding 40
metric definition variables 146
metric grouping 132, 133, 134, 150, 151
metrics
converting spool to data 190
displaying min/max values 37
duplicate names 56
grayed out 54, 56
Heuristics 86
heuristics and alerts 102
historical data, viewing 89
inactive 54, 56
Overall Capacity (%) 190, 191
MIB 29
min and max 162
min/max metric values 37

N
navigation
from Console to Investigator 34
from trace viewer to Investigator 114
nodes 161

O
Overall Capacity (%) metric
defined 190, 191
spiking 191

P
PBDs (ProbeBuilder Directives) 56
permissions 24, 29
viewing domains 57
pie chart 83
platform monitor 170
PMI 172
properties
data, in reports 136
display, in reports 138, 139
report 138

Q
query 122
historical events 123
historical query 122
query syntax 122, 124

196 Index

R
reports
creating templates 130
creating, generating, and viewing 129
data properties 134
defining properties 134
display properties 135, 139
elements
adding 132
link to a metric grouping 133, 150
generating 152
properties 138
sample report templates 154
setting custom group definitions 145
specifying data properties 136
specifying display properties 138
specifying report formats 154
specifying report properties 138
text settings 132
time range, default 136
time range, overriding 133
titles 132
using the Report Editor 134
response time 159
Responses Per Interval 159
responses per interval 166
RMI 179

S
sample dashboards 96
searchable help 27
troubleshooting 28
seasonality, metric baseline 67
servlet metrics 76
Servlets 176
socket metrics 77
sockets 169
special characters 125
SQL 72, 116
SSL
using with Workstation 20
stall 86, 159, 168
starting the Workstation
on Windows 12
string data views 26
struts 78, 184
Summary View 114
SuperDomain 56

Workstation User Guide

supportability metrics
syntax 122, 124

55, 56

T
text data views 26
text settings in reports 132
thread pool 172
threads metrics 79
threshold 31
time range 39
custom 47
for historical data 45, 89
timestamp (relative) 116, 117
tool tip 84
tool tips 38, 59, 117
Trace View header 115
traffic lights--see alerts 25
Transaction trace
tool tip information 117
Transaction Tracer
about 106
and anti-flooding logic 107
and shutoff implications 106
automatic tracing 106
overhead 107
printing 121
restarting a session 110
saving a selected Transaction Trace to a text
file 126
starting a session 108
stopping a session 110
Summary View 114
Trace View header 115
trace viewer tabs 113
trace viewer, using 112
Transaction Trace Table 112
Tree View 120
using with previous-version Agents 107
transactions
clamped 118
correlated 126
querying 123
truncated 119
transient data spikes 103
transport.tcp 20
Tree View 120
truncated 119
truststore 20

tunneling, HTTP

19

U
UDP 169
URL
in a historical query 125
in a Tranaction Trace 116
starting Workstation 12
to dashboards 36
URL metrics 63, 87
URL to start Workstation 15
user permissions 24, 29, 57

V
variables 146
viewing 92
alert messages 92
historical data in the Console 45
historical data in the Investigator 89
viewing metrics 160

W
warning threshholds 31, 110
web start
see Java Web Start 11
WebView
compared with Java Web Start 11
Workstation
about 21
as part of Introscope 11
bar chart views 26
Console 21
Dashboard Editor 24
data viewers 25
dial meter views 26
exiting 19
graph views 25
graphic equalizer views 26
help system 27
HTTP tunneling 19
Investigator 22
language settings 31
logging out of 18
Management Module Editor 24
overview 9
reporting 129
sample dashboards 95
shared state in multiple 9

Index 197

CA Wily Introscope

starting on Windows 12
starting using a URL 15
starting using Java Web Start 13
string data views 26
text views 26
using SSL 20
Workstation Investigator tree 161

X
XML 179
XML component metrics
-Xms/-Xmx 18

80

Z
zoom in on historical data
zoom slider 115

198 Index

48