CA Wily Introscope

Java Agent Guide

Version 7.2

Date: 01-2008

Copyright 2008, CA. All rights reserved. Wily Technology, the Wily Technology Logo, Introscope, and All Systems Green are registered trademarks of CA. Blame, Blame Game, ChangeDetector, Get Wily, Introscope BRT Adapter, Introscope ChangeDetector, Introscope Environment Performance Agent, Introscope ErrorDetector, Introscope LeakHunter, Introscope PowerPack, Introscope SNMP Adapter, Introscope SQL Agent, Introscope Transaction Tracer, SmartStor, Web Services Manager, Whole Application, Wily Customer Experience Manager, Wily Manager for CA SiteMinder, and Wily Portal Manager are trademarks of CA. Java is a trademark of Sun Microsystems in the U.S. and other countries. All other names are the property of their respective holders. For help with Introscope or any other product from CA Wily Technology, contact Wily Technical Support at 1-888-GET-WILY ext. 1 or [email protected]. If you are the registered support contact for your company, you can access the support Web site directly at http://support.wilytech.com. We value your feedback. If you have comments or suggestions about Wily documentation, please send us an e-mail at [email protected].

6000 Shoreline Court, Suite 200 South San Francisco, CA 94080

US Toll Free 888 GET WILY ext. 1 US +1 630 505 6966 Fax +1 650 534 9340 Europe +44 (0)870 351 6752 Asia-Pacific +81 3 6868 2300 Japan Toll Free 0120 974 580 www.wilytech.com

CONTENTS

Table of Contents

SECTION I
Chapter 1

Java Agent Installation and ProbeBuilding

The Java Agent Overview The Introscope environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . . .

1
3 . 4 . 5 . 5 . 5 . 6 . 6 . 7 . 7 . 7 . 8 . 9 12 12 12 13 13 13

Planning a Java Agent implementation Discover Introscope functionality . Define Java Agent configuration .

Determine configuration requirements .

Evaluate Java Agent performance overhead . Validate and deploy Java Agent configuration Implementing the Java Agent . Basic implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Java Agent installation directories and files . Java Agent Configuration Options Wily CEM integration . Before you start . . . . . . . . . .

Application server support .

Enterprise Manager connection information . ProbeBuilding method and options .

Planning the installation, configuration, and evaluation process . Chapter 2 Installing and Configuring the Java Agent . Installing the Java Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Agent Installer Archives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 15 . . . . . . . 16 16 17 18 18 19 19

Java Agent-to-Enterprise-Manager connection properties . Configuring the Java Agent name Configuring ProbeBuilder options Configuring JVM AutoProbe . .

Sun, IBM, or HP JVM AutoProbe .

Contents iii

CA Wily Introscope

JRockit JVM AutoProbe JVM 5.0 AutoProbe . .

. .

. . . . .

. . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . . .

. . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . .

24 25 25

Uninstalling the Java Agent Chapter 3 ProbeBuilding Options .

. 27 . . . . . . . . . . 28 28 29 29 30 30 30 31 31 32

Full or typical tracing options .

Dynamic ProbeBuilding (JVM 1.5)

Configuring dynamic ProbeBuilding (JVM 1.5)

Recommended dynamic instrumentation configuration settings . ProbeBuilding multi-level class hierarchies (JVM 1.5) Enable instrumentation of multiple levels of subclasses . Configure periodic polling for uninstrumented subclasses Disable directive updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Controlling directive logging

Removing line numbers in bytecode Chapter 4 ProbeBuilder Directives . . . . .

. 33 . . . . . . . . . . . . . . . . . . . . 34 34 34 35 36 36 37 37 39 39 40 41 42 42 43 43 43 47 48 48

Directives and the ProbeBuilding process . Default component tracing . ProbeBuilder Directive files . Instrumentation and inheritance . Default ProbeBuilder Directives files

Full and typical ProbeBuilder Directives sets . Modifying default ProbeBuilder Directives . Default Tracer Groups and toggles files Turning Tracer Groups on or off . Adding classes to a Tracer Group . Blame Tracers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Setting toggles to gather additional metric information .

Using Blame Tracers to mark blame points . Blame Tracers in standard PBDs . Creating Custom Tracers . . . . .

Boundary Blame and Oracle backends . Custom Tracers and keywords .

Custom method tracer examples . Creating advanced Custom Tracers . Advanced single-metric tracers .

iv Contents

Java Agent Guide

Skips

. . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

50 51 51 51 52 52 56 56 57

Combining custom tracers . Counting object instances . Directive keywords . . . . ProbeBuilder Skip keywords

ProbeBuilder action keywords .

Applying new and changed ProbeBuilder Directives . Copy in new ProbeBuilder Directives files . Introscope-enabling with new and changed PBDs . Chapter 5 AutoProbe for Application Servers . Before You Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 59 . . . . . . . . . . . . . . . . . . . 60 60 60 60 61 61 62 62 62 63 63 63 64 65 65 66 66 67 68

Configuring WebLogic Server to use AutoProbe WebLogic Server 6.1 . WebLogic Server 7.0 . WebLogic Server 6.1 SP3

WebLogic Server 8.1, 9.0, or 9.1 . WebSphere 4.0 (Distributed) . WebSphere 6.0/5.1/5.0 . Java2 Security Policy . . . . . . . . .

Configuring WebSphere to Use AutoProbe

Configuring WebSphere z/OS to Use AutoProbe WebSphere 5.x and 6.0 for z/OS . Java2 Security Policy . Sun ONE 7.0 . . . . . . . . . . . . . . . . . . . . . . .

Configuring Sun ONE to use AutoProbe

Configuring Oracle 10g to use AutoProbe . Oracle 10g 10.0.3 . Configuring HTTP servlet tracing . Modifying Java2 Security Policy . Chapter 6 Manual ProbeBuilding . Before you begin . . . . . . .

. 69 . . . . . 70 70 70 72 72

Manual ProbeBuilding options . Using the ProbeBuilder wizard

Update application startup script.

Using ProbeBuilder from the command line .

Contents v

CA Wily Introscope

Adding Probes to bytecode using ProbeBuilder from the command line 72 Editing the classpath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 75 75 76 Running Introscope-enabled code . ProbeBuilder Wizard.lax . . . . .

Switching back to non-Introscope-enabled code .

SECTION II
Chapter 7

Java Agent Operations and Management .

Configuring Agent HTTP Tunneling . Configuring Agent HTTP tunneling . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . . . . . . . . . . . . .

. 77
. 79 . . 80 80

Configuring a proxy server for Agent HTTP tunneling Chapter 8 Java Agent Naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. 83 . . . . . . . . . . . . . . . . . . 84 84 86 86 87 88 88 89 89 89 90 90 91 92 92 92 92 93

Understanding the Java Agent name . Aspects of the Java Agent name . How the Agent determines its name.

How Introscope resolves Agent naming conflicts

Agent naming considerations for clustered applications . Specifying an Agent name using a system property key Obtaining an Agent name from the application server . Application servers that support Agent naming . How automatic Agent naming works Enabling automatic Agent naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Specifying an Agent name using a Java system property .

Automatic Java Agent naming and renamed agents . Advanced automatic Agent naming options . Disabling Agent naming for WebSphere Cloned Agent naming scenario. . . .

Enabling cloned Agent naming in clustered environments . Enabling cloned Agent naming in the Agent profile.

Configuring unique names for application instances . Chapter 9 Agent Monitoring and Logging . . . . . . . . . . . . . . . . . . . . . . .

. 95 . . . . 96 96 97 97

Configuring Java Agent connection metrics . Configuring Agent logging options . . . . . .

Turning off socket Metrics in the Agent profile . Running the Agent in verbose mode .

vi Contents

Java Agent Guide

Redirecting Agent output to a file.

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . .

97 98 98 99 99 99 101

Agent log files and automatic Agent naming . Logging considerations for WebSphere z/OS . Managing the ProbeBuilder Log . . . . . . . . . . . . . ProbeBuilder log name and location . AutoProbe log name and location . Chapter 10

Using Virtual Agents to Aggregate Metrics . Understanding Virtual Agents . Virtual Agent requirements Configuring Virtual Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 102 . 102 . 102 105 . 106 . 106 . 107 . 107 . 108

Chapter 11

Configuring Agent Failover . Understanding Agent failover .

Define backup Enterprise Managers. Define failover connection order .

Configuring Java Agent failback to primary Enterprise Manager . Agent failover and domain/user configuration .

SECTION III Tailoring and Extending Data Collection

Chapter 12

.
. . . . . . . . . . . . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . .

.
. . . . . . . . . . . . . . . .

109
111 . 112 . 112 . 113 115 . 116 . 116 . 117 . 121 . 122 . 122 123 . 124 . 124 . 124 . 125

Configuring Access to Application Server Data . Application Server management data and the Agent . Configuring startup class for WebLogic 8.1 or 9.0 . .

Configuring a custom service in WebSphere 5.0 or 6.0 . Chapter 13 Configuring Boundary Blame . Understanding Boundary Blame . Using URL groups . . . . . . . . . . . . . Configuring URL groups . Running the URLGrouper. Using Blame tracers . . . Disabling Boundary Blame . Chapter 14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Configuring Transaction Trace Options Transaction Trace component clamp .

. . . . .

Controlling automatic transaction tracing behavior Transaction trace sampling by URL group . Configuring automatic trace sampling .

Contents vii

CA Wily Introscope

Configuring cross-process transaction tracing . Enabling cross-process tracing in WebSphere Extending transaction trace data collection . About User ID data. . . . . . . . . . . . . . . . . . . . . . . . . . . . About servlet request data . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 125 . 125 . 126 . 126 . 126 . 127 . 127 . 128 131 . 132 . 133 . 133 . 134 . 134 . 134 . 136 . 136 . 138 . 138 . 138 141 . 142 . 142 . 142 . 143 . 144 . 145 . 145 . 147 149 . 150 . 150 . 150 . 151 . 152

Enabling cross-process tracing in WebLogic Server

Configuring Agent to collect additional transaction trace data . Disabling the capture of stalls as Events . Chapter 15 Configuring Introscope SQL Agent . SQL Agent overview . SQL Agent files . . . . . . . . . . . . . . .

Supported JDBC drivers and data sources Configuring SQL Agent for WebSphere

Step 1: Configure the JDBC datasource or Driver in WebSphere . Step 2: Introscope-enable the JDBC DataSource or Driver . Configuring SQL Agent for WebLogic Server . Turning off Blame Metrics for SQL Agent . Turning Off Statement Metrics SQL Agent Metrics . Chapter 16 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Manually Introscope-enabling the JDBC DataSource or Driver .

Enabling JMX Reporting .

Introscope Java Agent JMX support . Default JMX Metric conversion process

Introscope support for WebLogic 9.0 JMX Metrics .

Using primary key conversion to streamline JMX Metrics Managing Metric volume with JMX filters . JMX filters for WebLogic . Configuring JMX reporting . . . . . . . . . . . . . . . . .

Enabling JSR-77 data for WAS 6.x . Chapter 17 Configuring Platform Monitoring Understanding platform monitors .

Enabling platform monitors on Windows Server 2003 Enabling platform monitors on AIX . Disabling platform monitors . . .

Troubleshooting platform monitoring .

viii Contents

Java Agent Guide

Chapter 18

Configuring WebSphere PMI Enabling PMI in WebSphere . . .

. . .

. . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

153 . 154 . 154 . 156 . 156 . 157 159 . 160 . 160 . 161 163 . 163 . 163 . 163 165 . 166 . 167 . 169 . 169 . 169 . 169 . 170 . 170 . 172 . 173 . 174 . 175 . 176 . 177 . 177 . 177 . 177 . 178 . 178 . 180

Agent support for WebSphere PMI .

Using PMI with Introscope on z/OS . Configuring PMI in Introscope. Viewing WebSphere Agent PMI data Chapter 19

Enabling WebLogic Diagnostic Framework . Understanding WLDF Metric conversion . Enabling WLDF reporting . . . . . . . . . . . . . . . . . . . . . . . . .

. . .

Agent support for WebLogic Diagnostic Framework (WLDF) .

Appendix A

Wily Knowledge Base Articles . The Wily Java Agent . . . . . . . Accessing the Java Agent

. . . .

Searching and Browsing Java Agent Articles . Appendix B Agent Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. .

. . . . . . . . . . . . . . . . . . . .

Configuring IntroscopeAgent.profile location Agent naming properties Wily CEM integration . . Agent failover properties

Agent to Enterprise Manager connection properties . Error Detector properties . Extensions properties JMX properties . . . . . . . .

Transaction tracing properties LeakHunter properties .

Platform monitoring properties . WebSphere PMI properties . Socket metrics properties . SQL Agent properties. . . . . . . . Stall metrics properties . URL grouping properties WLDF metrics properties AutoProbe properties . Logging properties . . .

Agent thread priority properties .

Contents ix

CA Wily Introscope

Blame properties .

. . .

. . .

. . .

. . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. 182 . 182 . 182 . 183 . 183 185 . 186 . 186 . 186 . 187 189

Agent HTTP Tunneling Properties CPU Utilization Properties . . .

Agent HTTP Tunneling Proxy Server Properties . Cross-process tracing in WebLogic Server Appendix C Using the Introscope PBD Generator . About the Wily PBD Generator . . . . . . . . . . . . . . Configuring the PBD Generator . Using the PBD Generator Index . . . . . . . . . . . . . . . .

Required PBD Generator parameters

x Contents

SECTION

Java Agent Installation and ProbeBuilding

The chapters in this section describe the Java Agent implementation process.
Installing and Configuring the Java Agent on page 15 ProbeBuilding Options on page 27 ProbeBuilder Directives on page 33 AutoProbe for Application Servers on page 59 Manual ProbeBuilding on page 69

Java Agent Installation and ProbeBuilding 1

CA Wily Introscope

2 Java Agent Installation and ProbeBuilding

SECTION

The Java Agent Overview

This chapter explains the Java Agent implementation process. The Introscope environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . 5 . 7 . 9 12

Planning a Java Agent implementation . Implementing the Java Agent . Before you start . . . . . Java Agent Configuration Options .

The Java Agent Overview 3

CA Wily Introscope

The Introscope environment

CA Wily Introscope is an application management tool that provides end-to-end performance management of your applications. The Introscope Java Agent collects performance data from applications running on Java Virtual Machines (JVMs), and sends it to the Introscope Enterprise Manager. The Enterprise Manager processes the data received from all agents and sends it to the Introscope Workstation where users can set up actions and alerts based on the data received. To monitor an application in a JVM, the application is Introscope-enabled through the ProbeBuilding process. The ProbeBuilding process involves defining tracers in the ProbeBuilder Directives (PBD) file, then applying these tracers to your application using Introscope AutoProbe. Standard PBD files containing basic monitoring tracers come with your Introscope installation, but you can also create custom PBDs to better define the data you want to monitor for your application. The Java Agent then collects the tracer data and sends it to the Enterprise Manager. The figure below illustrates the key components of an Introscope environment.

4 The Java Agent Overview

Java Agent Guide

Planning a Java Agent implementation

It is important to develop the right Java Agent configuration for your application and the environments in which it runs. The figure below illustrates the key processes in a Java Agent implementation process.

Discover Introscope functionality

The first step in developing an Introscope implementation involves test driving the default Introscope Java Agent configuration. A default Java Agent configuration demonstrates data collection functionality and is key to understanding and evaluating the out-of-the box features of the Java Agent and Introscope as a whole. When you install Introscope, a default Java Agent configuration is included. The Java Agent provides a variety of data collection options out-of-the box and can be customized to collect more environment-specific data. However, the more metrics a Java Agent collects, the more system resources it consumes. When evaluating the environment, the primary goal is to understand the depth and breadth of Introscopes data collection and application management features. As you refine your Java Agent configuration, you will streamline data collection to balance the depth of data collection against overhead constraints and configure Java Agent features that help manage and limit resource consumption.

Determine configuration requirements

Before introducing Introscope into your environment, whether pre-production or live, you should determine your data collection requirements. This information will help you tailor the data collection behaviors of the Java Agent, and evaluate the impact on overhead through alternative configurations of the Java Agent.

Planning a Java Agent implementation 5

Java Agent Guide

Since Introscope is employed across an application lifecyclein development, test, and productionyour monitoring goals, environmental constraints, and service level requirements will change over time. You will need to configure Java Agents differently in each phase or environment. Java Agent configuration is a trade-off between visibility vs. overhead. The goal is to obtain optimal visibility at a reasonable cost. In pre-production environments, such as development and QA, you typically configure a higher level of data collection to provide deeper visibility into the performance characteristics of the application. In production or production-like environments, you reduce the level of metric reporting to control Java Agent overhead, and when appropriate, implement optional configurations, such as Virtual Agents or agent failover. If you intend to collect data from multiple environments, you will need to develop an appropriate Java Agent configurations for each.

Define Java Agent configuration

After defining your configuration requirements based on your application and its operating environment, you should create a candidate agent configuration. Most Java Agent behaviors are configured in the agent profile. Some features may also require some configuration in your application server, or required other configuration steps. Depending on the complexity of your configuration and the target environment, you may choose to build up the Agent configuration in stages, so that you can evaluate the impact of each add-on componentsuch as LeakHunter, ErrorDetector, or Introscope PowerPacks and ensure it is working before adding more.

Evaluate Java Agent performance overhead

When evaluating a Java Agent configuration, verify that the metrics collected provide sufficient visibility into application performance and availability, and that the volume of metrics do not impose an unacceptable load on the operating environment. The Java Agent should not report more metrics than are necessary to identify and localize performance and availability problems. To effectively understand and evaluate Java Agent overhead, you must understand the performance characteristics of the application prior to Introscopeenabling it.

Planning a Java Agent implementation 6

Java Agent Guide

For example, you can load test your application before and after implementing out-of-the-box monitoring to verify impact. Similarly, a conservative approach is to extend data collection in a controlled fashionfor instance, one PowerPack at a timeand evaluate the impact of each add-on individually. Occasionally, too many Java classes are selected for monitoring in a ProdeBuilding Directive (PBD) file, causing the Java Agent to start incorrectly, or to experience a hang. If this happens, use the AutoProbe.log file to identify the classes that caused the Java Agent to hang and add a skip directive to the PBD file, skipping the classes that may have caused the problem. For more information about adding skip directives to your custom PBD files, see Skips on page 50.

Validate and deploy Java Agent configuration

After you have verified that a candidate Agent configuration provides the visibility required for the target environment without imposing unacceptable overhead, you should deploy the validated configuration to that environment. In practice, the process of deploying a validated configuration includes installing the validated configuration artifactsspecifically IntroscopeAgent.profile and modified or custom .pbd filesto the target environment.

Implementing the Java Agent

The sections that follow describe the steps in implementing data collection and the Java Agent directory structure and configuration artifacts.

Basic implementation
The Java Agent implementation process is as follows: Step 1 Install the Agent on the target JVM. For more information, see Installing and Configuring the Java Agent on page 15. Step 2 Configure the properties in IntroscopeAgent.profile that govern the operating and data collection behaviors of the Java Agent, including which PBDs to use during the ProbeBuilding process and optional ProbeBuilding behaviors. For more information, see ProbeBuilding Options on page 27 and ProbeBuilder Directives on page 33. Step 3 Use a supported method of ProbeBuilding and desired PBDs to Introscope-enable your applications, and configure ProbeBuilding options. Step 4 Restart your application and start data collection.

Implementing the Java Agent 7

Java Agent Guide

Java Agent installation directories and files

A Java Agents operating and data collection behaviors are controlled by configuration properties stored in the agent profile and the PBDs it references. Installing a Java Agent creates the following directory structure:
Wily Connectors Ext Tools

Contents of the wily directory

The Wily directory contains:
Agent.jarThe Java Agent executable .jar file WebAppSupport.jarContains startup classes that you can configure to allow

the Java Agent to obtain management information from an application server.

IntroscopeAgent.profileThe Agent profile contains properties that

control the behavior of the Agent and AutoProbe. Defaults are supplied for many properties; the default values for certain properties vary, depending on the application server your Agent installation supports. If you have requirements to locate the Agent profile in other than the Wily directory, see Configuring IntroscopeAgent.profile location on page 166.
PBDsContains the standard PBDs provided with the Java Agent, which vary,

depending on the application server your Agent installation supports. AutoProbe looks for the PBDs you configure for use during the ProbeBuilding process. A template custom.pbd file is located in this directory for your use when creating custom PBDs. If you create custom PBDs, they must be saved in this directory.

Contents of the wily\connectors directory

CreateAutoProbeConnector.jarUsed in configuring JVM AutoProbe in
supported environments.

Contents of the wily\ext directory

CEMTracer.jarUsed in environments that integrate Wily Customer

Experience Manager (CEM) with the Introscope Agent.

ServletHeaderDecorator.jarUsed in environments that integrate Wily

CEM with the Introscope agent.

Interitance.jarUsed in Java 1.5 environments to enable AutoProbe to

Introscope-enable multiple levels of subclasses of a probed class.

Implementing the Java Agent 8

Java Agent Guide

JavaI5DynamicInstrumentation.jarUsed in Java 1.5 environments to

enable dynamic instrumentation, which allows you to implement new and changed PBDs without restarting the managed application or the Agent.
ProbeBuilder.jarContains ProbeBuilder executable. One of the following extensions to ProbeBuilder.jar

BasicDirectiveLoader.jar SignedJARDirectiveLoader.jar UnifiedDirectiveLoader.jar

SQLAgent.jar Contains SQL Agent executable. Supportability-Agent.jarSupportability extensions for use by Wily

Technical Support in Agent support.

Contents of the wily\tools directory

URLGrouper.jarContains the URLGrouper, a command-line utility that

analyzes web server log files and produces BRTA property settings for a set of URL Groups. For more information about the URLGrouper, see Running the URLGrouper on page 121.

Java Agent Configuration Options

This section is an overview of configurable Agent behaviors.
Communications with Enterprise Manager Java Agent naming Virtual Agents Logging options Domains ProbeBuilding alternatives and options ProbeBuilder Directives (PBDs) Data collection and reporting option

Java Agent Configuration Options 9

Java Agent Guide

Communications with Enterprise Manager

You must configure the location of the Enterprise Manager to which the Java Agent reports. If you do not, the Java Agent will try to connect with the Enterprise Manager on localhost port 5001 by default. If the Agent will connect to an Enterprise Manager cluster, you must configure it to connect to a Collector Enterprise Manager. To enable an Agent to failover to a secondary Enterprise Manager, you must define connection properties and connection order as well. For more information, see Java Agent-to-Enterprise-Manager connection properties on page 17.

Java Agent naming

You can define a Java Agents name explicitly or configure an automated mechanism for agent naming. By default, the Java Agent profile explicitly assigns an agent name, for instance WebLogic Agent. For more information, see Java Agent Naming on page 83.

Virtual Agents
If you want multiple agents to monitor separate instances of a clustered application, you must configure those agents as a Virtual Agents which allows you to aggregate metrics at the application level. For more information, see Using Virtual Agents to Aggregate Metrics on page 101.

Logging options
By default, the Java Agent writes information log messages to the console window. You can configure the Java Agent for more detailed logging and direct messages to log files. For more information, see Configuring Agent logging options on page 97.

Domains
Unless you assign a Java Agent to a custom Introscope Domain, it is part of the SuperDomain by default. For information about Domains and their use in configuring user permission, see the Introscope Installation Guide.

ProbeBuilding alternatives and options

Introscope provides multiple methods of Introscope-enabling your applications: JVM AutoProbe, Application Server AutoProbe, and Manual ProbeBuilder. JVM AutoProbe is typically used in environments that support it, and is enabled by default in the Agent profiles for those environments. There are optional features, such as dynamic instrumentation, available if your Agent runs on JVM 1.5. For more information, see ProbeBuilding Options on page 27.

Java Agent Configuration Options 10

Java Agent Guide

Important Application Server Autoprobe is not supported on any JVM 1.5 and above platforms.

ProbeBuilder Directives (PBDs)

The ProbeBuilder Directives (PBDs) used during the ProbeBuilding process determine the metrics that the Agent reports. You configure a list of the desired PBDs or PBLs (lists of PBDs) in the Agent profile. The default list specified in the profile results in the typical level of probe-building, appropriate for production environments where overhead is at a premium. You configure more detailed probe building to obtain more metrics in development or QA environments in the agent profile. You can further control the probe-building process by customizing PBDs to skip classes or packages, or to Introscope-enable custom classes and custom or private methods that the default PBDs do not specify. For more information, see ProbeBuilder Directives on page 33.

Data collection and reporting option

Most data collection behaviors are controlled by Java Agent properties.
Socket MetricsBy default, the Java Agent does not report input and output

bandwidth rate metrics for individual sockets. For more information, see Turning off socket Metrics in the Agent profile on page 96.
URL Groups for Blame ReportingTo control the way that metrics for front-

ends are aggregated and presented in the Investigator in WebView and the Workstation, you must configure URL groups. The Java Agent profile contains properties for specifying URL groups. For more information, see Using URL groups on page 116.
Stall Event ReportingBy default, the Java Agent reports stalls as events, and

stores them in the Transaction Event Database. You can disable this behavior, or tailor the stall reporting behavior. For more information, see Disabling the capture of stalls as Events on page 128.
JMX and JSR-77You can optionally configure the Java Agent to report JMX

metrics, JSR-77 metrics, or WebLogic Diagnostic Framework metrics. For more information, see Enabling JMX Reporting on page 141 and Enabling WebLogic Diagnostic Framework on page 159
Transaction Tracing BehaviorYou can tailor the behavior of the automatic

transaction tracing the Agent performs, and configure the collection of User IDs for Servlet and JSP invocations. For more information, see Configuring Transaction Trace Options on page 123.
PMIIn WebSphere environments you can configure the reporting of

WebSphere Performance Monitoring Infrastructure (PMI) metrics. For more information, see Configuring WebSphere PMI on page 153.
Platform MonitoringPlatform monitors enable the Agent to report system

Java Agent Configuration Options 11

CA Wily Introscope

metrics, including CPU statistics, to the Enterprise Manager. Platform monitors on all operating systems except Windows Server 2003 and AIX are automatically enabled upon Agent installation. Windows Server 2003 and AIX platform monitors require a minimal configuration to work. For more information, see Configuring Platform Monitoring on page 149.
SQL AgentIntroscope SQL Agent is installed automatically with the Agent

installation. This Agent extension provides visibility into the performance of individual SQL statements. For more information, see Configuring Introscope SQL Agent on page 131.

Wily CEM integration

There are several steps you are required to perform to ensure that the Java Agent integrates with a Wily CEM installation. See the CA Wily Customer Experience Manager Integration Guide for more information about integrating your Java Agent with Wily CEM.

Before you start

This section lists some of the key information, decisions, and resources that should be identified or obtained before you start to configure your Java Agent.

Application server support

The following table lists the application servers supported by the Java Agent, and any requirements that application server has for functionality with the Java Agent: Application server Requirements
WebLogic WebSphere WebSphere on z/OS Sun ONE Version 6.1, or higher, and any required patches. WebSphere Application Server 4.0, or higher, and any required patches. WebSphere Application Server 5.0, or higher, and any required patches. Sun ONE Application Server and any required updates. For Sun ONE version 7.0, the minimum Sun ONE versions required for Introscope integration are: Sun ONE AS Platform Edition 7.0.0_01 (update 1) Sun ONE AS Standard Edition 7.0.0_01 (update 1) To download the appropriate application server versions, see http://wwws.sun.com/software/download/app_servers.html.

12 The Java Agent Overview

Java Agent Guide

Oracle 10g SAP NetWeaver

Oracle 10g version 10.0.3 Application Server and any required updates. SAP NetWeaver 6.2 and 6.4 (NWO4)

Enterprise Manager connection information

The Java Agent runs on the JVM where the applications you wish to monitor run, and connects to the Introscope Enterprise Manager. If your Agent reports to a clustered Enterprise Manager you must configure it to connect to a Collector Enterprise Manager. If you have multiple Enterprise Managers, clustered on not, you can configure your Java Agent to failover to an alternate Enterprise Manager if it disconnects from its primary Enterprise Manager.

ProbeBuilding method and options

The Java Agent supports several methods of Introscope-enabling your applications. JVM AutoProbe, the most typical choice, dynamically Introscope-enables all classes loaded by the JVM, adding probes that generate metrics from the Java bytecode. JVM AutoProbe is supported if you use:
Java 5 JVM a Sun or IBM JVM JRockit HP Hotspot 1.3 or higher. This can be obtained from http://www.hp.com/

products1/unix/java. For more information, see Installing and Configuring the Java Agent on page 15. If your JVM does not support it, or you prefer to use another method, you can either configure Application Server AutoProbe, as described in AutoProbe for Application Servers on page 59, or perform the ProbeBuilding process manually as described in Manual ProbeBuilding on page 69.

Planning the installation, configuration, and evaluation process

Most Introscope deployments include one or more extension products such as LeakHunter, Error Detector, or PowerPacks that extend data collection. Before starting to implement the agent, map out the sequence of installations, review and decide what configuration options are appropriate for the Agent and extension products, and identify at which points you wish to load test or otherwise evaluate the configuration.

Before you start 13

CA Wily Introscope

Note The Java Agent will require at least 20 MB of free disk space on the system on which it will be installed.

14 The Java Agent Overview

SECTION

Installing and Configuring the Java Agent

This chapter has instructions for installing the Introscope Java Agent and configuring the JVM AutoProbe method of Introscope-enabling your applications. To verify that your environment supports JVM AutoProbe or to learn about other methods of Introscope-enabling your applications, see ProbeBuilding method and options on page 13. Installing the Java Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 17 18 19 25

Java Agent-to-Enterprise-Manager connection properties Configuring the Java Agent name . Configuring JVM AutoProbe. Uninstalling the Java Agent . . . .

Installing and Configuring the Java Agent 15

CA Wily Introscope

Installing the Java Agent

Unzip the Java Agent installer archive into a Java working directoryoften the application servers home directory. For information about the directory structure of the Agent installation, see Java Agent installation directories and files on page 8. The Java Agent requires 20 MB of free disk space to be installed. To install the Java Agent: 1 Select an installer archive for your specific JVM. For a list of installer archives, see Agent Installer Archives, below. 2 Extract the files of the installer archive into a location your JVM can access. 3 Configure the properties in IntroscopeAgent.profile to your specific environmental needs. This file governs the Java Agents operating and data collection behaviors, including which PBDs to use during the ProbeBuilding process, optional ProbeBuilding behaviors, and the connection to the Enterprise Manager. You will need to know the location of the files extracted in step 2 above. Configuration options for the IntroscopeAgent.profile are covered in the next sections of this chapter and the rest of this Guide. 4 Use a supported method of ProbeBuilding and desired PBDs to Introscope-enable your applications, and configure ProbeBuilding options. For more information about ProbeBuilding, see Configuring ProbeBuilder options on page 18. 5 Restart your application and start data collection.

Agent Installer Archives

The table below lists the Java Agent installer archives for different application servers and operating systems. Application Server
WebLogic

Installer Archive Packages

IntroscopeAgentInstall7.2.0.0weblogic.unix.tar IntroscopeAgentInstall7.2.0.0weblogic.windows.zip IntroscopeAgentInstall7.2.0.0weblogic.zOS.tar IntroscopeAgentInstall7.2.0.0weblogic.os400.zip

WebSphere

IntroscopeAgentInstall7.2.0.0websphere.unix.tar IntroscopeAgentInstall7.2.0.0websphere.windows.zip IntroscopeAgentInstall7.2.0.0websphere.zOS.tar IntroscopeAgentInstall7.2.0.0websphere.os400.zip

Sun ONE

IntroscopeAgentInstall7.2.0.0sunoneas.unix.tar IntroscopeAgentInstall7.2.0.0sunoneas.windows.zip

16 Installing and Configuring the Java Agent

Java Agent Guide

Application Server
Oracle 10g SAP NetWeaver

Installer Archive Packages

IntroscopeAgentInstall7.2.0.0oracleas.unix.tar IntroscopeAgentInstall7.2.0.0oracleas.windows.zip

Windows:

IntroscopeAgentInstall7.2.0.0netweaver.windows.zip
Solaris, HP-UX, AIX, and Linux:

IntroscopeAgentInstall7.2.0.0netweaver.unix.tar
For SAP support only: Windows:

IntroscopeAgentInstall7.2.0.0netweaver.windows.SAP. zip
Solaris, HP-UX, AIX, and Linux:

IntroscopeAgentInstall7.2.0.0netweaver.unix.SAP. tar
Other
IntroscopeAgentInstall7.2.0.0default.unix.tar IntroscopeAgentInstall7.2.0.0default.windows.zip IntroscopeAgentInstall7.2.0.0default.zOS.tar IntroscopeAgentInstall7.2.0.0default.os400.zip

Java Agent-to-Enterprise-Manager connection properties

The default communications settings in the Java Agent profile enable an agent to connect to a local Enterprise Manager. In a recommended Introscope implementation, an agent and the Enterprise Manager to which it reports do not reside on the same systemso you must update IntroscopeAgent.profile, to identify the remote Enterprise Manager. These properties specify connection behavior for the DEFAULT Enterprise Manager communication channel:
introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT

Defines the host name or IP address of the target Enterprise Manager.

introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT

Defines the Enterprise Managers listen port, which should be the same port specified in the Enterprise Managers introscope.enterprisemanager.port.DEFAULT property.
introscope.agent.enterprisemanager.transport.tcp.socketfactory.

DEFAULT

Java Agent-to-Enterprise-Manager connection properties 17

CA Wily Introscope

Defines the socket factory used for connections to the Enterprise Manager. To specify an alternative Enterprise Manager channel, define its connection properties by adding these properties to the Java Agent profile:
introscope.agent.enterprisemanager.transport.tcp.host.NAME introscope.agent.enterprisemanager.transport.tcp.port.NAME Introscope.agent.enterprisemanager.transport.tcp.socketfactory.

NAME Replace

NAME with an identifier for the new Enterprise Manager channel, and

specify the Enterprise Managers connection details. For more information about connection properties, Agent to Enterprise Manager connection properties on page 169.

Configuring the Java Agent name

The Java Agent profile provides default values for the agent name and Custom Process name, which appear as nodes in the Investigator hierarchy of metrics reported by the Java Agent. Depending on your environment, you may wish to configure the Java Agent to obtain a name automatically. For more information about agent naming, and autonaming capabilities, see Java Agent Naming on page 83. If desired, you can define the Agent name and process name using these properties in IntroscopeAgent.profile.
introscope.agent.agentNameName the application server that the Java

Agent is monitoring.The agentName value must start with an alphabetical character, and cannot contain a percent (%) character. See introscope.agent.agentName on page 167 for a description.
introscope.agent.customProcessName Name the process being

monitored. See introscope.agent.customProcessName on page 168 for a description.

Configuring ProbeBuilder options

By default, the AutoProbe will use the typical PBD set provided with the Java Agent, which results in the collection of a moderated volume of metric collection. For instructions on how to customize the metric collection level, or to configure optional ProbeBuilding behaviors, see ProbeBuilding Options on page 27.

18 Installing and Configuring the Java Agent

Java Agent Guide

Configuring JVM AutoProbe

Configure JVM AutoProbe using the instructions that correspond to your JVM.
Sun, IBM, or HP JVM AutoProbe on page 19 JRockit JVM AutoProbe on page 24 JVM 5.0 AutoProbe on page 25

Sun, IBM, or HP JVM AutoProbe

This section has instructions for creating and running the AutoProbe Connector for a Sun, IBM, or HP JVM. If your JVM is v5.0, follow the instructions in JVM 5.0 AutoProbe on page 25.

Create AutoProbe Connector

1 Change the working directory to wily/connectors under the installation directory. 2 Run the Create AutoProbe Connector tool using one of these commands:
to specify the JVM using the JVM that is running the tool:

java -jar CreateAutoProbeConnector.jar -current

to specify the JVM by passing the JVM directory on the command line:

java -jar CreateAutoProbeConnector.jar -jvm <directory>

The output is a file with the form: wily/connectors/

AutoProbeConnector.jar

Run AutoProbe Connector

After you create the AutoProbe Connector for the Sun or IBM JVM, you run it. The way you run the Connector depends on the application server you use. For more information, see the following sections:
Run AutoProbe Connector for WebLogic on page 20 Run AutoProbe Connector for WebSphere 4.0 on page 20 Run AutoProbe Connector for WebSphere 5.0, 5.1, or 6.0 on page 21 Run AutoProbe Connector for SAP J2EE 6.20 on page 22 Run AutoProbe Connector for NetWeaver 04/SAP J2EE 6.40 on page 23 Run AutoProbe Connector for Sun ONE on page 23 Run AutoProbe Connector for Oracle 10g on page 24 Run AutoProbe Connector for other application servers on page 24

Configuring JVM AutoProbe 19

CA Wily Introscope

Run AutoProbe Connector for WebLogic

To run the AutoProbe Connector for WebLogic: 1 Edit the bootstrap classpath in the application startup script to include the AutoProbeConnector.jar you created (such as startMedRecServer.cmd) using this command:

-Xbootclasspath/p:PathToAutoProbeConnectorJar:PathToAgentJar
add the -X switch to the final start command at the end of the script, after the JAVA_VM and JAVA_OPTIONS. The excerpt below shows the correct place to insert the switch:
"$JAVA_HOME/bin/java" ${JAVA_VM} ${MEM_ARGS} ${JAVA_OPTIONS} -Xbootclasspath/p:${WL_HOME}/wily/connectors/ AutoProbeConnector.jar:${WL_HOME}/wily/Agent.jar -Dweblogic.Name=${SERVER_NAME} -Dweblogic.management.username=${WLS_USER} -Dweblogic.management.password=${WLS_PW} -Dweblogic.ProductionModeEnabled=${PRODUCTION_MODE} -Djava.security.policy="${WL_HOME}/server/lib/weblogic.policy" weblogic.Server

2 If you are using something other than the default bootstrap classpath, add the Agent.jar and AutoProbeConnector.jar files to the beginning of your customized bootstrap classpath.

Run AutoProbe Connector for WebSphere 4.0

To run the AutoProbe Connector for WebSphere 4.0: 1 Start the WebSphere Administrators Console, and navigate to the JVM Settings section for the application server you want to modify. 2 Click Advanced JVM Settings. A new window opens. 3 Enter the following command in the Boot classpath (prepend) field:

wily/connectors/AutoProbeConnector.jar:PathToAgentJar
4 Apply the changes in the Administrators Console. 5 In WebSphere environments with Java2 Security enabled, for AutoProbe to run correctly, it may be necessary to add permissions to your Java2 Security Policy. To add permissions, edit the <WebSphere home>/properties/server.policy file to include these lines:

// permissions for Introscope AutoProbe grant codeBase "file:${was.install.root}/-" { permission java.io.FilePermission "${was.install.root}${/ }wily${/}-", "read"; permission java.net.SocketPermission "*", "connect,resolve";

20 Installing and Configuring the Java Agent

Java Agent Guide

permission permission permission permission permission permission permission }; grant { permission };

java.lang.RuntimePermission java.lang.RuntimePermission java.lang.RuntimePermission java.lang.RuntimePermission java.lang.RuntimePermission java.lang.RuntimePermission java.lang.RuntimePermission

"setIO"; "getClassLoader"; "modifyThread"; "modifyThreadGroup"; "loadLibrary.*"; "accessClassInPackage.*"; "accessDeclaredMembers";

java.util.PropertyPermission "*", "read,write";

6 Restart WebSphere.

Run AutoProbe Connector for WebSphere 5.0, 5.1, or 6.0

To run the AutoProbe Connector for WebSphere 5.0, 5.1, or 6.0: 1 Start the WebSphere Administrators Console, and navigate to the JVM Settings section for the application server you want to modify:
For WebSphere 6.0: navigate to Application Servers > your_server > Java

and Process Management > Process Definition > Java Virtual Machine.
For WebSphere 5.1/5.0: navigate to Application Servers > your_server >

Process Definition > Java Virtual Machine 2 Set the Generic JVM Arguments field in the following format:
-Xbootclasspath/p:<Path-ToAutoProbeConnector.jar>AutoProbeConnector.jar;<Path-ToAgent.jar>Agent.jar -Dcom.wily.introscope.agentProfile=<path-toIntroscopeAgent.profile>IntroscopeAgent.profile -Dcom.wily.introscope.agent.agentName=<your-Agent-Name>

For example:
-Xbootclasspath/p:<AppServerHome>/wily/connectors/ AutoProbeConnector.jar:<AppServerHome>/wily/Agent.jar -Dcom.wily.introscope.agentProfile=<path-toIntroscopeAgent.profile>IntroscopeAgent.profile -Dcom.wily.introscope.agent.agentName=<your-Agent-Name>

Use a semicolon (;) as a path separator on Windows systems. Note The /p: option in the command above prepends the supplied path to the default bootstrap classpath. For example: If you installed the Java Agent in c:\Program Files\IBM\WebSphere\AppServer\profiles\default , the Generic JVM Arguments field would be set to:

Configuring JVM AutoProbe 21

CA Wily Introscope

-Xbootclasspath/p:C:/PROGRA~1/IBM/WebSphere/AppServer/profiles/ default/wily/connectors/AutoProbeConnector.jar;C:/PROGRA~1/IBM/ WebSphere/AppServer/profiles/default/wily/Agent.jar -Dcom.wily.introscope.agentProfile=C:/PROGRA~1/IBM/WebSphere/ AppServer/profiles/default/wily/IntroscopeAgent.profile -Dcom.wily.introscope.agent.agentName=WebSphereAgent

For troubleshooting Java Agent start-up problems, review the basic WebSphere Application Server log (i.e native_stderr.log). If an incorrect configuration is placed in the Generic JVM Arguments section and you experience errors starting the server, please review and correct the generic JVM arguments in the server.xml file. 3 Click OK. 4 Apply the changes in the Administrators Console. 5 The default for file encoding for WebSphere 5.x and 6.0 on z/OS is now ASCII, but Introscope expects an EBCDIC file format. When you configure JVM AutoProbe for z/OS, click on Custom Properties and add the name value pair:
name: com.wily.introscope.default.encoding value: Cp1047

Note This step is for z/OS operating systems only. 6 In WebSphere environments with Java2 Security enabled, for AutoProbe to run correctly, it may be necessary to add permissions to your Java2 Security Policy. If Java2 Security is enabled, follow the instructions in Modifying Java2 Security Policy on page 68. 7 Restart WebSphere. In a moment you will see WebSphere metrics in your Introscope Workstation.

Run AutoProbe Connector for SAP J2EE 6.20

To run the AutoProbe Connector for SAP J2EE 6.20: 1 Open the file:

<drive>:\usr\sap\<J2EE_ENGINE_ID>\j2ee\j2ee_<INSTANCE>\cluster\ server\cmdline.properties
2 Append these commands to JavaParameters section:

-Xbootclasspath/p:PathToAutoProbeConnectorJar;PathToAgentJar -Dcom.wily.introscope.agentProfile=<path-toIntroscopeAgent.profile> -Dcom.wily.introscope.agent.agentName=<yourAgentName>

For example:

22 Installing and Configuring the Java Agent

Java Agent Guide

-Xbootclasspath/ p:C:\usr\sap\P602\j2ee\j2ee_00\ccms\wily\connectors\AutoProbeC onnector.jar;C:\usr\sap\P602\j2ee\j2ee_00\ccms\wily\Agent.jar -Dcom.wily.introscope.agentProfile=C:\usr\sap\P602\j2ee\ j2ee_00\ccms\wily\IntroscopeAgent.profile

3 Restart the SAP server.

Run AutoProbe Connector for NetWeaver 04/SAP J2EE 6.40

1 Run the SAP J2EE Configtool. 2 Select the server to modify. 3 Add these new java parameters in the Java Parameters field:

-Xbootclasspath/p:PathToAutoProbeConnectorJar;PathToAgentJar -Dcom.wily.introscope.agentProfile=<path-toIntroscopeAgent.profile>
For example:

-Xbootclasspath/p:D:/usr/sap/ccms/wily/connectors/ AutoProbeConnector.jar;D:/usr/sap/ccms/wily/Agent.jar -Dcom.wily.introscope.agentProfile=D:/usr/sap/ccms/wily/ IntroscopeAgent.profile

Note For NetWeaver 6.40 on Windows, the slashes for these java parameters must be forward slashes. 4 Click the Disk button to save. 5 Repeat steps 2 - 4 for each server. 6 Restart the SAP server. 7 To verify that Configtool changes were made, open the file

<drive>:\usr\sap\ccms\P66\JC00\j2ee\cluster\instance.properties
8 Look for a line beginning with ID<server_id>.JavaParameters, and confirm that it contains the lines you entered.

Run AutoProbe Connector for Sun ONE

To run the AutoProbe connector for Sun ONE: 1 Log in as Administrator or Root. You must be logged in with Administrator or Root permissions to add Introscope information to startup scripts for Sun ONE 7.0. 2 Open the server.xml file, located at:

<SunONE install dir>/domains/domain1/server1/config/

Note The item separator is a colon (:).

Configuring JVM AutoProbe 23

CA Wily Introscope

3 Add this line to the server.xml file:

<jvm-options> -Xbootclasspath/p:PathToAutoProbeConnectorJar:PathToAgentJar </jvm-options>

For example:
<jvm-options> -Xbootclasspath/p:/sw/sun/sunone7/wily/connectors/ AutoProbeConnector.jar:/sw/sun/sunone7/wily/Agent.jar </jvm-options>

Run AutoProbe Connector for Oracle 10g

To run the AutoProbe Connector, modify the bootstrap classpath:

-Xbootclasspath/p:wily/connectors/ AutoProbeConnector.jar:PathToAgentJar

Run AutoProbe Connector for other application servers

To run the AutoProbe Connector, add the Agent.jar and the AutoProbe Connector to the Application Server bootstrap classpath using this command:

-Xbootclasspath/p:wily/connectors/ AutoProbeConnector.jar:PathToAgentJar

JRockit JVM AutoProbe

To configure the JRockit JVM for AutoProbe, start the JRockit JVM using these command-line options:
For JRockit 7.0, SP2 and below:

-Djrockit.preprocessor.class=com.wily.introscope.api.weblogic. PreProcessor -Xbootclasspath/a:PathToAgentJar

For JRockit 7.0, SP3 and above, and for WebLogic JRockit 8.1:

-Xbootclasspath/a:PathToAgentJar -Xmanagement:class=com.wily.introscope.api.jrockit. AutoProbeLoader

For WebLogic 9.0 with JRockit 5.0:

JAVA_VENDOR=Bea JAVA_OPTIONS=%JAVA_OPTIONS% -javaagent:PathToAgentJar

24 Installing and Configuring the Java Agent

Java Agent Guide

JVM 5.0 AutoProbe

If you use JVM 5.0 from any vendor, configure AutoProbe to use Java 5.0 JVM by adding these options to the JVM command line:

-javaagent:PathToAgentJar -Dcom.wily.introscope.agentProfile=PathToAgentProfile

Uninstalling the Java Agent

Uninstalling the Java Agent requires you to know where the Java Agent was installed in each of your applications. The basic process is to remove all Wily elements from your JVMs. To uninstall the Java Agent from JVM 1.5.x, JVM 1.6.x, HP, JRockit JVMs, or WLS 6.1+: 1 Remove the Java Agent switches from the JVM command line. These include:
-Xbootclasspath -javaagent any other Wily-specific arguments (for example:

Dcom.wily.introscope.agentProfile=xxxxx)
2 Reboot your application. To uninstall the Java Agent from WebSphere Application Server (WAS) 5.0, 5.1, or 6.0: 1 Connect to the Administration Console for your WebSphere Application Server. 2 Remove the Java Agent switches from the "Generic JVM Arguments". 3 Reboot your application.

Uninstalling the Java Agent 25

CA Wily Introscope

26 Installing and Configuring the Java Agent

SECTION

ProbeBuilding Options

This chapter has information about ProbeBuilding options. Full or typical tracing options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 28 30 32

Dynamic ProbeBuilding (JVM 1.5) .

ProbeBuilding multi-level class hierarchies (JVM 1.5) Removing line numbers in bytecode .

ProbeBuilding Options 27

CA Wily Introscope

Full or typical tracing options

In Introscope, ProbeBuilder List (PBL) files govern which Tracer Groups are used in the instrumentation process. The introscope.autoprobe.directivesFile property specifies one or more PBL files. Introscope provides two versions of each default PBLa full version which enables a larger set of Tracer Groups than the typical version which results in more detailed metric reporting, and a typical version that enables a smaller set of Tracer Groups, resulting in less detailed metric reporting, and as a result, reduced overhead. By default, introscope.autoprobe.directivesFile specifies the typical version of the default PBL file. To change the tracing level between full and typical, specify the name of the PBL file you wish to use in introscope.autoprobe.directivesFile. For example, to use the Full version of the standard PBL for WebLogic Server, set the property to:
introscope.autoprobe.directivesFile=weblogic-full.pbl

and restart the managed application. Note For more information on full and typical system directives files and customizing the TYPICAL settings, see Default ProbeBuilder Directives files on page 36.

Dynamic ProbeBuilding (JVM 1.5)

Introscopes dynamic instrumentation feature allows you to implement new and changed PBDs without restarting the managed application or the Java Agent. This is useful for making corrections to PBD, or to temporarily change data collection levels during triage or diagnosis without interrupting application service. When dynamic instrumentation is enabled, Introscope periodically checks for new and changed PBDs. To minimize overhead, Introscope selectively re-instruments classes affected by the modified PBDs. Arrays, interfaces, and classes specified in Skip directives are not reinstrumented. In addition, you can exclude all classes loaded by particular classloaders from the re-instrumentation process and limit the scope of the reinstrumentation process to specific class packages. Dynamic instrumentation is not enabled by default. If a class is re-instrumented so that it no longer reports data for a metric, the metric is still displayed in the Introscope Investigator. Existing metrics do not disappear from the Investigator window if their classes are reinstrumented.

28 ProbeBuilding Options

Java Agent Guide

Configuring dynamic ProbeBuilding (JVM 1.5)

To configure dynamic ProbeBuilding: 1 Verify that introscope.autoprobe.enable is set to true. 2 Add this property setting to the IntroscopeAgent.profile file:
introscope.autoprobe.dynamicinstrument.enabled=true.

3 Exclude classes from the dynamic instrumentation in terms of the classloader which loads them, adding this property to the agent profile:
introscope.autoprobe.dynamicinstrument.avoidClassLoaders

Supply a comma-separated list of classloader class names. Specify the system classloader with the string System. 4 Limit the classes that are subject to dynamic instrumentation by explicitly identifying them by package name, adding this property in the agent profile:

introscope.autoprobe.dynamicinstrument.instrumentList
Specify all or a portion of a package name, starting with the beginning of the package name. Wildcards are not supported. For example:
introscope.autoprobe.dynamicinstrument.instrumentList=com.xyz, com.foo.ejb.session

If you do not specify this property, no classes are instrumented. To instrument all classes, set the property to all:
introscope.autoprobe.dynamicinstrument.instrumentList=all

5 To change the frequency which Introscope polls for uninstrumented classes from the default of 1 minute, add this property to the Agent profile and set its value to the desired frequency:
introscope.autoprobe.dynamicinstrument.pollIntervalMinutes

Recommended dynamic instrumentation configuration settings

The recommended configuration for dynamic instrumentation is:
introscope.autoprobe.dynamicinstrument.enabled=true introscope.autoprobe.dynamicinstrument.instrumentList=

com.xyz,com.foo.ejb.session
substituting com.xyz,com.foo.ejb.session with a list of the packages you wish to be dynamically instrumented.
introscope.autoprobe.dynamicinstrument.avoidClassLoaders=System

Dynamic ProbeBuilding (JVM 1.5) 29

CA Wily Introscope

ProbeBuilding multi-level class hierarchies (JVM 1.5)

In pre-5.0 JVMs, Introscope does not automatically instrument classes in the deeper levels of class hierarchyonly the classes that explicitly extend a probed class. For more information, see Instrumentation and inheritance on page 34. On JVM 5.0, you can configure Introscope to instrument multiple levels of subclasses of a probed classthe Tracer Groups in the associated internal directive will be updated appropriately, and the classes will be dynamically instrumented. Directive changes will be written to a log file as well. If you prefer to update your PBDs manually, you can disable directive updates and use the log file to determine appropriate updates.

Enable instrumentation of multiple levels of subclasses

Follow these steps to configure Introscope to dynamically update internal directives. To enable instrumentation of multiple levels of subclasses: 1 Verify that dynamic instrumentation is enabled as described in Configuring dynamic ProbeBuilding (JVM 1.5) on page 29. 2 To enable instrumentation of multiple levels of subclasses, add this property setting to the Java Agent profile:

introscope.autoprobe.hierarchysupport.enabled=true

Configure periodic polling for uninstrumented subclasses

When multi-level subclass instrumentation is enabled, Introscope will check for uninstrumented subclasses at application startup. To configure Introscope to poll for uninstrumented subclasses: 1 Add this property setting to the Java Agent profile:

introscope.autoprobe.hierarchysupport.runOnceOnly=false
2 To change the frequency with which Introscope polls for uninstrumented subclasses from its default value of 5, add this property to the Java Agent profile, and set it to the desired polling frequency:
introscope.autoprobe.hierarchysupport.pollIntervalMinutes

30 ProbeBuilding Options

Java Agent Guide

3 Optionally, you can limit the number of times that Introscope polls uninstrumented subclasses by adding this property, set to the desired limit, to the Agent profile:
introscope.autoprobe.hierarchysupport.executionCount

Disable directive updates

If multi-level subclass instrumentation is enabled, when Introscope detects uninstrumented subclasses, by default, it updates internal directives appropriately to ensure the classes are instrumented. If you prefer to update PBDs manually, you can disable internal directive updates by adding this property setting to the Java Agent profile:
introscope.autoprobe.hierarchysupport.disableDirectivesChange=true

Controlling directive logging

When multi-level subclass instrumentation is enabled, Introscope logs the directive updates required to instrument all subclasses. For example:
1/24/06 03:39:26 PM PST [INFO] [IntroscopeAgent.inheritance] IdentifyClassAs: C [Tracer_Group]

This information is logged in both the pbdupdate.log file and the agent.log file. To prevent this information from being logged in the agent.log file, add the following property to the agent profile:
log4j.additivity.IntroscopeAgent.inheritance=false

To disable directive logging, add this property to the agent profile:

introscope.autoprobe.hierarchysupport.disableLogging=true

Recommended Logging Configuration

log4j.logger.IntroscopeAgent.inheritance=INFO,pbdlog log4j.appender.pbdlog.File=pbdupdate.log log4j.appender.pbdlog=com.wily.introscope.agent.AutoNaming RollingFileAppender log4j.appender.pbdlog.layout=com.wily.org.apache.log4j. PatternLayout log4j.appender.pbdlog.layout.ConversionPattern=%d{M/dd/yy hh:mm:ss a z} [%-3p] [%c] %m%n

ProbeBuilding multi-level class hierarchies (JVM 1.5) 31

CA Wily Introscope

Recommended Subclass Hierarchy Instrumentation Configuration

introscope.autoprobe.hierarchysupport.enabled=true introscope.autoprobe.hierarchysupport.runOnceOnly=false

Removing line numbers in bytecode

When you Introscope-enable application bytecode, the AutoProbe or ProbeBuilder preserves the bytecode line numbers by default. Preserving bytecode line number information is helpful when using debuggers, or when obtaining stack trace information. You can turn off this feature, (which will remove all line numbers when AutoProbe or ProbeBuilder Introscope-enables the application code), by adding a system property on the Java command line. To remove line numbers in bytecode when using AutoProbe or ProbeBuilder:
define the following system property on the Java command line with the -D

option:

com.wily.probebuilder.removeLineNumbers=true

32 ProbeBuilding Options

SECTION

ProbeBuilder Directives

This chapter describes how to create and modify ProbeBuilder Directives. Directives and the ProbeBuilding process Default ProbeBuilder Directives files . Creating Custom Tracers Directive keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 36 41 43 48 51 56

Using Blame Tracers to mark blame points . Creating advanced Custom Tracers .

Applying new and changed ProbeBuilder Directives .

ProbeBuilder Directives 33

CA Wily Introscope

Directives and the ProbeBuilding process

ProbeBuilder Directives tell the ProbeBuilder how to add probes, such as timers and counters, to Java components, to create an Introscope-enabled application. The directives used by ProbeBuilder to Introscope-enable your applications determine what metrics your agents report to the Enterprise Manager. Introscope includes a set of default directives that are generic to many Java systems, as well as those that can extend support of default tracers for WebLogic, WebSphere and Sun ONE. You can create your own custom directives that track classes and methods unique to your application. For more information about directives and tracers for custom PBDs, please see the Directive & Tracer Type Definitions document on the Wily community site.

Default component tracing

Introscope default PBDs implement tracing of the following Java components:
Oracle JDBC JSP Tag Libraries JSP IO Tag Libraries JSP DB Tag Libraries Struts Servlets Java Server Pages (JSPs) Enterprise JavaBeans (EJBs) Java Database Connectivity (JDBC) Network Sockets Remote Method Invocation (RMI) Extensible Markup Language (XML) Java Transaction API (JTA) Java Naming and Directory Interface

(JNDI)
Java Message Service (JMS) Common Object Request Broker

Architecture (CORBA)
User Datagram Protocol (UDP) File Systems Threads System Logs Thrown and Caught Exceptions (off by

default)

Instrumentation and inheritance

WARNING In pre-5.0 JVMs, Introscope does not automatically instrument classes in the deeper levels of a class hierarchy. When subclasses of a probed class more than one level deep are loaded, the new and overridden methods are not automatically instrumented. Likewise, classes that do not explicitly name a probed interface as being implemented, even though they implement the interface indirectly, will not be instrumented either.

34 ProbeBuilder Directives

Java Agent Guide

For example, assume a class hierarchy in which ClassB extends ClassA, and ClassC extends ClassB. In pre-5.0 JVMs, when you instrument ClassA, ClassB is also instrumented because it explicitly extends ClassA. However, Introscope does not instrument ClassC, because ClassC does not explicitly extend ClassA. In pre-1.5 Java environments, to ensure that subclasses are instrumented, follow the instructions in EJB subclass tracing on page 40. If you run under JVM 1.5, you can configure Introscope to instrument multiple levels of subclasses of a probed class. For instructions, see ProbeBuilding multilevel class hierarchies (JVM 1.5) on page 30. If you wish to instrument a private method, you may need to use more specific tracers. For more information about directives and tracers for custom PBDs, please see the Directive & Tracer Type Definitions document on the Wily community site.

ProbeBuilder Directive files

There are two kinds of files used to specify ProbeBuilder Directives. You specify one or more of these files to tell the ProbeBuilder where to find the directives to use.
ProbeBuilder Directive files (.pbd files)

A ProbeBuilder Directive file contains ProbeBuilder Directives.

ProbeBuilder List files (.pbl files)

A ProbeBuilder list file contains a list of ProbeBuilder Directives files. Different

.pbl files can reference the same .pbd files.

Multiple ProbeBuilder Directive files can be specified and are all used together. In a directives file, each line that causes the reporting of metrics is called a tracer. Tracers track methods, classes, and sets of classes. A single-metric tracer causes the reporting of a specific metric for a specific method. The smallest unit that Introscope can track is a method. By default, Introscope traces many common Java and J2EE components using Tracer Group. A Tracer Group consists of a set of tracers that is applied to a set of classes. For example, there are Tracer Groups which report the response times and rates for all RMI classes. You can turn Tracer Groups on or off, and/or add new classes to the set affected by the Tracer Group. See the section, Modifying default ProbeBuilder Directives on page 37.

Directives and the ProbeBuilding process 35

CA Wily Introscope

Default system ProbeBuilder files

If you are using AutoProbe, the relevant PBD and PBL files for your specific application server are placed in the <appserver home>/wily directory when you install the Java Agent. Introscope provides basic system ProbeBuilder Directives files. These files should generally not be edited, but there are some modifications you can perform. See Modifying default ProbeBuilder Directives on page 37.

Custom ProbeBuilder Directives files

While system ProbeBuilder Directives files track basic information, you can customize Introscope to track any classes or methods to obtain specific information about your application. See Creating Custom Tracers on page 43 and Creating advanced Custom Tracers on page 48.

Default ProbeBuilder Directives files

Introscope provides a set of ProbeBuilder Directives files that gather basic system information. ProbeBuilder Directives files included with the Agent for a specific application server will contain ProbeBuilder directives files specific to that application server.

Full and typical ProbeBuilder Directives sets

Tracer Groups cause the reporting of information about a set of classes. Tracer Groups are found in ProbeBuilder Directive files, and referred to in PBL (ProbeBuilder list) files. There are two sets of PBL files available with each Agent installer:
Fullreferences PBD files in which most Tracer Groups in PBD files are turned

on.
Typical (default)a subset of Tracer Groups in the referenced PBD files are

turned on. The typical set includes common settings, and is the set you can customize for a particular environment. Note For instructions on switching between full and typical .PBL files, see Full or typical tracing options on page 28.

36 ProbeBuilder Directives

Java Agent Guide

Modifying default ProbeBuilder Directives

The ProbeBuilder Directive files included with Introscope generally should not be edited. However, the Tracer Groups in the files can be modified in the toggles.pbd files (explained in the following section) by:
turning off Tracer Groups to save on system overhead (and turn back on) adding Classes to a Tracer Group

Default Tracer Groups and toggles files

The <appserver>-full.pbl and <appserver>-typical.pbl files reference toggles-full.pbd and toggles-typical.pbd files. Tracer Groups are turned on or off within the toggles-full.pbd or toggles-typical.pbd files. In the toggles-full.pbd file, most of the Tracer Groups are turned on by default. In the toggles-typical.pbd file, only a subset of Tracer Groups are turned on. The following table lists the default Tracer Groups, and shows which are turned on in the full and typical toggles files. Name Definition Default full setting
on on on on on on on on

Default typical setting

on on on on on on on on on on on on on on

CorbaTracing EntityBeanTracing SessionBeanTracing MessageDrivenBeanTracing J2eeConnectorTracing JavaMailTransportTracing JDBCQueryTracing JDBCUpdateTracing JMSListenerTracing JMSConsumerTracing JMSPublisherTracing JMSSenderTracing JSPTracing RMIClientTracing

CORBA method invocations Entity EJB method invocations Session EJB method invocations Message-driven EJB method invocations J2EE connector information Mail sending times JDBC queries JDBC updates

JMS message processing times on JMS message processing times on JMS message broadcast times JMS message broadcast times JSP service responses RMI client method invocations on on on on

Default ProbeBuilder Directives files 37

CA Wily Introscope

Name

Definition

Default full setting

Default typical setting

on on

RMIServerTracing HTTPServletTracing

RMI server method invocations on HTTP servlet service responses on

Note: If you are using Application Server AutoProbe, turn on this Tracer Group:

HTTPAppServerAutoProbeServletTracing
StrutsTracing InstanceCounts Execution times of actions in the Struts framework Counts number of instances of object type identified with Tracer Group. on on on on

Note: Nothing will be traced until classes are identified with this Tracer Groups.
FileSystemTracing JAXMListenerTracing JNDITracing JSPDBTagsTagLibraryTracin g JSPIOTagLibraryTracing File system bytes written and read JAXM message sends JNDI lookup times Jakarta DB Tags custom tag library for reading and writing from a SQL database Jakarta IO custom tag library for a variety of input and output tasks Commit times using JTA EJB activity at method level Network socket bandwidth Network socket bandwidth Number of active threads by class Time spent parsing XML document XML transformation time Processing time of custom JSP tags on on on on off off off off

on

off

JTACommitTracing EJBMethodLevelTracing SocketTracing UDPTracing ThreadTracing XMLSAXTracing XSLTTracing JSPTagLibraryTracing

on on on on on on on off

off off off off off off off off

Tracer Groups report information only when turned on (uncommented). Tracer Groups are activated with the keyword TurnOn. For more information about directives and tracers for custom PBDs, please see the Directive & Tracer Type Definitions document on the Wily community site.

38 ProbeBuilder Directives

Java Agent Guide

Setting toggles to gather additional metric information

The following toggles, when turned on, cause the collection of additional metrics, across all APIs, for Wily-provided tracer groups that are enabled. You must add these toggles to your full or typical toggle file to change the configuration. Name Definition Default full setting
on on off

Default typical setting

on off

DefaultStalledMethod Tracing Stalled method tracing DefaultConcurrent InvocationTracing DefaultRateMetrics Concurrent invocation information Invocation rate metrics

Turning Tracer Groups on or off

To turn a Tracer Group on: 1 Locate the toggles-full.pbd or toggles-typical.pbd file (depending on which file type (<appserver>-full.pbl or <appserver>-typical.pbl is in use by AutoProbe or Agent). These files are found within the <appserver home>/wily directory or <Introscope_Home>/config/systempbd directory. 2 Locate the Tracer Group to turn on, and uncomment the line by removing the pound sign from the beginning of the line. The directive in the following example is turned on, and will cause the tracing of all HTTP Servlets.
TurnOn: HTTPServletTracing

Note Any uncommented (turned on) directive for a Tracer Group causes the Tracer Group to be used. To turn a Tracer Group off:
Comment the Tracer Group by placing a pound sign at the beginning of the line,

as in the following example:

#TurnOn: HTTPServletTracing

Turning on InstrumentPoint directives

There are two types of directives identified by the keyword, InstrumentPoint: those that trace exceptions, and one that causes agent initialization when the application starts up (instead of when the first Probe is run).

Default ProbeBuilder Directives files 39

CA Wily Introscope

Exceptions
The following directives are used to turn on tracing of exceptions either where thrown or caught. They can cause performance degradation so they are not turned on by default. To turn either of these on, uncomment the appropriate line:
#InstrumentPoint: ThrowException #InstrumentPoint: CatchException

Agent Initialization
The Agent Initialization instrument point directive does not cause additional overhead and is turned on by default in both full and typical PBD sets.
#InstrumentPoint: AgentInitialization

If multiple ProbeBuilder Directive files are used, any settings (such as Tracer Groups, Skips, InstrumentPoints, Custom Method Tracers) turned on in any file take effect.

Adding classes to a Tracer Group

You can turn on tracing for a particular class by adding the class to an existing Tracer Group. To identify a class as being part of a Tracer Group, use one of the Identify keywords. For example, to add the class, com.myCo.ejbentity.myEJB1, to the Tracer Group, EntityBeanTracing:
IdentifyClassAs: com.myCo.ejbentity.myEJB1 EntityBeanTracing

The identify keywords are:

IdentifyInheritedAs IdentifyClassAs IdentifyCorbaAs

For details on the Identify keywords, see Directive keywords on page 51.

EJB subclass tracing

By default, entity and session EJB-related directives add probes only for EJBs that directly and explicitly implement the entity, session, or message-driven EJB interfaces. Often, an applications EJBs are subclasses of classes which directly and explicitly implement the entity or session EJB interface. These are not tracked by default by Introscope.

40 ProbeBuilder Directives

Java Agent Guide

For EJB subclasses to be tracked by Introscope, they must be added to the appropriate Tracer Group. To do this, add entries that refer to the direct ancestors of the EJB subclasses to be tracked. From these models, replace <entity.bean.ancestor.class> or <session.bean.ancestor.class> with the fully-qualified class name of the immediate ancestor of the EJBs to be Introscope-enabled. For entity EJBs:
IdentifyInheritedAs: <entity.bean.ancestor.class> EntityBeanTracing

For session EJBs:

IdentifyInheritedAs: <session.bean.ancestor.class> SessionBeanTracing

The examples below are based on this class hierarchy:

mySessionEJB implements javax.ejb.SessionBean mySessionEJBsubclass1 extends mySessionEJB mySessionEJBsubclass1a extends mySessionEJBsubclass1 mySessionEJBsubclass1b extends mySessionEJBsubclass1 mySessionEJBsubclass2 extends mySessionEJB

The Tracer Group, SessionBeanTracing, causes the tracking of mySessionEJB: The following tracer traces mySessionEJBsubclass1 and mySessionEJBsubclass2.

IdentifyInheritedAs: mySessionEJB SessionBeanTracing

The following tracer traces mySessionEJBsubclass1a and mySessionEJBsubclass1b.

IdentifyInheritedAs: mySessionEJBsubclass1 SessionBeanTracing

Note This example does not use packages. If your code is in a package, it needs to include the package name with the class name. See Directive keywords on page 51.

Using Blame Tracers to mark blame points

Introscopes Blame Technology works in a managed Java Application to enable you to view metrics at the application tiers: the front and backends of your application. This capability, referred to as boundary blame, allows users to triage problems to the application frontend or backend. For information about how Introscope determines frontends and backends, and about options for configuring URL Groups to control how metrics for frontends are aggregated, see Configuring Boundary Blame on page 115.

Using Blame Tracers to mark blame points 41

CA Wily Introscope

The following sections describe how you can use tracers to explicitly mark the frontends and backends in your application.

Blame Tracers
Introscope provides tracers for capturing front and backend metrics: FrontendMarker and BackendMarker. These tracers explicitly mark a frontend and backend, respectively. You can use FrontendMarker and BackendMarker to instrument your own code, for instance code that accesses a backend, to cause Introscope to capture and present metrics for custom components in the Investigator tree. If no FrontendMarker is configured, the first component in the blame stack will be the default frontend. In some environments this may not be desirable. For example, if your environment includes Introscope Browser Response Time Adapter (BRTA), by default the browser component will appear as the default frontend. In this case, you might configure servlets as frontends. If no BackendMarker is configured, Introscope will infer a backendany component that opens a client socket will be a default backend if none is explicitly marked. It is useful to use BackendMarker:
to assign a desired name to an item that Introscope detects as a backend to mark custom Java sockets that Introscope does not instrument for native sockets that are called through the Java Native Interface (JNI), to

identify a Java/JNI bridging method as the backend. FrontendMarker and BackendMarker are instances of BlamePointTracer which provides metrics such as average response time, per interval counts, concurrency, stalls, and errors for a blamed component. A BlamePointTracer can be applied to middle components for a more granular Blame Stack.

Blame Tracers in standard PBDs

Two of the standard PBDs provided with Introscope, j2ee.pbd and sqlagent.pbd, implement Boundary Blame Tracing.
HttpServletTracer in j2ee.pbd is an instance of FrontendMarker. SQLBackendMarker in sqlagent.pbd is an instance of BackendMarker.

The following Blame Tracers used in previous versions of Introscope still exist, but are not typically used in Introscope PBDs:
BlamedMethodTimer

42 ProbeBuilder Directives

Java Agent Guide

BlamedMethodRateTracer BlamedMethodTraceIncrementor BlamedMethodTraceDecrementor

Boundary Blame and Oracle backends

In the current version of Introscope, Oracle databases are not detected based on the socket connectionSQL Agent must be available for Introscope to automatically detect Oracle backends. To enable Introscope to detect Oracle backends in the absence of SQL Agent, make the following modification to oraclejdbc.pbd: In this portion of oraclejdbc.pbd:

#Socket data from the Oracle driver reports too many metrics SkipPackagePrefixForFlag: oracle.jdbc. SocketTracing SkipPackagePrefixForFlag: oracle.net. SocketTracing
comment out the skips, as shown below:

#Socket data from the Oracle driver reports too many metrics #SkipPackagePrefixForFlag: oracle.jdbc. SocketTracing #SkipPackagePrefixForFlag: oracle.net. SocketTracing

Creating Custom Tracers

Custom Tracers are constructed using a specific syntax and keywords. This section describes how to construct Custom Tracers. Choose methods to trace carefully. More methods traced means more overhead. Custom Tracers must be saved in a custom ProbeBuilder Directive file (.pbd). Introscope includes an example file with Custom Tracers in it, located in <Introscope_Home>/examples/CustomTracerExample.pbd. You can use this file as a template for your own custom PBD file.

Custom Tracers and keywords

To construct a Custom Tracer, you define:
which method or class to trace the information to collect about the method or class

There are four directives available for Custom Tracing:

TraceOneMethodOfClass traces a specified method in the specified class. TraceAllMethodsOfClasstraces all methods in the specified class.

Creating Custom Tracers 43

CA Wily Introscope

TraceOneMethodIfInheritstraces one method in all direct subclasses or

direct interface implementations of the specified class or interface.

TraceAllMethodsIfInheritstraces all methods in all direct subclasses or

direct interface implementations of the specified class or interface. Note Only concrete, implemented methods can be traced and report metric data while running. An abstract method specified in a Custom Tracer results in no metric data being reported. You must use a specific syntax when constructing Custom Tracers, or Introscope will not be able to recognize and process your tracers, and wont collect the desired information. This syntax is described in the next section.

Tracer syntax
A tracer is composed of a directive and information about the method or class to trace, in the following format:
<directive>: <class> <method> <Tracer-name> <metric-name>

Tracer component
<class>

Definition
A fully qualified class or interface name to trace. Fully qualified means it includes the full package of the class as well as the name, for example, com.mycompany.mypackage.MyClass. The simplest method specifier is the name of the method to trace. If just the name of the method is given, all instances of the method are traced, regardless of signature. If a signature is also given, only the instance with that signature will be traced. (See also Signature differentiation on page 48.)

<method>

44 ProbeBuilder Directives

Java Agent Guide

Tracer component
<Tracer-name>

Definition
Specifies how the method is to be traced. See the table below for descriptions of tracer names. Controls how the collected data is displayed in the Workstation. The following examples describe the three ways to specify the name and location of a metric at different levels of the metrics tree. metric-name The metric appears immediately inside the Java Agent. resource:metric-name The metric appears inside one Resource (folder) below the Java Agent. resource|sub-resource|sub-subresource:metric-name The metric appears more than one Resource (folder) level deep below the Agent, use pipe characters (|) to separate the Resources.

<metric-name>

The following table describes tracer names and what they trace. Tracer name What It Traces
Reports as a count the number of times a method has started but not yet finished. The result is reported under <metric-name> in the Investigator tree. An example use of this Tracer would be counting the number of simultaneous database queries Average CPU time (in milliseconds) used during method execution and reports it under <metricname> in the metrics tree.

ConcurrentInvocation Counter

MethodCPUTimer

Note: This tracer requires a platform monitor on

the supported platform (either AIX 5.2 or RedHat Enterprise Linux 3.0).

MethodNanoCPUTimer

Average CPU time (in nanoseconds) used during method execution and reports it under <metricname> in the metrics tree. This tracer provides nanosecond precision, but not necessarily nanosecond accuracy; it relies on the JVM to provide the current value of the most precise available system timer, in nanoseconds.

Note: This tracer requires Java 5.

Creating Custom Tracers 45

CA Wily Introscope

Tracer name

What It Traces
Average method execution time in nanoseconds, reported under <metric-name> in the metrics tree. This tracer provides nanosecond precision, but not necessarily nanosecond accuracy; it relies on the JVM to provide the current value of the most precise available system timer, in nanoseconds.

MethodNanoTimer

Note: This tracer requires Java 5.

MethodRateTracer

Number of invocations per second. An example use of this would be logins per second when tracing a method that represents a user logging into the system. It is reported under <metric-name> in the Investigator tree. Average method execution time in milliseconds and reports it under <metric-name> in the metrics tree Adds (or subtracts, respectively) one to the value of a counter specified by <metric-name> every time the metric is traced. These two metrics can be combined to trace complex metrics like number of connected users, when one method is called to create a new user session, and a second method is called to end a user session. Number of invocations per interval. This interval will change based on the view period of the consumer of the data (for example, the View pane in the Investigator). It is reported under <metric-name> in the Investigator tree. Dumps a stack trace to the instrumented application's stderr for methods to which it is applied. The exception stack trace thrown by the Dump Stack Tracer is not a true exceptionit is a mechanism for printing the method stack trace. This feature is useful for determining callpaths to a method.

MethodTimer MethodTraceIncrementor MethodTraceDecrementor

and

PerIntervalCounter

DumpStackTraceTracer

WARNING This feature imposes heavy system overhead. It is strongly recommended that this tracer only be used in a diagnostic context where a sharp increase in overhead is acceptable.

46 ProbeBuilder Directives

Java Agent Guide

Custom method tracer examples

The following are tracer examples found in the CustomTracerExample.pbd file. Quotes are used around the metric names in these examples because of spaces in the metric names.

Example average tracer

This tracer tracks the average execution time of the given method in milliseconds.
TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search BlamedMethodTimer "Petstore|Catalog|search:Average Method Invocation Time (ms)"

Example rate tracer

This tracer counts the number of times the method is called per second, and reports this rate under the specified metric name.
TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search BlamedMethodRateTracer "Petstore|Catalog|search:Method Invocations Per Second"

Example per interval counter tracer

This method tracer counts the number of times the method is called per interval, and reports the per interval count under the specified metric name. The interval is determined by the monitoring logic in the Enterprise Manager, such as the Graph frequency. The preview pane in the Investigator defaults to 15 second intervals.
TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search PerIntervalCounter "Petstore|Catalog|search:Method Invocations Per Interval"

Example counter tracer

This tracer counts the total number of times the method is called.
TraceOneMethodOfClass: com.sun.petstore.cart.ShoppingCart placeOrder BlamedMethodTraceIncrementor "Petstore|ShoppingCart|placeOrder:Total Order Count"

Creating Custom Tracers 47

CA Wily Introscope

Example combined counter tracers

These tracers combine incrementor and decrementor Tracers to keep a running count.
TraceOneMethodOfClass: com.sun.petstore.account.LoginEJB login MethodTraceIncrementor "Petstore|Account:Logged In Users" TraceOneMethodOfClass: com.sun.petstore.account.LogoutEJB logout MethodTraceDecrementor "Petstore|Account:Logged In Users"

Creating advanced Custom Tracers

This section describes advanced Custom Tracers:
Advanced single-metric tracers

Signature differentiation Metric-name-based parameters

Skips Combining custom tracers

Advanced single-metric tracers

Signature differentiation
Tracers can be applied to a method based on the method signature. To trace a single instance of a method with a specific signature, append the signature to the method name (including return type) specified using the internal method descriptor format. For example, myMethod(Ljava/lang/String;)V traces the instance of the method with a string argument and void return type. For complete information about this format, see the Sun Java Virtual Machine Specification, section 4.3.3, Method Descriptors, and section 4.3.2, Field Descriptors, http://java.sun.com/docs/books/vmspec/2nd-edition/html/ VMSpecTOC.doc.html.

Keyword-based substitution
Keyword-based substitution allows runtime substitution of values into the metric name.

48 ProbeBuilder Directives

Java Agent Guide

The parameters in the metric name in the tracer are substituted at runtime for the actual values into the metric name. This feature can be used with any directive. Parameter
{method} {classname} {packagename}

Runtime substitution
Name of the method being traced Runtime class name of the class being traced Runtime package name of the class being traced

{packageandclassname} Runtime package and class name of the class being traced

Note If Introscope processes a class which does not have a package, it will replace {packagename} with the string <Unnamed Package>.

Example 1:
If the metric name for a tracer in the pbd file is:

"{packagename}|{classname}|{method}:Response Time (ms)"

and the tracer is applied to method myMethod with a runtime class of myClass that is in package myPackage, the resulting metric name would be:

"myPackage|myClass|myMethod:Response Time (ms)"

Example 2:
If a tracer with a metric name in the .pbd file of

"{packageandclassname}|{method}:Response Time (ms)"

was applied to the same method, the resulting metric name would be

"myPackage.myClass|myMethod:Response Time(ms)"
Note Note the . between the package and class instead of the | in the first example.

Metric-name-based parameters
You can create a single-method Tracer that creates a metric name based on parameters passed to a method using the TraceOneMethodWithParametersOfClass keyword.

TraceOneMethodWithParametersOfClass: <class-name> <method> <tracer-name> <metric-name>

Creating advanced Custom Tracers 49

CA Wily Introscope

Parameters can be used in the metric name. This is accomplished by substituting the value of parameters for placeholder strings in the metric name. The placeholder strings to use are {#} where # is the index of the parameter to substitute. The indices start counting at zero. Any number of parameter substitutions can be used in any order. All parameters are converted to strings before substitution into the metric name. Object parameters other than strings should be used with caution because they are converted using the toString() method. If you are unclear about what string the parameter will be converted to, you should refrain from using it in the metric name.

Example:
A Web site uses a class named order, with a method named process. The method has parameters for different kinds of orders, either book or music. You can create a tracer like this:
TraceOneMethodWithParametersOfClass: order process(LJava/lang/string;)V MethodTimer Order|{0}Order:Average Response Time (ms)

This tracer produces metrics like these: Order BookOrder Average Response Time (ms) MusicOrder Average Response Time (ms) You can also use the TraceOneMethodWithParametersIfInherits keyword. For more information on both keywords, see Directive keywords on page 51.

Skips
Certain packages, classes, or methods can be skipped by AutoProbe or ProbeBuilder by using skip directives. By default, Introscope Agent and fundamental Java classes and packages are skipped by AutoProbe or ProbeBuilder. For more information, see ProbeBuilder Skip keywords on page 52.

50 ProbeBuilder Directives

Java Agent Guide

Combining custom tracers

You can use multiple tracers that affect the same metric, in effect combining them. This is most commonly used with incrementors and decrementors.

Example
Create tracers for the login and logout methods of the user class:
TraceOneMethodOfClass user login MethodTraceIncrementor Logged-in Users TraceOneMethodOfClass user logout MethodTraceDecrementor Logged-in Users

This increments the metric Logged-in Users when someone logs in and decrements Logged-in Users when someone logs out.

Counting object instances

The InstanceCounts Tracer Group counts the number of instances of the particular object types associated with it (for information on associating object types with the InstanceCounts Tracer Group using the standard IdentifyClassAs and IdentifyInheritedAs directives, see Adding classes to a Tracer Group on page 40). Any instances explicitly allocated in your code will be counted. Subtypes will also be counted. Objects created through different mechanisms, such as deserialization or cloning, might not be counted. Tracing using this Tracer Group could potentially incur incremental performance (and memory) impact, depending on the number of instances counted.

Directive keywords
This section contains a list of Introscope Directive keywords. Specifics for using these keywords with arguments to create directives are described in this section. Two tracers, StalledMethodTracer and ThresholdMethodTimer use their own keywords (see Threshold ProbeBuilder action keywords on page 55). The table below defines parameters used in theIntroscope Directive keyword syntax. Parameter
<package-name> <class-name>

Value
Period-separated, fully qualified package name, such as

java.lang.
Period-separated, fully qualified class name, such as

java.lang.Object.

Directive keywords 51

CA Wily Introscope

Parameter
<Tracer-name>

Value
Name of Tracer to use, one of the following in this list: MethodTimer PerIntervalCounter MethodRateTracer MethodTraceIncrementor MethodTraceDecrementor ConcurrentInvocationCounter BlamePointTracer FrontendMarker BackendMarker

<method> <name> <Tracer-group>

Method name with an optional descriptor, such as Main or

Main([Ljava/lang/String;)V.
Valid metric name or resource category depending on the specific tracer class, such as Resource|Sub-resource. Name of tracer group to use.

ProbeBuilder Skip keywords

The Skip keywords cause the ProbeBuilder to skip over a class or a package. SkipClass: <class-name> Prevents the listed class from being Introscope-enabled. Only the specific class is skipped. SkipPackage: <package-name> Prevents the listed package from being Introscope-enabled. All classes in the listed package are skipped. SkipMethodForClass: <class-name> <method> Allows a method in the class to be skipped. SkipMethodForFlag: <Tracer-group> <method> Allows a method in one or more classes to be skipped.

ProbeBuilder action keywords

The keywords below cause the ProbeBuilder to take action: IdentifyClassAs: <class-name> <Tracer-group> Adds the specific class to the specified Tracer Group.

52 ProbeBuilder Directives

Java Agent Guide

IdentifyCorbaAs: <Tracer-group> Adds all CORBA classes to the specified Tracer Group. IdentifyInheritedAs: <class-name> <Tracer-group> Adds all direct subclasses or interfaces to the specified Tracer Group. If a class is specified, it adds all direct subclasses. If an interface class name is specified, it adds all direct implementers of that interface. InstrumentPoint: <ThrowException> or <CatchException> ThrowException Track all exceptions at the location where they are thrown. High performance impact, turned off by default. CatchException Track all exceptions at the location where they are caught. High performance impact, turned off by default. TraceAllMethodsIfInherits: <class-name> <Tracer-name> <name> Traces all methods in direct subclasses or implementers of the named interface with the given tracer class and metric name. Only concrete, implemented methods are Introscope-enabled and report metric data while running. The result is a single aggregate metric for all methods traced. To get detail results for particular methods, use one of the TraceOneMethod keywords. An abstract method specified in a tracer results in no metric data being reported. TraceAllMethodsOfClass: <class-name> <Tracer-name> <name> Traces all methods in the named class with the given tracer class and metric name. Only concrete, implemented methods are Introscope-enabled and report metric data while running. The result is an aggregate metric for all methods traced. To get detailed results for particular methods, use one of the TraceOneMethod keywords. An abstract method specified in a tracer results in no metric data being reported. TraceOneMethodOfClass: <class-name> <method> <Tracer-name> <name> Traces the specific method in the named class with the given tracer class and metric name.

Directive keywords 53

CA Wily Introscope

Only concrete, implemented methods are Introscope-enabled and report metric data while running. An abstract method specified in a tracer results in no metric data being reported. TraceOneMethodWithParametersIfInherits: <class-name> <method> <Tracer-name> <name> Traces the specific method in direct subclasses or implementers of the named superclass or interface with the given tracer class and metric name. Only concrete, implemented methods are Introscope-enabled and report metric data while running. An abstract method specified in a tracer results in no metric data being reported. The specified method name must include a signature. The metric name can include strings like {#} that are substituted with the value of the parameter at index # (where 0 is the first parameter, 1 is the second parameter, etc). TraceOneMethodWithParametersOfClass: <class-name> <method> <Tracername> <name> Traces the specific method in the named class with the given tracer class and metric name. Only concrete, implemented methods are Introscope-enabled and report metric data while running. An abstract method specified in a tracer results in no metric data being reported. The specified method name must include a signature. The metric name can include strings like {#} that are substituted with the value of the parameter at index # (where 0 is the first parameter, 1 is the second parameter, etc). TurnOn: <Tracer-group> Enables the specified Tracer Group. There is no corresponding Directive to disable a Tracer Group. To disable a Tracer Group, comment out the associated TurnOn Directive.

54 ProbeBuilder Directives

Java Agent Guide

Threshold ProbeBuilder action keywords

The following directives, TraceOneMethodWithThresholdOfClass, and TraceAllMethodsWithThresholdOfClass, only work with the following tracers:
StalledMethodTracer OverThresholdPerIntervalCounter UnderThresholdPerIntervalCounter

Keyword syntax
The table below defines parameters used in the keyword syntax for these specific directives. Parameter
<class-name>

Value
Period-separated, fully qualified class name, such as

java.lang.Object.
<Tracer-name> Name of tracer to use, one of the following in this list: StalledMethodTracer ThresholdMethodTimer <method> <name> <threshold> Method name with an optional descriptor, such as Main or Main([Ljava/lang/String;)V). Valid metric name or resource category depending on the specific tracer class, such as Resource|Sub-resource. Length of threshold in milliseconds

Action keywords
TraceOneMethodWithThresholdOfClass: <class-name> <method> <Tracername> <name> <threshold> Traces the specific method in the named class with the given tracer class and metric name. Only concrete, implemented methods are Introscope-enabled and report metric data while running. An abstract method specified in a tracer results in no metric data being reported. TraceAllMethodsWithThresholdOfClass: <class-name> <Tracer-name> <name> <threshold> Traces all methods in the named class with the given tracer class and metric name.

Directive keywords 55

CA Wily Introscope

Only concrete, implemented methods are Introscope-enabled and report metric data while running. The result is an aggregate metric for all methods traced. To get detailed results for particular methods, use one of the TraceOneMethod keywords. An abstract method specified in a tracer results in no metric data being reported.

Threshold tracers
StalledMethodTracer traces the number of method invocations that have exceeded a user-defined execution threshold value and have not finished. This number serves as an approximation of the number of hung threads. The result is reported in the metrics tree using the fully qualified metric name specified in the directives file. If desired, users can set up Alerts on this metric. Over and Under Threshold Counter Tracers:
OverThresholdPerIntervalCounter, which traces the number of invocations

greater than a user-defined threshold.

UnderThresholdPerIntervalCounter, which traces the number of invocations

equal to or less than a user-defined threshold. Each of these tracers produces one metric each, which is displayed in the Investigator tree. The fully qualified metric name is specified in the directives file.

Applying new and changed ProbeBuilder Directives

This section describes how to apply new and changed ProbeBuilder Directives files in Introscope.

Copy in new ProbeBuilder Directives files

When you are ready to implement a custom Probebuilder Directive file, add it to the directory where the ProbeBuilder looks for .pbd files, which varies depending on whether you use an AutoProbe or manual ProbeBuilding method.

PBD file location for AutoProbe

AutoProbe looks for directives files in the directory that contains the IntroscopeAgent.profile file, and resolves filenames relative to this directory. 1 Copy custom directives files (PBD and/or PBL) into the <appserver home>/wily directory.

56 ProbeBuilder Directives

Java Agent Guide

2 Update the introscope.autoprobe.directivesFile property, in wily/ IntroscopeAgent.profile file, to include the names of any new directives files, separated by commas. In the following example, a directives file named petstore.pbd has been added:

introscope.autoprobe.directivesFile=default.pbl,petstore.pbd
Note Do not remove any ProbeBuilder list files (.pbl) already in the property.

PBD location for ProbeBuilder wizard or command line Probebuilder

Place custom directives files in the <Introscope_Home>/config/custompbd directory. Note The Command-line ProbeBuilder looks for any custom directives files in the same directory where the ProbeBuilder is run from, and resolves filenames relative to this directory.

Introscope-enabling with new and changed PBDs

For new or changed directives to take effect, your applications must be Introscope-enabled using the latest .pbds. This process varies depending on the ProbeBuilding method you use.

Using AutoProbe on JVM 5.0 systems

You can configure dynamic instrumentation, allowing changed PBDs to take effect without application or Agent restart. This enables you to perform PBD corrections, or perform triage-driven instrumentation without interrupting application service. For more information see Dynamic ProbeBuilding (JVM 1.5) on page 28.

Using AutoProbe on Pre-JVM 5.0 systems

New and changed ProbeBuilder Directive files or ProbeBuilder List files will take effect the next time the application server loads the application classes. The Introscope Agent automatically detects when changes have been made to directives specified by the introscope.autoprobe.directivesFile property and reloads the directives. As application classes are reloaded by the application server, they are re-Introscope-enabled in accordance with the latest ProbeBuilder Directives. If your managed applications are not running when you add or change directives, when you next start the applications, they will be instrumented using the updated directives.

Applying new and changed ProbeBuilder Directives 57

CA Wily Introscope

If your managed applications are running, you do not need to restart the application server to apply the new or changed directives, but it is necessary to load, or reload, the managed application classes. How you cause the classes to reload depends upon the application server you use. For instance, on SAP NetWeaver 6.40, a redeploy is sufficient. Other environments may require a production redeploy.

Using the ProbeBuilder Wizard

1 The Custom Directives screen will list the .pbd files you placed in the .pbd directory described in PBD location for ProbeBuilder wizard or command line Probebuilder on page 57. 2 Select the custom directives files to use. For more information on running ProbeBuilder Wizard, see Using the ProbeBuilder wizard on page 70.

Using the command-line ProbeBuilder

CA Wily recommends using the command-line ProbeBuilder as your last option for Introscope-enabling your latest .pbds. 1 Stop your managed application. 2 Run the command-line ProbeBuilder or the ProbeBuilder Wizard, supplying the custom .pbd and .pbl files in the command line. For more information on the Command Line ProbeBuilder, see Adding Probes to bytecode using ProbeBuilder from the command line on page 72). 3 Start the managed application. 4 If they are not already running, start the Enterprise Manager and the Workstation.

58 ProbeBuilder Directives

SECTION

AutoProbe for Application Servers

This chapter provides instructions for configuring AutoProbe for application servers. AutoProbe dynamically Introscope-enables all applications loaded by a specific application server. Before You Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 60 62 63 65 66 67 68

Configuring WebLogic Server to use AutoProbe Configuring WebSphere to Use AutoProbe . Configuring Sun ONE to use AutoProbe . Configuring Oracle 10g to use AutoProbe Configuring HTTP servlet tracing . Modifying Java2 Security Policy . . . . . . . . . Configuring WebSphere z/OS to Use AutoProbe

AutoProbe for Application Servers 59

CA Wily Introscope

Before You Start

This chapter details one of three ways to Introscope-enable your applications. The other two methods of Introscope-enabling applications supported by CA Wily are:
AutoProbe for Java Virtual Machines. For more information, see Configuring

JVM AutoProbe on page 19.

Manual ProbeBuilding. For more information, see Manual ProbeBuilding on

page 69. The instructions in this chapter assume that you have performed the following installation and configuration tasks:
Installing the Java Agent on page 16 Java Agent-to-Enterprise-Manager connection properties on page 17 Configuring the Java Agent name on page 18 Configuring ProbeBuilder options on page 18

WARNING Application Server AutoProbe is not supported on any JVM 1.5 and above platforms.

Configuring WebLogic Server to use AutoProbe

The following sections detail how to configure WebLogic installations to use AutoProbe to Introscope-enable applications. Use instructions for your specific version of WebLogic.

WebLogic Server 6.1

To configure WebLogic Server 6.1 to use AutoProbe: 1 Edit the Java classpath in the WebLogic Server startup script to include the wily/Agent.jar file. 2 Configure Tracer Groups to collect servlet data. For more information, see Configuring HTTP servlet tracing on page 67.

WebLogic Server 6.1 SP3

To configure WebLogic Server 6.1 SP3 to use AutoProbe: 1 Edit the Java classpath in the WebLogic Server startup script (for example, StartPetstore.cmd) to include the wily/Agent.jar file. 2 Set this property on the Java command line with the -D option, to activate Introscope AutoProbe:

60 AutoProbe for Application Servers

Java Agent Guide

-Dweblogic.classloader.preprocessor= com.wily.introscope.api.weblogic.PreProcessor
3 Configure Tracer Groups to collect servlet data. For more information, see Configuring HTTP servlet tracing on page 67.

WebLogic Server 7.0

To configure WebLogic Server 7.0 to use AutoProbe: 1 Edit the Java classpath in the WebLogic Server startup script (startWLS.cmd) to include the wily/Agent.jar file. 2 Set this property on the Java command line (in the same startup script startWLS.cmd) with the -D option, to activate Introscope AutoProbe:

-Dweblogic.classloader.preprocessor= com.wily.introscope.api.weblogic.PreProcessor
3 Configure Tracer Groups to collect servlet data. For more information, see Configuring HTTP servlet tracing on page 67.

WebLogic Server 8.1, 9.0, or 9.1

To configure WebLogic Server 8.1, 9.0, or 9.1 to use AutoProbe: 1 Edit the classpath in the application startup script (such as startMedRecServer.cmd) to include the wily/Agent.jar file. 2 Set the following property in the application startup script on the Java command line with the -D option to activate Introscope AutoProbe:

-Dweblogic.classloader.preprocessor= com.wily.introscope.api.weblogic.PreProcessor
3 Configure Tracer Groups to collect servlet data. For more information, see Configuring HTTP servlet tracing on page 67.

Configuring WebLogic Server to use AutoProbe 61

Java Agent Guide

Configuring WebSphere to Use AutoProbe

The following sections detail how to configure WebSphere installations to use AutoProbe to Introscope-enable applications. Use the instructions for your specific version of WebSphere.

WebSphere 4.0 (Distributed)

To configure WebSphere 4.0 (Distributed) to use AutoProbe: 1 Add the Agent.jar to the runtime extensions directory for WebSphere at <product_installation_root>/lib/ext. 2 In WebSphere, start the Administrators Console, and go to the JVM Settings section for the application server you want to modify. 3 Add the following two system properties:

Name: com.ibm.websphere.classloader.plugin Value: com.wily.introscope.api.websphere. WASAutoProbe Name: com.wily.introscope.agentProfile Value: <path-to-IntroscopeAgent.profile>

4 Apply the changes in the Administrators Console. 5 Restart the Web Application Server. 6 Configure Tracer Groups to collect servlet data. For more information, see Configuring HTTP servlet tracing on page 67.

WebSphere 6.0/5.1/5.0
To configure WebSphere 6.0/5.1/5.0 to use AutoProbe: 1 Add the Agent.jar to the runtime extensions directory for WebSphere at <product_installation_root>/lib/ext. 2 In WebSphere, start the Administrators Console and go to the JVM Settings section for the application server you want to modify:
For WebSphere 6.0, navigate to Application Servers > your_server > Java

and Process Management > Process Definition > Java Virtual Machine.
For WebSphere 5.0, navigate to Application Servers > your_server >

Process Definition > Java Virtual Machine. 3 Set the Generic JVM Arguments field to specify the classloader plug-in and the location of the IntroscopeAgent.profile file. You will set EITHER the com.wily.introscope.agentProfile, OR

Configuring WebSphere to Use AutoProbe 62

Java Agent Guide

com.wily.introscope.agentResource. The argument will then have the

following value (there are several properties set in one argument):

-Dcom.ibm.websphere.classloader.plugin=com.wily.introscope.api .websphere.WASAutoProbe -Dcom.wily.introscope.agentProfile=<path to IntroscopeAgent.profile>

OR

-Dcom.ibm.websphere.classloader.plugin=com.wily.introscope.api .websphere.WASAutoProbe -Dcom.wily.introscope.agentResource=<path to Resource containing

IntroscopeAgent.profile>

Note Although the examples shown break across lines, make sure that your argument does not have any breaks. 4 Apply the changes in the Administrators Console. 5 Restart the Web Application Server. 6 Configure Tracer Groups to collect servlet data. For more informaiton, see Configuring HTTP servlet tracing on page 67.

Java2 Security Policy

If you have Java2 Security enabled, you may need to add permissions to your Java2 Security Policy. For more information, see Modifying Java2 Security Policy on page 68.

Configuring WebSphere z/OS to Use AutoProbe

The following sections detail how to configure WebSphere on z/OS installations to use AutoProbe to Introscope-enable applications. Use instructions for your specific version of WebSphere on z/OS.

WebSphere 5.x and 6.0 for z/OS

To WebSphere 5.x and 6.0 for z/OS to use AutoProbe: 1 In WebSphere, start the Administrators Console, and go to the JVM Settings section for the application server you want to modify. 2 Select Application Servers > <your server> > Process Definition. 3 You should see two items, Control and Servant. Click Servant, then JavaVirtualMachine. 4 Set the Generic JVM Argument field to specify the classloader plug-in, and the location of the IntroscopeAgent.profile file. You will set EITHER the com.wily.introscope.agentProfile,

Configuring WebSphere z/OS to Use AutoProbe 63

Java Agent Guide

OR

com.wily.introscope.agentResource. The argument will then have the

following value (there are several properties set in one argument):
-Dcom.ibm.websphere.classloader.plugin=com.wily.introscope.api .websphere.WASAutoProbe -Dcom.wily.introscope.agentProfile=<path to IntroscopeAgent.profile> OR -Dcom.ibm.websphere.classloader.plugin=com.wily.introscope.api .websphere.WASAutoProbe -Dcom.wily.introscope.agentResource=<path to Resource containing IntroscopeAgent.profile>

5 Place the Agent.jar file in the <WebSphere Instance dir>/lib/ext directory. Note Do not place the Agent.jar file in the WebSphere installation directory. The following shows examples of the wrong and right directory:

NO: /usr/lpp/zWebSphere/V5R0M0/lib/ext YES: /WebSphere/V5R0M0/AppServer/lib/ext

6 The default for file encoding for WebSphere 5.x and 6.0 on z/OS is now ASCII, but Introscope expects EBCDIC file format. When you configure JVM AutoProbe for z/OS, click on Custom Properties and add the name value pair:
name: com.wily.introscope.default.encoding value: Cp1047

Note This step is fo z/OS operating systems only. 7 Confirm that all newly created Introscope files and directories within the ./wily directory are read-accessible by the WebSphere process. 8 Confirm that all *.log files (written by the Introscope Agent and ProbeBuilder) in the ./wily folder have write-access to the WebSphere process. These include:
all the Introscope files and directories the Introscope files inside <WAS instance dir>/lib/ext

9 Restart WebSphere application server. 10 When WebSphere says open for e-business, open the Administrators Console. Metrics should start reporting. 11 Configure Tracer Groups to collect servlet data. For more informaiton, see Configuring HTTP servlet tracing on page 67.

Java2 Security Policy

If you have Java2 Security enabled, you may need to add permissions to your Java2 Security Policy. For more information, see Modifying Java2 Security Policy on page 68.

Configuring WebSphere z/OS to Use AutoProbe 64

Java Agent Guide

Configuring Sun ONE to use AutoProbe

The following sections detail how to configure Sun ONE installations to use AutoProbe to Introscope-enable applications. Use instructions for your specific version of Sun ONE.

Sun ONE 7.0

To configure Sun ONE 7.0 to use AutoProbe: Note The use of ... in the .xml examples below indicates that there is additional information in the .xml code (not relevant to the example) that is not shown. 1 In order to add Introscope information to startup scripts for Sun ONE 7.0, you must be logged in as Administrator or Root. 2 Open the server.xml file, located at:

<Sun ONE install dir>/domains/domain1/server1/config/

Note The item separator is a colon (:). 3 Add the full path of wily/Agent.jar to the server-classpath property of the java-config element in the server.xml file. For example:
<java-config ... server-classpath="/sw/sun/sunone7/wily/Agent.jar:..." ...>

4 Add the following to the java-config element:

Add the bytecode-preprocessors property and set it to the value

com.wily.introscope.api.sun.appserver.SunONEAutoProbe.
For example:
<java-config ... bytecodepreprocessors="com.wily.introscope.api.sun.appserver.SunONEAutoProbe" >
Add a jvm-options element to define the location of the Agent profile. Define

either com.wily.introscope.agentProfile , or com.wily.introscope.agentResource. The following is an example of com.wily.introscope.agentProfile:

<java-config ...> ... <jvm-options>-Dcom.wily.introscope.agentProfile=/sw/sun/sunone7/wily/ IntroscopeAgent.profile </jvm-options> </java-config>

The following is an example of com.wily.introscope.agentResource:

<java-config ...>

Configuring Sun ONE to use AutoProbe 65

Java Agent Guide

... <jvm-options>-Dcom.wily.introscope.agentResource=<virtual path to>/ IntroscopeAgent.profile</jvm-options> </java-config>

5 OPTIONAL: If you configured com.wily.introscope.agentResource, add the resource file to the server classpath. 6 Configure Tracer Groups to collect servlet data. For more informaiton, see Configuring HTTP servlet tracing on page 67.

Configuring Oracle 10g to use AutoProbe

The following sections detail how to configure Oracle 10g installations to use AutoProbe to Introscope-enable applications. Use instructions for your specific version of Oracle 10g.

Oracle 10g 10.0.3

To configure Oracle 10g 10.0.3 to use AutoProbe: 1 Add Agent.jar to the application server classpath. 2 Set the system property

oracle.classpreprocessor.classes with the

value of com.wily.introscope.api.oracle.OracleAutoProbe. 3 Set the system property oracle.j2ee.class.preprocessing with the value of true. 4 Run this command at the command line:

-Dcom.wily.introscope.probebuilder.oracle.enable=true
5 Restart the Oracle Application Server 10g, using this command:

java -Doracle.classpreprocessor.classes=com.wily.introscope.api. oracle.OracleAutoProbe -Doracle.j2ee.class.preprocessing=true -Dcom.wily.introscope.probebuilder.oracle.enable=true -classpath oc4j.jar:<path to wily install dir>/wily/Agent.jar com.evermind.server.OC4JServer -config <path to oracle install

dir>/config/server.xml
6 Configure Tracer Groups to collect servlet data. For more informaiton, see Configuring HTTP servlet tracing, below.

Configuring Oracle 10g to use AutoProbe 66

Java Agent Guide

Configuring HTTP servlet tracing

Before you use AutoProbe with your application servers to Introscope-enable your applications, you must configure Tracer Groups in the toggles-full.pbd and toggles-typical.pbd files. This will enable servlet data to be collected. You will turn one Tracer Group off, and turn another Tracer Group on. To configure HTTP servlet tracing: 1 Navigate to the <your-app-server-home>/wily/toggles-full.pbd file and open it. 2 Go to the HTTP Servlets Configuration section of the PBD. 3 Turn off the HTTPServletTracing Tracer Group by placing a pound sign at the beginning of the line. For example:
#TurnOn: HTTPServletTracing

4 Turn on the HTTPAppServerAutoProbeServletTracing Tracer Group by removing the pound sign from the beginning of the line. For example:
TurnOn: HTTPAppServerAutoProbeServletTracing

5 Repeat steps 2-4 for <your-app-server-home>/wily/toggles-typical.pbd file.

Configuring HTTP servlet tracing 67

CA Wily Introscope

Modifying Java2 Security Policy

If you have Java2 Security enabled, you may need to add the following permissions to your Java2 Security Policy. To add permissions to your Java2 Security Policy
Edit the file <WebSphere home>/properties/server.policy to include the

following:
// permissions for Introscope AutoProbe grant codeBase "file:${was.install.root}/-" { permission java.io.FilePermission "${was.install.root}${/ }wily${/}-", "read"; permission java.net.SocketPermission "*", "connect,resolve"; permission java.lang.RuntimePermission "setIO"; permission java.lang.RuntimePermission "getClassLoader"; permission java.lang.RuntimePermission "modifyThread"; permission java.lang.RuntimePermission "modifyThreadGroup"; permission java.lang.RuntimePermission "loadLibrary.*"; permission java.lang.RuntimePermission "accessClassInPackage.*"; permission java.lang.RuntimePermission "accessDeclaredMembers"; }; grant { permission java.util.PropertyPermission "*", "read,write"; };

Note Line breaks are shown for user readability and are not needed when adding the permissions to the server.policy file.

68 AutoProbe for Application Servers

SECTION

Manual ProbeBuilding

This chapter provides instructions for manually Introscope-enabling your applications. Manual ProbeBuilding is a non-dynamic method of Introscopeenabling your classes. When you run ProbeBuilder manually, it Introscope-enables classes on disk before the application server is run. You can use Manual ProbeBuilder when your environment does not support AutoProbe, or you prefer not to use AutoProbe. WARNING Introscope supports two other methods of instrumenting your applications. CA Wily recommends you use these other methods before using manual ProbeBuilding. Manual ProbeBuilding should not be used with the other methods of instrumentation, and should be used as a last resort. For more information, see Configuring ProbeBuilder options on page 18 and AutoProbe for Application Servers on page 59. Follow the steps in this chapter to manually Introscope-enable your applications. Before you begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 70 72 72 75 75 76

Using the ProbeBuilder wizard .

Update application startup script . Running Introscope-enabled code. ProbeBuilder Wizard.lax . . . .

Using ProbeBuilder from the command line.

Switching back to non-Introscope-enabled code .

Manual ProbeBuilding 69

CA Wily Introscope

Before you begin

The instructions in this chapter assume that you have performed these installation and configuration tasks:
Installing the Java Agent Java Agent-to-Enterprise-Manager connection properties Configuring the Java Agent name Configuring ProbeBuilder options

WARNING Introscope supports two other methods of instrumenting your applications. CA Wily recommends you use these other methods before using manual ProbeBuilding. Manual ProbeBuilding should not be used with the other methods of instrumentation, and should be used as a last resort. For more information, see Configuring ProbeBuilder options on page 18 and AutoProbe for Application Servers on page 59.

Manual ProbeBuilding options

There are two ways to Introscope-enable your bytecode manually:
ProbeBuilder Wizarda GUI dialog for running ProbeBuilder. Follow the

instructions in Using the ProbeBuilder wizard, below.

Command-line ProbeBuilderA command-line interface for environments

without a windowing system. Follow the instructions in Using ProbeBuilder from the command line, below.

Using the ProbeBuilder wizard

If your machine has a windowing environment, use the instructions in this section to add probes to your bytecode. These instructions assume that you have:
Installed the Introscope agent. Selected the the ProbeBuilder Wizard option from the Enterprise Manager

installation process.
Configured basic Agent settings as described in Installing and Configuring the

Java Agent on page 15. 1 If you have custom .pbds, add them to the <Introscope

home>/config/

custompbd directory.
2 Launch the ProbeBuilder Wizard from your <IntroscopeHome> directory. 3 On the Welcome screen, Click Next.

70 Manual ProbeBuilding

Java Agent Guide

4 Enter or browse to your bytecode directory and click Next. Note You can also select .jar files or individual .class files. 5 Click Select Java Bytecode to enter your desired directory and click Next. 6 Enter the name and location for the new directory to contain the Introscopeenabled code, or click Browse to select a location. The default name is the original directory with the suffix isc. Note If you select a directory that already exists, ProbeBuilder Wizard will display the following dialog:

Click Overwrite to overwrite existing files as necessary, or click Cancel to go back to the previous dialog to select another location. 7 Click Next if you didnt overwrite an existing directory. 8 In the System Directives window, locate the set of system Directives which correspond to your environment (if your application server isnt listed, use the Default Java selection). Note that you now have a choice of either using system directives files in which most Tracer Groups are turned on (FULL) or only a subset of Tracer Groups are turned on (TYPICAL). For more information on full and typical system directives files and the what kind of information they provide, see Full and typical ProbeBuilder Directives sets on page 36. 9 Click Next. If you installed custom directives files in your config/custompbd directory, they appear in the Custom Directives window. Check the box next to any custom directives that you want to use with this bytecode. Note For information on creating custom directives, see ProbeBuilder Directives on page 33. 10 Check the box for any custom .pbd files you want to use, then click Add Probes. 11 Introscope adds Probes to the specified bytecode. This operation may take several minutes. 12 When the Finished window opens, click Exit or Add Additional Probes to add Probes to another directory.

Using the ProbeBuilder wizard 71

Java Agent Guide

Update application startup script

After adding Probes to the bytecode, update the application startup script to specify the location of the Introscope-enabled code and the Agent. 1 Edit the classpath of the application startup script to include locations of the directories containing the Introscope-enabled code created with the ProbeBuilder. Make sure this reference precedes the reference to the original code in the classpath. 2 Edit the classpath in the application startup script to include the path to the <Introscope home>/lib/Agent.jar. For example, you might edit a classpath that looks like this:
<your-applicationpath>/classes:/<your-applicationpath>/lib/app.jar MainClass

to look like this:

<your-applicationpath>.isc/classes:/<your-applicationpath>.isc/lib/ app.jar:/Introscope/lib/Agent.jar MainClass

3 Save the changes. 4 Start your application with the new startup script.

Using ProbeBuilder from the command line

If your machine does not have a windowing environment, use the instructions in this section to add probes to your bytecode. There are three main steps required to install and configure the Introscope Agent to report information to the Enterprise Manager:
install the Introscope Agent Introscope-enable the application code using ProbeBuilder from the command

line
configure basic Introscope Agent settings

Adding Probes to bytecode using ProbeBuilder from the command line

ProbeBuilder can be activated by a command. The following code is an example of the command. This example is being run from the /wily directory, so paths are relative to the /wily directory. The syntax for the Java command depends on your Java version and other settings in your environment. Note: If you are processing a very large .jar file, you may need to increase the
memory in your JVM memory settings. Consult your JVM documentation for instructions.

Update application startup script 72

Java Agent Guide

The ProbeBuilder classpath varies, depending on whether you installed it using an agent installer or the full Introscope installer. This is an example of invoking ProbeBuilder from the agent directory:
java -cp ext/ProbeBuilder.jar com.wily.introscope.api.IntroscopeProbeBuilder -directives default.pbl,stream.pbd -origdir /usr/myApp/classes -destdir /usr/myApp/classes.isc verbose

This is an example of invoking ProbeBuilder from the Enterprise Manager directory:

java -cp lib/ProbeBuilder.jar com.wily.introscope.api.IntroscopeProbeBuilder -directives default.pbl,stream.pbd -origdir /usr/myApp/classes -destdir /usr/myApp/classes.isc verbose

These are the command line ProbeBuilder commands Option

-help -h -? -directives (or -pbd) comma-separated-filelist

Result
Displays a help screen REQUIRED: a comma-separated list of ProbeBuilder Directives files (.pbd), ProbeBuilder list files (.pbl), or directories to scan. Select either a full or typical .pbl file, depending on how much information you want gathered (see Full or typical tracing options on page 28). If you used the default Agent installer, you will see all possible default PBD and PBL files in the \wily directory. However, if you used an application-server specific Agent installer, you will only see PBD and PBL files specific to that application server.

Select one set of the next three pairs to specify your original code location and your Introscope-enabled code destination, using either directories, jar files, or classes. -origdir directory -destdir directory -origjar file -destjar file -origclass file -destclass file Specifies the original directory (including subdirectories) Specifies the destination directory Specifies the original archive, including .jar, .zip, .war, .rar and .ear archives Specifies the destination archive Specifies the original class Specifies the destination class

Using ProbeBuilder from the command line 73

Java Agent Guide

Option
-skipitems

Result
Skips any files that are not Java bytecode files or archive files (.jar and .zip files). If -skipitems is not set, the non-Java bytecode files are copied to the destination, unchanged. Turns on an interactive text UI when problems arise.

-prompt

Editing the classpath

After adding Probes to the bytecode, update the classpath of the application startup script to reflect the locations of the Introscope-enabled code and the Agent. 1 Edit the classpath of the application startup script to include locations of the directories containing the Introscope-enabled code created with the ProbeBuilder. Make sure this reference precedes the reference to the original code in the classpath. 2 Edit the classpath in the application startup script to include the path to the Agent.jar file. For example, you might edit a classpath that looks like this:
<your-applicationpath>/lib/app.jar MainClass

to look like this:

<your-applicationpath>.isc/lib/app.jar:/<appserver home>/wily/Agent.jar MainClass

3 Save the changes. 4 Start your application with the new startup script.

Using ProbeBuilder from the command line 74

Java Agent Guide

Running Introscope-enabled code

There are three ways to point to Introscope-enabled code instead of your original code:
In classpaths, replace original class paths with Introscope-enabled code paths.

The instructions in this chapter directed you to perform this process when you Introscope-enabled your application for the first time.
Prepend paths to classpaths.

If only part of the applications code was Introscope-enabled, you could place the Introscope-enabled code paths before the original-code paths (prepend) in the classpath. If you do this, Introscope-enabled code loads and reports performance data. Non-Introscope-enabled code still loads and works normally, but does not report performance data.
Place Introscope-enabled code in original classpath.

Use this method when classpaths are set in many places, or to conduct an evaluation. Be careful using this method in a production environment, when with this method it is easy to forget whether you are using the original or the Introscope-enabled code.

Move the original code to a new location. Leave the classpaths unchanged. Then move the Introscope-enabled code to the original location. On a UNIX machine, you could also create a symbolic link from the current location of the Introscope-enabled code to the original location.

Switching back to non-Introscope-enabled code

If you want to switch back to using non-Introscope-enabled code, undo the steps of modifying classpaths to run Introscope-enabled code, as described below:
If you put the paths to your Introscope-enabled code into the Java classpaths,

then replace paths to the Introscope-enabled code in Java classpaths with original paths.
If you added paths to the Introscope-enabled code in front of the paths to the

original code, remove prepended paths to classpaths Remove the prepended portion of the classpath so that only the original classpath remains.
If you put Introscope-enabled code in the original classpath, then remove the

Introscope-enabled code from the original path and place the original code in the original classpath.

Running Introscope-enabled code 75

CA Wily Introscope

If you used symbolic links on a UNIX system, point the symbolic link to the original directory or remove the link and move the code into the original classpath.

ProbeBuilder Wizard.lax
File: <Introscope home>/Introscope ProbeBuilder Wizard.lax

lax.nl.current.vm
VM to use the next time the ProbeBuilder is started. Can be set to any installed JDK or Default: JRE version 1.3. Varies by operating system

lax.stderr.redirect
Standard Error Output. Leave blank for no output, console to send to a console window, or any path to a file to save to the file. Default: blank

lax.stdin.redirect
Standard Input. Leave blank for no input, console to read from the console window, or any path to a file to read from that file. Default: blank

lax.stdout.redirect
Standard Output. Leave blank for no output, console to send to a console window, or any path to a file to save to the file. Default: blank

76 Manual ProbeBuilding

SECTION

II

Java Agent Operations and Management

The chapters in this section have information about Java Agent administration and operations.
Configuring Agent HTTP Tunneling on page 79 Java Agent Naming on page 83 Agent Monitoring and Logging on page 95 Using Virtual Agents to Aggregate Metrics on page 101 Configuring Agent Failover on page 105

Java Agent Operations and Management 77

CA Wily Introscope Java Agent

78 Java Agent Operations and Management

SECTION

Configuring Agent HTTP Tunneling

This chapter describes how to configure Introscope Agents for HTTP tunneling. This feature enables Agents at a remote site to connect securely to an Enterprise Manager that is managed elsewherefor example, offshore. Configuring Agent HTTP tunneling . . . . . . . . . . . . . . . . . . . . 80 80

Configuring a proxy server for Agent HTTP tunneling

Configuring Agent HTTP Tunneling 79

CA Wily Introscope

Configuring Agent HTTP tunneling

You can configure Agents to tunnel communication with an Enterprise Manager over HTTP, allowing communication to pass through firewalls permitting only HTTP traffic. Note Wily Technology does not recommend configuring Agents to tunnel over HTTP if a direct socket connection to the Enterprise Manager is feasible, because tunneling imposes additional CPU and memory overhead on the managed host and Enterprise Manager beyond that expected for a direct socket connection. To use Agent tunneling, configure the Agent to connect to the Enterprise Manager's port where the embedded web server is hosted, using an HTTP tunneling socket factory. Use these IntroscopeAgent.profile properties to specify the Agent tunneling connection:
the host name of machine running the Enterprise Managersee

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT on page 169. The default for this property is localhost.

the connection port to the Enterprise Manager Web serversee

introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT on page 169. The default for this property is 8081.

the HTTP tunneling socket factorysee

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT on page 169. The default for this property is

com.wily.isengard.postofficehub.link.net.HttpTunnelingSocketFa ctory.

Configuring a proxy server for Agent HTTP tunneling

You can configure the HTTP tunneled Agent to connect through a proxy server to the Enterprise Manager. This is necessary for a forward-proxy server configuration where the Agent is running behind a firewall that only allows outbound HTTP traffic routed through the proxy server. These proxy server configuration properties apply only if the Agent is configured to tunnel over HTTP. The proxy server configuration applies to any configured HTTP tunneled connection on the Agent, not to a single connection. This is especially important to consider when configuring failover between multiple Enterprise Managers, where the connection to each Enterprise Manager is over HTTP. Important If you are using HTTP tunneling, your proxy server must support HTTP Post.

80 Configuring Agent HTTP Tunneling

Java Agent Guide

To configure Agent HTTP tunneling: 1 Configure an agent connection to the Enterprise Manager by setting the host name, connection port, and socket factory properties:
introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT on page 169 introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT on page 169 introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT on

page 169 2 Specify the proxy server properties:

introscope.agent.enterprisemanager.transport.http.proxy.host on page 182 introscope.agent.enterprisemanager.transport.http.proxy.port on page 183

3 If the proxy server requires the agent to authenticate with it, set these properties:
introscope.agent.enterprisemanager.transport.http.proxy.username on

page 183
introscope.agent.enterprisemanager.transport.http.proxy.password on

page 183

Configuring a proxy server for Agent HTTP tunneling 81

CA Wily Introscope

82 Configuring Agent HTTP Tunneling

SECTION

Java Agent Naming

This chapter has information about Agent naming, related environmental and deployment considerations, and options for automatically naming your Agents. Understanding the Java Agent name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 88 88 89 91 92

Specifying an Agent name using a Java system property Specifying an Agent name using a system property key Obtaining an Agent name from the application server . Advanced automatic Agent naming options. . . . .

Enabling cloned Agent naming in clustered environments .

Java Agent Naming 83

CA Wily Introscope

Understanding the Java Agent name

This section explains alternatives for assigning a name to the Java Agents in your Introscope environment.

Aspects of the Java Agent name

Each Java Agent running in your Introscope environment has a name, whether you assigned it one explicitly, configured a method of automatically assigning it a name, or simply started up an Introscope-enabled application that the Java Agent monitors. The Java Agent name is important because it is central to many views and presentations in the Introscope clients, and it is key to the process of associating monitoring logic with target applications. The Investigator tree in the Workstation and WebView has a node for an Java Agent when your Introscope-enabled application is up and has started reporting metrics to the Agent. The Investigator tree on the following page shows Agents named domain1//myserver and domain2//myserver, running on host dwdemo12 under the WebLogic process.

84 Java Agent Naming

Java Agent Guide

When you configure management logic in the Workstationfor instance, Dashboards, Alerts, and Actionsthe Agent name is a component in the regular expressions you define that identify the applications to which the management logic applies.
SuperDomain Custom Metric Host Custom Metric Process Custom Metric Agents

Host Machine Process Agent

User-defined Domains

Understanding the Java Agent name 85

CA Wily Introscope

How the Agent determines its name

The Introscope Agent uses the following sequence to determine its name. If it finds a name using the first method, it accepts that name and connects to the Enterprise Manager. If it doesnt find a name using the first method, it tries the second method, and so on. If it doesnt find a name using any method, it calls itself Unknown Agent. 1 Name specified in Java system property The Agent name is defined using a Java system property on the command line. Using this method will override any other Agent naming method. See Specifying an Agent name using a Java system property on page 88. 2 Name specified in System Property Key in Agent Profile The Agent name is obtained from a Java system property specified in a property in the IntroscopeAgent.profile. See Specifying an Agent name using a system property key on page 88. 3 Name obtained automatically from Application Server If you use certain versions of WebLogic or WebSphere, the Agent name can be automatically obtained from the application server using Automatic Agent Naming functionality. You can configure a time delay, to give the Agent as much time as necessary to determine its name before connecting to the Enterprise Manager. See Obtaining an Agent name from the application server on page 89. 4 Name specified explicitly in Agent profile The Agent name is defined in the IntroscopeAgent.profile, in the property introscope.agent.agentName. This was the standard method for naming Agents in early Introscope versions. Use this option if you already have an Agent profile for every application. For more information, see Configuring the Java Agent name on page 18.

How Introscope resolves Agent naming conflicts

The fully qualified Agent namecomprised of host name, process name and Agent nameis typically unique to each Agent in an Introscope environment. Agents with the same Agent name usually have a unique fully-qualified Agent name because their host name and process names are likely to be different. Multiple Agents will have the same fully-qualified Agent name only if they reside on the same host, monitor the same process, and have the same Agent name.

86 Java Agent Naming

Java Agent Guide

If an Agent tries to connect to an Enterprise Manager to which an Agent with the same fully-qualified Agent name is already connected, the Enterprise Manager appends a unique identifier to the name of the newly connecting Agent. The identifier consists of a percent (%) character and a digit. This mechanism ensures that multiple Agents that connect using the same fully-qualified name can be uniquely identified for the duration of the connection. The Enterprise Manager renames the first duplicate Agent to connect by appending %1 to its Agent name. For instance, assume that two Agents with the fully qualified Agent name:
hostPA|processNIM|PodAgent

connect to the Enterprise Manager, one after the other. The Enterprise Manager renames the second Agent:
PodAgent%1

If other Agents with the same fully qualified name connect, they are renamed, in succession, PodAgent%2, PodAgent%3, PodAgent%4, and so on, where the digit following the percent character is the next number in sequence. When a renamed agent disconnects, the suffix it was assigned can be re-used. For example, if PodAgent%1 disconnects while PodAgent remains connected, the next Agent with the fully qualified name hostPA|processNIM|PodAgent to connect will be renamed PodAgent%1. Reuse of suffix identifier makes it possible that the Enterprise Manager might assign the same suffix to a particular Agents name from connection to connection. However, on subsequent connections, a given Agent could just as well be renamed differently. Having an Agents name vary from connection to connection is problematic when querying historical datait is preferable to configure a naming strategy that avoids the Enterprise Manager renaming Agents.

Agent naming considerations for clustered applications

If you run multiple instances of the same application, Introscope attempts to resolve identical Agent names, including custom metric Agents, by appending the Agent name with a character and a random number. Wily recommends, however, that you tell Introscope how to resolve the naming. The options for resolving identical Agent naming are:
Tell Introscope that the Agents in question are cloned Agents by enabling

cloned Agent naming (described in Enabling cloned Agent naming in clustered

Agent naming considerations for clustered applications 87

CA Wily Introscope

environments on page 92.)

Define unique Agent names yourself and make separate Agent profiles for each

Agent (described in Configuring unique names for application instances on page 93.)
Let Introscope uniquely name each Agent using its own naming scheme

(described in How Introscope resolves Agent naming conflicts on page 86.)

Specifying an Agent name using a Java system property

To specify an Agent name using Java system property:
On the Java command line, supply the desired name using this property:

com.wily.introscope.agent.agentName

Specifying an Agent name using a system property key

This method is the second the Agent uses to look for its name. Use this method if you want the Agent to be named from the value of an existing Java system property in your deployment. To specify an Agent name using the System Property Key: 1 Open the IntroscopeAgent.profile. 2 Under the Agent Name section, specify the Java system property that will provide the Agent name in this property:
introscope.agent.agentNameSystemPropertyKey

Note If the Java system property specified here doesnt exist, this property will be ignored. 3 Restart the application server.

88 Java Agent Naming

Java Agent Guide

Obtaining an Agent name from the application server

You can configure the Agent to extract the application server instance name automatically from the application server, and use that information to name itself. This eliminates the need to configure individual Agent names in a separate Agent profile file. The Agent can also rename itself if there are changes in the application server environment. This enables you to deploy an Agent profile across a large number of environments that might consist of a mix of application server platforms.

Application servers that support Agent naming

Automatic Agent Naming is supported when you use Introscope with these supported application server versions:
WebLogic 6.1 WebLogic 7.0 WebLogic 8.1 WebLogic 9.x WebSphere 6.0.x distributed WebSphere 6.1.x distributed WebSphere 5.0.x distributed WebSphere 5.1 distributed

The name of the application server displayed in the Introscope Workstation is determined by a Java J2EE API. This sometimes causes the name of the application servers to display differently in the Workstation because all application servers implement the API differently. The names of multiple application servers may be formatted differently in the Workstation, and even the same application server name may be formatted differently from release to release.

How automatic Agent naming works

When Automatic Agent Naming is enabled, the Agent starts, and looks for name information from the application server. The Agent waits until an Agent name is obtained before attempting to connect to the Enterprise Manager. When the Agent locates naming information, Introscope edits the information to make the Agent name compliant with Introscope Agent naming rules. Agent names on supported application servers are comprised of several pieces of information, which differ according to application server.
For WebLogic, the Agent name is comprised of:

Domain (data center) + cluster + instance (of WLS)

For WebSphere, the Agent name is comprised of:

Obtaining an Agent name from the application server 89

CA Wily Introscope

cell (domain) + process (instance of WAS) When information is obtained, segments are separated by forward slashesfor example:
medrec/MyCluster/MedRecServer

Any forward slashes in the segment name are converted to underscores. For example, if a Domain is named Petstore/West, it will be converted to Petstore_West. Note: When constructing the Agent name that appears in Introscope, Introscope
edits the information to make the Agent name compliant with Introscope Agent naming rules:
characters such as pipes, colons, or percentage signs are replaced by

underscores
names that begin with any character other than a letter will have the letter A

prepended to them
empty names are replaced by UnnamedAgent (so as to be distinguishable

from the UnknownAgent condition)

Automatic Java Agent naming and renamed agents

Using Automatic Agent Naming, the Agent always tries to obtain the most current application-server-specific Agent name. The Agent periodically checks for a new name. If a change to application server configuration results in an Agent name change, the Agent automatically renames itself. In the Investigator tree, the Agent appears to disconnect. The disconnected Agent remains in the Investigator tree, and unmounts automatically after the unmount time period has elapsed, or can be unmounted manually. The renamed Agent reconnects to the Enterprise Manager and appears in the Investigator tree. The Agent logs these changes. See Advanced automatic Agent naming options on page 91, for information on configuring Automatic Agent Naming properties for Enterprise Manager connection delay, and rename checking interval time.

Enabling automatic Agent naming

To enable Automatic Agent naming: 1 In the Java Agent profile, make sure that

introscope.agent.agentAutoNamingEnabled is set to true

2 Make these application server-specific changes:

90 Java Agent Naming

Java Agent Guide

For WebLogic, create an Introscope Startup Class. See Configuring startup

class for WebLogic 8.1 or 9.0 on page 112.

For WebSphere, create an Introscope Custom Service. See Configuring a

custom service in WebSphere 5.0 or 6.0 on page 113.

Advanced automatic Agent naming options

The You can change these Automatic Agent Naming configurations if appropriate. Naming options
Initial Enterprise Manager Connection Delay

How to change
When using the Automatic Agent Naming feature, the Agent waits up to a configurable amount of time before connecting to the Enterprise Manager while trying to find Agent name information. The default delay is 120 seconds.
To change the delay value:

1 Open the IntroscopeAgent.profile. 2 Under the Agent Name section, configure the desired delay in the property

3 Restart the application server.

Agent Rename Check Interval

introscope.agent.agentAutoNamingMaximumConnectionDelayIn Seconds.

When using the Automatic Agent Naming feature, the Agent periodically checks to see if the naming information from the application server has changed. The default interval is ten minutes.
To change this interval:

1 Open the IntroscopeAgent.profile. 2 Under the Agent Name section, configure the desired interval in the

introscope.agent.agentAutoRenamingIntervalInMinutes
property.

3 Restart the application server.

Turning Off Agent Log File Automatic Naming By default, when the Agent name is found automatically, either by information provided by a Java system property or application server, the log files associated with that Agent are named automatically using that same information. However, you can turn off this automatic log naming, and continue to use the Agent log name specified in the IntroscopeAgent.profile.
To turn off Agent log file automatic naming:

1 Open the Agent profile, usually IntroscopeAgent.profile. 2 Locate the property,

3 Restart the application server.

introscope.agent.disableLogFileAutoNaming, and enter a value of true

Advanced automatic Agent naming options 91

CA Wily Introscope

Disabling Agent naming for WebSphere

Agent automatic naming is enabled by default for all WebSphere platforms, but is not supported in WebSphere 4.0, or in WebSphere 5.x and 6.0 for z/OS. If you use one of these WebSphere versions, you must disable the Automatic Agent Naming feature. 1 Open the Agent profile, usually IntroscopeAgent.profile. 2 Set the value of the property introscope.agent.agentAutoNamingEnabled, to false.

Enabling cloned Agent naming in clustered environments

If two Agents exist with the same name monitoring the same host and process and are not uniquely named by a user, the name is appended with a number. Cloned Agent Naming enables you to correlate an Agent with a particular application instance in a clustered application. You are running cloned Agents if you:
are running Agents that share a host, process and Java Agent name with one

or more other Agents, or

are running two or more Agents that are using the same Agent profile

Cloned Agent naming scenario

If you have four Agents, all named AgentX, the Enterprise Manager might name them AgentX, AgentX%1, AgentX%2 and AgentX%3. If AgentX%1 disconnects and then reconnects, it is renamed another random number when it reconnects for example, AgentX%845. With the Agent cloning property turned on, in this same situation, the Agents in this example would be named AgentX-1, AgentX-2, AgentX-3 and AgentX-4. If Agents AgentX-2 and AgentX-3 disconnected, and another Agent with the same name (AgentX) came back on, it would be given the name AgentX-2. With this naming, you will never have more Agent names in the database than the number of Agents originally cloned.

Enabling cloned Agent naming in the Agent profile

To enable cloned Agent naming: 1 Open the Agent profile, IntroscopeAgent.profile. 2 Set this property:

92 Java Agent Naming

Java Agent Guide

introscope.agent.clonedAgent=true
3 Restart your managed application.

Configuring unique names for application instances

If you monitor multiple instances of the an application on the same machine, you can configure unique Agent names explicitly: 1 Create a separate Agent profile for each application. 2 Uniquely name each Agent in the Agent profile. 3 Specify which Agent profile each application should use.

Enabling cloned Agent naming in clustered environments 93

CA Wily Introscope

94 Java Agent Naming

SECTION

Agent Monitoring and Logging

This chapter has information about Agent configuration options that relate to Agent operations and monitoring. Configuring Java Agent connection metrics . Turning off socket Metrics in the Agent profile Configuring Agent logging options Managing the ProbeBuilder Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 96 97 99

Agent Monitoring and Logging 95

CA Wily Introscope

Configuring Java Agent connection metrics

By default, Introscope generates metrics on the connect/disconnect status of the Agents connected to an Enterprise Manager. You can monitor Agent connection metrics to determine the current state of the connection between an Agent and the Enterprise Manager. Java Agent connection metrics appear in the Investigator under the Enterprise Manager process in the Workstation (the custom metric host).
Custom Metric Host (Virtual) \ Custom Metric Agent (Virtual) \ Agents \ [Host_Name] \ [Agent Process Name] \ [Agent_Name] \ Connected

The Connected metric can have these values:

0No data about the Agent is available 1Agent is connected 2Agent is slow to report 3Agent is disconnected

An Agent disconnect also generates a Whats Interesting event. As with other events, users can query for Agent disconnects using the historical query interface. Agent disconnect events form a portion of the data used in assessing application health in the Overview tab for Agents and Applications in the Investigator. Once an Agent disconnects, Introscope continues to generate not connected state metrics until the Agent is timed out, as specified by the

introscope.enterprisemanager.agentconnection.metrics.agentTimeou tInMinutes property, after which no additional connection metrics are

generated. For information about Enterprise Manager properties, see the Introscope Configuration and Administration Guide.

Turning off socket Metrics in the Agent profile

Metrics that trace per-socket bandwidth have a potential for high overhead, and can be turned off if you notice that collecting network metrics is consuming a lot of processor or I/O time. To turn off the reporting of server and client socket input and output metrics:
In the IntroscopeAgent.profile, for the property,

introscope.agent.sockets.reportRateMetrics, enter a value of false.

96 Agent Monitoring and Logging

Java Agent Guide

Configuring Agent logging options

The following section describes how to run the Agent in verbose mode and set logfile details for the Agent. Introscope uses Log4J functionality for these functions. If you want to use other Log4J functionality, please see Log4J documentation at http://jakarta.apache.org/log4j/docs/documentation.html.

Running the Agent in verbose mode

Running the Agent in verbose mode records details to the log, which is helpful in debugging. To run the Agent in verbose mode: 1 Open the Agent profile, usually IntroscopeAgent.profile. 2 In the property log4j.logger.IntroscopeAgent, replace INFO with the following:
VERBOSE#com.wily.util.feedback.Log4JSeverityLevel

3 Save changes.

Redirecting Agent output to a file

To redirect Agent output to a file: 1 Open the Agent profile, usually IntroscopeAgent.profile. 2 In the property, log4j.logger.IntroscopeAgent, replace the word console with the word logfile. For example, if you wanted the Agent to report in verbose mode to a logfile, the property would look like this:

log4j.logger.IntroscopeAgent=VERBOSE#com.wily.util.feedback.Log4JS
everityLevel,logfile

By default the Agent log, IntroscopeAgent.log is written to the Wily\logs directory, unless you configure one agent autonaming options described in Java Agent Naming on page 83, in which case agent log files are also automatically named, as described in the following section. 3 If desired, change the name and/or location of the Introscope Agent logfile in the property, log4j.appender.logfile.File.

Configuring Agent logging options 97

CA Wily Introscope

Agent log files and automatic Agent naming

If you are using Automatic Agent Naming functionality, by default, when the Agent name is found automatically, either by information provided by a Java system property, or provided through the application server, the log files associated with that Agent are named automatically using that same information. The following examples show how the Agent logfile will be named. The examples use an Agent name of DOM1//ACME42 (where DOM1 is the WebLogic domain, and ACME42 is the instance). When an Agent log file is created (by default, named AutoProbe.log), if the Agent name is not yet available, a timestamp will be included in the filename, as in:
AutoProbe.20040416-175024.log

Once the Agent name becomes available, the logfile will be renamed, as in:
AutoProbe.DOM1_ACME42.log

You can disable this automatic log naming. See Advanced automatic Agent naming options on page 91. Note: These notes apply to Agent logfile automatic naming:
if the original name of the logfile does not end in .log, a period is appended,

and log added.

all characters that are not letters or digits will be replaced by underscores if advanced Log4J functionality is used, the Agent logfile automatic naming

capability might not work

Logging considerations for WebSphere z/OS

Tagging output as EBCDIC
Beginning with Version 5.0, WebSphere for z/OS changed its default encoding from EBCDIC CP1047 to ASCII ISO8859-1. Because z/OS is normally an EBCDIC machine, any logging data written by the Introscope Agent or AutoProbe must be tagged to use EBCDIC as the final output stream, instead of ASCII. To tag data as EBCDIC instead of ASCII:
Enable these new properties in the IntroscopeAgent.profile:

log4j.appender.console.encoding=IBM-1047 log4j.appender.logfile.encoding=IBM-1047
If these properties are not present in your existing IntroscopeAgent.profile, you must add them.

98 Agent Monitoring and Logging

Java Agent Guide

Eliminating startup timing issues with logging facilities

In addition, a new property has been added for z/OS WebSphere Version 5.0 and later, which is used to eliminate any startup timing window exposures that can occur with the Introscope logging facilities. To eliminate the timing window exposures:
Add this property to the Agent profile:

introscope.agent.logger.delay=100000

The value is in milliseconds, so the default delay in this example is 100 seconds.

Managing the ProbeBuilder Log

The ProbeBuilder logs the PBDs it used and the probes it added during the instrumentation process.

ProbeBuilder log name and location

The ProbeBuilder log file location is determined by where you specify Java classes with the ProbeBuilder Wizard or with the Command-Line ProbeBuilder. For a directory, the log file is located inside the destination directory. For a file, the log file is located next to the destination file. The ProbeBuilder log file is called:
<original-directory-or-original-file>.probebuilder.log

<original-directory> or <original-file> is the Java class location that

you specify with the ProbeBuilder Wizard or with the Command-Line ProbeBuilder. Only the most recent log is kept. Any prior log file is overwritten.

AutoProbe log name and location

By default the AutoProbe log file is named AutoProbe.log. To change the name, use the introscope.autoprobe.logfile property in the Agent profile. The AutoProbe.log file is generated relative to the location of the IntroscopeAgent.profile file. Note When loading the agent profile from a resource on a classpath, AutoProbe will be unable to write to the AutoProbe log file, because the IntroscopeAgent.profile file is located within a resource.

Managing the ProbeBuilder Log 99

CA Wily Introscope

100 Agent Monitoring and Logging

SECTION

10

Using Virtual Agents to Aggregate Metrics

This chapter has information about configuring Virtual Agents. Understanding Virtual Agents . Virtual Agent requirements. Configuring Virtual Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 102 102

Using Virtual Agents to Aggregate Metrics 101

CA Wily Introscope

Understanding Virtual Agents

You can configure multiple physical agents into a single Virtual Agent. A Virtual Agent enables an aggregated, logical view of the metrics reported by multiple agents. A Virtual Agent is useful if you manage clustered applications with Introscopea Virtual Agent comprised of the agents that monitor different instances of the same clustered application appears in Introscope as a single agent. This allows metrics from multiple instances of a clustered application to be presented at a logical, application level, as opposed to separately for each application instance. You can view performance and availability data for a specific application instance, by scoping your views and interactions in terms of a single agent.

Virtual Agent requirements

If you have multiple stand-alone Enterprise Managers, a Virtual Agent can contain only agents that report to the same Enterprise Manager. Agents that report to Enterprise Managers within a single cluster can belong to the same Virtual Agent, regardless of the Collector Enterprise Manager to which they report. See the Introscope Configuration and Administration Guide for more information about clustering. Consider these conditions when you configure Virtual Agents:
An agent can be assigned to multiple Virtual Agents. Virtual Agents cannot include other Virtual Agents. If you define multiple Virtual Agents, they must have unique names, including

custom metric agentsall agents in a cluster must have unique names.

Configuring Virtual Agents

Configure Virtual Agents in the agentclusters.xml file, in the

IntroscopeHome/config directory of the Enterprise Manager to which the

Agents report. If you run multiple Enterprise Managers that are clustered, define Virtual Agents in the agentclusters.xml file clusters MOM. The sample agentclusters.xml below, defines a Virtual Agent named BuyNowAppCluster, in the Introscope SuperDomain. The Virtual Agent includes all Agents, on any host, whose Agent name starts with BuyNow.

config directory of the

<?xml version="1.0" encoding="ISO-8859-1" ?>

102 Using Virtual Agents to Aggregate Metrics

Java Agent Guide

<agent-clusters xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance" xsi:noNamespaceSchemaLocation="agentclusters0.1.xsd" version="0.1" > <agent-cluster name="BuyNowAppCluster" domain="SuperDomain" >

<agent-specifier>.*\|.*\|BuyNow.*</agent-specifier>

<metric-specifier>Frontends\|.*</metric-specifier> </agent-cluster> </agent-clusters>

The root element, <agent-clusters>, is required. The <agent-cluster> element defines a Virtual Agent, and has two required attributes:
nameIf you define multiple Virtual Agents, each must have a unique name. domainAssigns the Virtual Agent to an Introscope domain.

If the domain is defined as domain="" in the agent-cluster definition, the Virtual Agent will default to the SuperDomain. If you define multiple Virtual Agents, you define an <agent-cluster> element for each. The <agent-cluster> element requires two child elements:
<agent-specifier>Contains a regular expression that specifies the Agents

in the Virtual Agent, using the standard fully qualified agent name:

<host> | <process> | <agentName>

<metric-specifier>Contains a prefix that specifies the metrics to collect

from the Agents in the Virtual Agent, in terms of Resource type, or subsets of the instances of a resource type. The recommended prefixes are:

CPU JMX WebSpherePMI Frontends

Note While the above are the recommended prefixes, any resource can be used as a metric specifier. The <agent-cluster> element can contain multiple <metric-specifier> stanzas. Note that a higher volume of matching metrics imposes high overhead on the Enterprise Manager, and can ultimately have an effect on Enterprise Manager capacity.

Configuring Virtual Agents 103

CA Wily Introscope

Note Regular expressions and wildcard metric specifiers such as ".*" and "(.*)" are allowed, but should be used with caution, if at all. Use of wildcards can result in a high volume of metrics and a performance impact. A sample agentclusters.xml is available in your IntroscopeHome/config directory.

104 Using Virtual Agents to Aggregate Metrics

SECTION

11

Configuring Agent Failover

This chapter has information about Agent failover. Understanding Agent failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 106 107 107 108

Define backup Enterprise Managers . Define failover connection order .

Configuring Java Agent failback to primary Enterprise Manager . Agent failover and domain/user configuration .

Configuring Agent Failover 105

CA Wily Introscope

Understanding Agent failover

An Agent that cannot connect to its Enterprise Manager, or loses its connection with it, can failover an alternative Enterprise Manager. To enable failover, you specify a list of alternative Enterprise Managers in the Agent profile. When an Agent configured for failover cannot connect to its default Enterprise Manager, it tries to connect to the next Enterprise Manager on its list of failover hosts. If the Agent does not connect with a failover host, it cycles through the Enterprise Managers on the list until it succeeds in connecting. If the Agent goes through the list without connecting to an Enterprise Manager, it waits 10 seconds before cycling through the list again. In a basic Introscope configuration, you define the host and port settings for one Enterprise Manager. To enable Agent Failover, you define connection properties for backup Enterprise Managers, and a list that specifies failover order.

Define backup Enterprise Managers

For each backup Enterprise Manager, create an alternate communication channel, as described in Java Agent-to-Enterprise-Manager connection properties on page 17. Be sure to assign each additional Enterprise Manager communication channel a unique namedo not use the name DEFAULT or the name of an existing channel when creating a new one. Given a primary Enterprise Manager and two backups, your Agent profile will include property definitions similar to the following:

introscope.agent.enterprisemanager.transport.tcp.host. DEFAULT=enterprise introscope.agent.enterprisemanager.transport.tcp.port. DEFAULT=5001 introscope.agent.enterprisemanager.transport.tcp.socketfactory. DEFAULT=com.wily.isengard.postofficehub.link.net.DefaultSocket Factory introscope.agent.enterprisemanager.transport.tcp.host.BackupEM1= voyager introscope.agent.enterprisemanager.transport.tcp.port.BackupEM1= 5002 introscope.agent.enterprisemanager.transport.tcp.socketfactory. BackupEM1=com.wily.isengard.postofficehub.link.net.DefaultSocket Factory

106 Configuring Agent Failover

Java Agent Guide

introscope.agent.enterprisemanager.transport.tcp.host.BackupEM2= space9 introscope.agent.enterprisemanager.transport.tcp.port.BackupEM2= 5003 introscope.agent.enterprisemanager.transport.tcp.socketfactory. BackupEM2=com.wily.isengard.postofficehub.link.net.DefaultSocket Factory

Define failover connection order

After specifying the connection properties for your backup Enterprise Managers, define the order in which the Agents will attempt to failover when it loses connection to its primary Enterprise Manager Use introscope.agent.enterprisemanager.connectionorder , described on page 169 to list the Enterprise Manager communication channelsfor the Agents primary Enterprise Manager as well as the backups. Put the primary Enterprise Manager first in the list. For the Enterprise Manager communication channels specified on the previous page, the connection order could be specified like this:

introscope.agent.enterprisemanager.connectionorder= DEFAULT,BackupEM1,BackupEM2

Configuring Java Agent failback to primary Enterprise Manager

In the default Java Agent failover scenario, if the Agent loses the connection to its primary Enterprise Manager (the first one defined in the Enterprise Connection order list), it tries to connect to the next Enterprise Manager defined in the Agent profile. You can optionally configure the Agent to periodically try to reconnect to the primary Enterprise Manager. To configure the Agent to attempt to reconnect to the primary Enterprise Manager: 1 In the Agent profile, IntroscopeAgent.profile, locate the Enterprise Manager Failback Retry Interval section. 2 Uncomment introscope.agent.enterprisemanager.failbackRetry IntervalInSeconds. 3 Save your changes. 4 Restart the application.

Define failover connection order 107

CA Wily Introscope

Agent failover and domain/user configuration

To use the Agent Failover feature and also have Users, Domains, and authentication settings defined, you must ensure that this information is in sync across the specified failover Enterprise Managers. For more information on Domains and user permissions, see the Introscope Configuration and Administration Guide.

108 Configuring Agent Failover

SECTION

III

Tailoring and Extending Data Collection

The chapters in this section have information about tailoring and extending Java Agent data collection.
Configuring Access to Application Server Data on page 111 Configuring Boundary Blame on page 115 Configuring Transaction Trace Options on page 123 Configuring Introscope SQL Agent on page 131 Enabling JMX Reporting on page 141 Configuring Platform Monitoring on page 149 Configuring WebSphere PMI on page 153 Enabling WebLogic Diagnostic Framework on page 159

Tailoring and Extending Data Collection 109

CA Wily Introscope Java Agent

110 Tailoring and Extending Data Collection

CHAPTER

12

Configuring Access to Application Server Data

This chapter has information about configuring an Agent to obtain management information from a WebLogic Server or WebSphere application server. Application Server management data and the Agent . Configuring startup class for WebLogic 8.1 or 9.0 . . . . . . . . . . . . . . . . . . . . . . 112 112 113

Configuring a custom service in WebSphere 5.0 or 6.0 .

Configuring Access to Application Server Data 111

CA Wily Introscope

Application Server management data and the Agent

In WebLogic Server and WebSphere environments, the Introscope Agent can obtain and report management information from the application server, above and beyond the metrics resulting from Introscope-enabling your applications. For example, you can configure an Agent to:
report JMX metrics from the application server report WebLogic Diagnostic Framework (WLDF) data from WebLogic 9.0 report Performance Monitoring Infrastructure (PMI) from WebSphere obtain its name from the application server

The Application Overview in the Workstation (available in Introscope v7.0 and later) uses JMX and PMI metrics, if available, in application health heuristics. Enabling the Agent to access application server management information is not required, but it enhances the visibility provided by the Application Overview. To enable an Agent to obtain and use data from the application server you configure an Introscope startup class or service in the application server, and target it at application server instances, or to an application server cluster.

Configuring startup class for WebLogic 8.1 or 9.0

This section describes how to create a startup class in WebLogic 8.1 or 9.0. For information about creating a startup class in other versions of WebLogic Server, or for more information about WebLogic Server, consult your WebLogic Server documentation. To configure a startup class for WebLogic 8.1 or 9.0: 1 Open the WebLogic Administrative Console. 2 In the left pane, expand the Deployments folder. 3 Click the Startup & Shutdown folder. The Startup and Shutdown page opens. 4 Click Configure a New Startup Class. The Configuration tab is shown. 5 In the Name field, enter:

Introscope Startup Class

6 In the ClassName field, enter:
com.wily.introscope.api.weblogic.IntroscopeStartupClass

7 Click Create. The Target and Deploy tab appears.

112 Configuring Access to Application Server Data

Java Agent Guide

8 Check the box(es) for the server(s) youd like to make this startup class available to. Click Apply. Use the run before deploying apps option. 9 Add the location of the WebAppSupport.jar to the application startup classpath. 10 Restart the application server.

Configuring a custom service in WebSphere 5.0 or 6.0

This section describes how to create a custom service in WebSphere 5.0. To create a custom service in previous versions, or for more information, consult your WebSphere documentation. To configure a custom service: 1 Open the WebSphere Administrative Console. 2 Select the server you'd like to configure, then:
For WebSphere 6.0, click Administration, then Custom Services. For WebSphere 5.0, click Custom Services.

3 Click New to add a new Custom Service, then:

For WebSphere 6.0, check the box for Enable service at server startup For WebSphere 5.0, check the Startup checkbox.

4 In the Classname field, enter:

com.wily.introscope.api.websphere.IntroscopeCustomService

5 In the Display Name field, enter:

Introscope Custom Service

6 In the Classpath field, enter:
<WebSphere home>/wily/WebAppSupport.jar

7 Click OK. 8 Restart the application server.

Configuring a custom service in WebSphere 5.0 or 6.0 113

CA Wily Introscope

114 Configuring Access to Application Server Data

SECTION

13

Configuring Boundary Blame

This chapter describes default Agent blame reporting behaviors, and related configuration options. Understanding Boundary Blame Using Blame tracers . . . . . Disabling Boundary Blame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 122 122

Configuring Boundary Blame 115

CA Wily Introscope

Understanding Boundary Blame

Introscopes Blame Technology works in a managed Java Application to enable you to view metrics at the front and backends of your application. This capability, referred to as boundary blame, allows users to triage problems in the application front or backends. Introscope uses the SQL statement in the SQL Agent monitoring functionality to automatically detect backends. If the SQL Agent is unavailable, Introscope automatically detects socket calls as backends, because backends such as client/ server databases, JMS servers, and LDAP servers are accessed through a socket. If you have Oracle backends and do not use the Introscope SQL Agent, see Boundary Blame and Oracle backends on page 43. For information about how boundary blame is presented in the Introscope Investigator, see the Introscope Workstation User Guide.

Using URL groups

You can use URL Groups to monitor browser response time for sets of requests whose path prefix begins with a string you define. The path prefix is the portion of the URL that follows the hostname. For example, in this URL:

http://burger1.com/testWar/burgerServlet?ViewItem&category= 11776&item=5550662630&rd=1
the path prefix is:

/testWar
You can define a URL group for any useful category of requests that can be derived from a URLs path prefix. For example, depending on the form of your application URLs, you could define URL groups for each customer your application supports, for each major application, or for sub-applications. This enables you to monitor performance in the context of committed service levels, or for missioncritical portions of your application. The following example is an excerpt from a Java Agent profile, showing how URL Groups are defined:

Example URL group property definitions

introscope.agent.urlgroup.keys=alpha,beta,gamma introscope.agent.urlgroup.group.alpha.pathprefix=/testWar introscope.agent.urlgroup.group.alpha.format=foo {host} bar {protocol} baz {port} quux {query_param:foo} red {path_substring:2:5} yellow {path_delimited:/:0:1} green {path_delimited:/:1:4} blue {path_substring:0:0} introscope.agent.urlgroup.group.beta.pathprefix=/nofilterWar

116 Configuring Boundary Blame

Java Agent Guide

introscope.agent.urlgroup.group.beta.format=nofilter foo {host} bar {protocol} baz {port} quux {query_param:foo} red {path_substring:2:5} yellow {path_delimited:/:0:1} green {path_delimited:/:1:4} blue {path_substring:0:0} introscope.agent.urlgroup.group.gamma.pathprefix=/examplesWebApp introscope.agent.urlgroup.group.gamma.format=Examples Web App

Configuring URL groups

This section provides information about the properties that configure URL Groups.

Defining keys for URL groups

The property introscope.agent.urlgroup.keys defines a list of the IDs, or keys, of all of your URL Groups. The key for a URL Group is referenced in other property definitions that declare an attribute of the URL group. The following example defines the keys for three URL Groups:

introscope.agent.urlgroup.keys=alpha,beta,gamma
If you define URL Groups so some URLs fall into multiple groups, the order in which you list the keys for the URL Groups in the property is important. The URL Group with the narrower membership should precede the URL Group with broader membership. For example, if the IP Group with key alpha has the path prefix /examplesWebApp and the URL Group with key delta has the path prefix /examplesWebApp/cleverones, delta should precede alpha in the keys parameter

Defining membership of each URL group

The property introscope.agent.urlgroup.group.groupKey.pathprefix specifies a pattern against which the path prefix of a URL is matched, defining which requests fall within the URL Group.

Example 1
This property definition assigns all requests in which the path portion of the URL starts with /testWar to the URL Group whose key is alpha:

introscope.agent.urlgroup.group.alpha.pathprefix=/testWar
Requests that match the specified pathprefix include:

http://burger1.com/testWar/burgerServlet?ViewItem&category= 11776&item=5550662630&rd=1 http://burger1.com/testWar/ burgerServlet?Command=Order&item=5550662630

Understanding Boundary Blame 117

CA Wily Introscope

Example 2
A company that provides call center services could monitor response time for functional areas by setting up a URL Group for each application function. If customers access services with this URL:

http://genesystems/us/application_function/
where

application_function corresponds to applications such as application_function.

OrderEntry, AcctService, and Support, the pathprefix property for each

URL group would specify the appropriate

Note You can use the asterisk symbol (*) as a wildcard in pathprefix properties.

Define name for a URL group

The property introscope.agent.urlgroup.group.groupKey.format
determines the names under which response time metrics for a URL group whose key is

groupKey appear in the Introscope Workstation.

Typically, the format property is used to assign a text string as the name for a URL. The following example causes metrics for the URL Group with key alpha to appear in the Workstation under the name Alpha Group:

introscope.agent.urlgroup.group.alpha.format=Alpha Group

Advanced naming techniques for URL groups (optional)

You can derive a URL Group name from request elements such as the server port, the protocol, or from a substring of the request URL. This is useful if your application modules are easily differentiated by inspecting the request. This section describes advanced forms of the format property.

Using host as URL group name

To organize metrics for a URL group under names that reflect the hostname of the HTTP server associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={host}
When format={host}, statistics for these requests would appear under the metric names us.mybank.com and uk.mybank.com respectively:

https://us.mybank.com/mifi/loanApp...... https://uk.mybank.com/mifi/loanApp.....

118 Configuring Boundary Blame

Java Agent Guide

Using protocol as URL group name

To organize statistics for a URL group under names that reflect the protocol associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={protocol}
When format={protocol} statistics are grouped in Investigator under metric names that correspond to the protocol portion of request URLs. For example, statistics for these requests would appear under the metric name https:

https://us.mybank.com/cgi-bin/mifi/scripts...... https://uk.mybank.com/cgi-bin/mifi/scripts......

Using port as URL group name

To organize statistics for a URL group under names that reflect the port associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={port}
When format={port}, statistics are grouped under names that correspond to the port portion of request URLs. For example, statistics for these requests would appear under the name 9001.

https://us.mybank.com:9001/cgi-bin/mifi/scripts...... https://uk.mybank.com:9001/cgi-bin/mifi/scripts......

Using parameter as URL group name

To organize statistics for a URL group in Investigator under metric names that reflect the value of a parameter associated with requests, define the format parameter like this:
introscope.agent.urlgroup.group.alpha.format={query_param:param}

When format={query_param:param} statistics are grouped in Investigator under metric names that correspond to value of the parameter specified. Requests without parameters are listed under <empty>. For example, given this parameter definition:
introscope.agent.urlgroup.group.alpha.format= {query_param:category}

Statistics for these requests would appear under the metric name 734
http://ubuy.com/ws/shoppingServlet?ViewItem&category=734 &item=3772&tc=photo

Understanding Boundary Blame 119

CA Wily Introscope

http://ubuy.com/ws/shoppingServlet?ViewItem&category=734 &item=8574&tc=photo

Using a substring of the request path as URL group name

To organize statistics for a URL group under names that reflect a substring of the path portion of request URLs, define the format parameter like this:
introscope.agent.urlgroup.group.alpha.format= {path_substring:m:n}

where

m is the index of the first character, and n is one greater than the index of

the last character. String selection operates like the java.lang.String.substring() method. For example, given this setting:
introscope.agent.urlgroup.group.alpha.format= {path_substring:0:3}

Statistics for this request would appear under the metric node /ht

http://research.com/htmldocu/WebL-12.html

Using delimited portion of the request path as URL group name

To organize statistics for a URL group under names that reflect a characterdelimited portion request URL path, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format= {path_delimited:delim_char:m:n}
where delim_char is the character that delimits the segments in the path, m is the index of the first segment to select, and n is one greater than the index of the last segment to select. For example, given this setting:

introscope.agent.urlgroup.group.alpha.format= {path_delimited:/:2:4}
statistics for the requests of this form:

http://www.buyitall.com/userid,sessionid/pageid
would appear under the metric name Note that:
a delimiter character counts as a segment the segment count starts at 0

/pageid

120 Configuring Boundary Blame

Java Agent Guide

This table shows the segments as delimited by the slash character: Segment Index Segment String
0 / 1 userid,sessionid 2 / 3 pageid

You can specify multiple delimiters as necessary. For example, given this setting:

introscope.agent.urlgroup.group.alpha.format= {path_delimited:/,:3:4}
statistics for requests of the form shown above would appear under the metric name

sessionid.

This table shows the segments as delimited by the slash and the comma character: Segment Index Segment String
0 / 1 userid 2 , 3 sessionid 4 / 5 pageid

Using multiple naming methods for URL groups

You can combine multiple naming methods in a single format string, as shown below:
introscope.agent.urlgroup.group.alpha.format=red {host} orange {protocol} yellow {port} green {query_param:foo} blue {path_substring:2:5} indigo {path_delimited:/:0:1} violet {path_delimited:/:1:4} ultraviolet {path_substring:0:0} friend computer

Running the URLGrouper

URLGrouper is a command-line utility that analyzes a web server log file in Common format, and produces BRTA property settings for a set of URL Groups. Using URLGrouper gives you a starting point for defining your own URL Groups. Note You can use the URLGRouper utility to analyze your Web server log file. URLGrouper outputs a set of property settings for potential URL Groups, based on the contents of the Web server log file. The asterisk symbol (*) can be used with URLGrouper as a wildcard. To run URLGrouper: 1 Open a command shell. 2 Enter this command

java -jar urlgrouper.jar logfile

where

logfile is the full path to your web server log file.

3 Property definitions for a set of URL Groups are output to STDOUT.

Understanding Boundary Blame 121

CA Wily Introscope

4 To configure the proposed URL Groups, copy the property statements produced by URL Grouper into the Agent profile.

Using Blame tracers

You can use tracers to explicitly mark the frontends and backends in your application. For more information, see Using Blame Tracers to mark blame points on page 41.

Disabling Boundary Blame

By default, Boundary Blame is enabled. To disable boundary blame in favor of the component-level blame implemented in Introscope versions earlier than 7.0, use the introscope.agent.blame.type described on page 182.

122 Configuring Boundary Blame

SECTION

14

Configuring Transaction Trace Options

This chapter has information about default transaction tracing behaviors and related configuration options. Controlling automatic transaction tracing behavior Configuring cross-process transaction tracing . Extending transaction trace data collection . Disabling the capture of stalls as Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 125 126 128

Configuring Transaction Trace Options 123

CA Wily Introscope

Controlling automatic transaction tracing behavior

Automatic transaction tracing enables historical analysis of potentially problematic transaction types without explicitly running transaction traces. Introscope offers two types of automatic transaction tracing:
Transaction trace sampling that is enabled by default, based on your URL

groupings
Configurable automatic trace sampling that gathers trace information

regardless of URL groupings

Transaction Trace component clamp

Introscope now sets a clamp (set by default to 5,000 components) to limit the size of traces. When this limit is reached, warnings appear in the log, and the trace stops. The problem was an infinitely expanding transactionfor example when a servlet executes hundreds of object interactions and back-end SQL calls. Transaction Tracer viewed this as one transaction, continuing infinitely. There was an upper limit as the engine did eventually finish but the JVM would run out of memory beforehand. The new property for clamping infinitely expanding transactions is in the IntroscopeAgent.profile file:
introscope.agent.transactiontrace.componentCountClamp= 5,000

Transaction trace sampling by URL group

Transaction trace sampling is enabled by default. As appropriate you can disable the behavior, change the sampling period, or de-randomize the timing of sampling. These behaviors are configured using these Agent properties:
introscope.agent.transactiontracer.sampling.enabled

Set to false to disable transaction trace sampling. The default value is true .
introscope.agent.transactiontracer.sampling.periodinseconds

To change the frequency with which a normalized unique URL is traced from the default value of 3,600 seconds (one hour) specify the new period in seconds.
introscope.agent.transactiontracer.sampling.randomize

By default, transaction trace sampling is performed on a random basis, so that not all samples are taken at the beginning of the period. Primarily for troubleshooting purposes, the sampling timing can be de-randomized, by changing the value of this property to true.

124 Configuring Transaction Trace Options

Java Agent Guide

Configuring automatic trace sampling

You can configure automatic tracing of transactions that takes place regardless of URL group membership. By configuring trace sampling in this way, you ensure sampling of a sufficient basis of normal transactions, even if no URL groups are configured. When you configure automatic trace sampling, you specify the number of transactions to trace, during a time interval you specify. To configure automatic trace sampling, use these properties:
introscope.agent.transactiontracer.sampling.perinterval.count

Specifies the number of transactions to trace, during the interval you specify. The default number of transactions is 1.
introscope.agent.transactiontracer.sampling.interval.seconds

Specifies the length of time to trace the number of transactions you specify. The default interval is every 2 minutes. Note These properties are located, by default, in the Enterprise Manager properties file. Before changing the defaults for the sampling.perinterval and sampling.interval properties, consider the potential for increased load in the Enterprise Manager with higher sampling rates. The Enterprise Manager will push this configuration to all agents connected to the Enterprise Manager. Configuring these properties in the agent will overwrite the configuration set by the Enterprise Manager for an individual agent.

Configuring cross-process transaction tracing

Transaction Tracer can trace transactions that cross JVM boundaries on WebLogic Server 8 or later, or WebSphere 6.0if the environment is comprised of compatible versions of the same vendors application server. Cross-process transaction tracing is supported for synchronous transactions, for instance, servlets to EJBs.

Enabling cross-process tracing in WebSphere

1 Configure web application support. Follow the instructions in Configuring a custom service in WebSphere 5.0 or 6.0 on page 113. 2 Turn on the work area service. From the admin page, servers->application servers, click on server1, click on Business Process Services, click on Work Area Service, check the Enable service at server startup box.

Configuring cross-process transaction tracing 125

CA Wily Introscope

3 Set introscope.agent.websphere.crossjvm=true in the agent profile.

Enabling cross-process tracing in WebLogic Server

1 Configure web application support. Follow the instructions in Configuring startup class for WebLogic 8.1 or 9.0 on page 112. 2 Add -Dweblogic.TracingEnabled=true to the java command line for starting WebLogic Server. 3 Set introscope.agent.weblogic.crossjvm=true in the agent profile.

Extending transaction trace data collection

The Introscope Agent collects basic Transaction Trace data such as Domain/Host/ Process/Agent, timestamp, duration, URL, and so forth, by default. You can configure the Introscope Transaction Tracer to obtain additional information, including User ID data for Servlet and JSP invocations, and other transaction trace data such as HTTP request headers, request parameters, and session attributes. To capture this information, you must define the criteria in the IntroscopeAgent.profile.

About User ID data

To configure the Agent to identify User IDs for Servlet and JSP invocations, you must first obtain information on how your managed application specifies user IDs. The Application Architect who developed the managed application can probably provide this information. Introscope Transaction Tracer can identify User IDs from managed applications that store User IDs in one of these ways:
HttpServletRequest.getRemoteUser() HttpServletRequest.getHeader (String key) HttpSession.getValue (String key), where returned object is either a String

representing the UserID, or an Object whose toString() returns to the UserID If your managed application stores User IDs using one of these methods, see Configuring Agent to collect additional transaction trace data on page 127, to configure Agent settings to collect User ID data.

126 Configuring Transaction Trace Options

Java Agent Guide

About servlet request data

Using Introscope, you can collect transaction trace data that matches userconfigurable parameters. For example, you can specify the Introscope Agent to collect transaction trace data for transactions that contain the User-Agent HTTP request header. Introscope can record this servlet request information:
request headers request parameters session attributes

To record this servlet request information for your managed application, see Configuring Agent to collect additional transaction trace data on page 127 to configure Agent settings to collect this data.

Configuring Agent to collect additional transaction trace data

You can configure the Introscope Agent to collect additional transaction trace data such as User ID, HTTP request headers, HTTP request parameters, or HTTP session attributes. To configure the Introscope Agent to collect additional transaction trace data: 1 Open the Agent profile, IntroscopeAgent.profile. 2 Locate the Transaction Tracer properties under the Transaction Tracer Configuration heading.

Collecting user id data

To configure the Introscope Agent to identify User IDs, you configure the

properties that correspond to the method your managed application uses to store User IDs. Note Ensure that only one set of properties are not commented, or the wrong properties might be used.
For HttpServletRequest.getRemoteUser(), uncomment the property:

introscope.agent.transactiontracer.userid.method=HttpServletRequest.get RemoteUser
For HttpServletRequest.getHeader (String key), uncomment the following pair

of properties, and define a key string for the second property:

introscope.agent.transactiontracer.userid.method=HttpServletReques
t.getHeader

introscope.agent.transactiontracer.userid.key=<application defined
key string>

Extending transaction trace data collection 127

CA Wily Introscope

For HttpSession.getValue (String key), uncomment the following pair of

properties, and define a key string for the second property:

introscope.agent.transactiontracer.userid.method=HttpServletReques
t.getValue

introscope.agent.transactiontracer.userid.key=<application defined
key string>

Collecting servlet request data

To record servlet request information such as HTTP request headers and parameters, use the agent configuration properties specified in steps 4 through 6: 1 To specify the HTTP request headers for which to collect transaction trace data, uncomment this property, and specify the HTTP request header(s) to track, in a comma-separated list:
#introscope.agent.transactiontracer.parameter.httprequest.headers=UserAgent

2 To specify the HTTP request parameters for which to collect transaction trace data, uncomment this property and specify the HTTP request parameter(s) to track, in a comma-separated list:
#introscope.agent.transactiontracer.parameter.httprequest.parameters=pa rameter1,parameter2

3 To specify the HTTP session attributes for which to trace data, uncomment this property and specify the HTTP session attribute(s) to track, in a commaseparated list, for example:
#introscope.agent.transactiontracer.parameter.httpsession.attributes=ca rtID,deptID

4 Restart the managed application.

Disabling the capture of stalls as Events

By default, Introscope captures transaction stalls as Events in the Transaction Event database, and generates stall metrics from the detected events. Stall metrics are generated for the first and last method in the transaction. Users can view stall Events and associated metrics in the Workstations Historical Event Viewer. Note Generated stall metrics are always available, but stall events are only visible if Introscope Error Detector is installed. Stalls are stored as ordinary errors, and will be visible in the Errors TypeView, or in the historical query viewer by querying for type:errorsnapshot.

128 Configuring Transaction Trace Options

Java Agent Guide

You can disable the capture of stalls as Events, change the stall threshold, or change the frequency with which the Agent checks for stalls using these properties:
introscope.agent.stalls.enable controls whether the Agent checks for

stalls and creates events for detected stalls.

introscope.agent.stalls.thresholdseconds, specifies the minimum

threshold response time at which time a transaction is considered stalled.

introscope.agent.stalls.resolutionseconds specifies the frequency

that the agent checks for stalls. Note Support for stall tracers in PBDs will be deprecated.

Disabling the capture of stalls as Events 129

CA Wily Introscope

130 Configuring Transaction Trace Options

SECTION

15

Configuring Introscope SQL Agent

This chapter has instructions for configuring Introscope SQL Agent. SQL Agent overview . SQL Agent files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 133 133 134 136 138 138

Supported JDBC drivers and data sources . Configuring SQL Agent for WebSphere . Turning off Blame Metrics for SQL Agent SQL Agent Metrics . . . . . . . Configuring SQL Agent for WebLogic Server

Configuring Introscope SQL Agent 131

CA Wily Introscope

SQL Agent overview

The Introscope SQL Agent reports detailed database performance data to the Enterprise Manager. SQL Agent provides visibility into the performance of individual SQL statements in your application. SQL Agent is inserted between the managed application and the driver or DataSource that processes JDBC activity to track the interaction between the managed application and the database. In the same way that Introscope monitors your Java application, SQL Agent is non-intrusive, does not interfere with the application or the database, and monitors the activity with very low overhead. To efficiently provide meaningful performance measurements down to the individual SQL statement level, SQL Agent summarizes the performance data it collects by stripping out transaction-specific data and converting the original SQL statements into an Introscope-specific format call Generalized statements. Since Generalized statements do not include sensitive information such as credit card numbers, this process also protects security of data. For example, SQL Agent converts this SQL query:
SELECT * FROM BOOKS WHERE AUTHOR = 'Atwood'

to this Generalized statement:

SELECT * FROM BOOKS WHERE AUTHOR = ?

Similarly, SQL Agent converts this SQL update statement:

INSERT INTO BOOKS (AUTHOR, TITLE) VALUES ('Atwood', 'The Robber Bride')

to this Generalized statement:

INSERT INTO BOOKS (AUTHOR, TITLE) VALUES (?, ?)

Note Only text within quotation marks ('xyz') is normalized. Metrics for Generalized statements are aggregated and can be viewed in the JDBC node of the Investigator tree in the Workstation and WebView.

132 Configuring Introscope SQL Agent

Java Agent Guide

SQL Agent files

As of Introscope 7.0, the Introscope Agent installer installs SQL Agent automatically. The following files are installed:
wily/ext/SQLAgent.jar wily/sqlagent.pbd

Note By default, Agent extensions such as SQLAgent.jar are installed in the wily/ext directory. You can change the location of the Agent extension directory with the introscope.agent.extensions.directory property in the Agent profile. If you change the location of the /ext directory, be sure to move the contents of the /ext directory as well.

Supported JDBC drivers and data sources

SQL Agent supports the following JDBC drivers and JDBC datasources. Configuration instructions have been simplified to apply to both a JDBC driver and a JDBC datasource. Supported JDBC drivers The SQL Agent supports the following JDBC drivers:
Oracleclasses111.zip, classes12.zip,

classes111_g.zip, classes12_g.zip
DB2db2java.zip Sybasejconn2.jar WebLogic jDriver for OraclejDriver 6.1

The SQL Agent fully supports the JDBC 1.0 and 2.0 specifications, including support for all JDBC driver types, I through IV. Supported JDBC datasources for WebSphere In db2java.zip:
COM.ibm.db2.jdbc.DB2ConnectionPoolDataSource COM.ibm.db2.jdbc.DB2XADataSource

in classes12.zip and classes12_g.zip:

oracle.jdbc.pool.OracleConnectionPoolDataSource oracle.jdbc.xa.client.OracleXADataSource

Supported JDBC datasources for WebLogic

in classes12.zip:
oracle.jdbc.xa.client.OracleXADataSource

SQL Agent files 133

CA Wily Introscope

Configuring SQL Agent for WebSphere

SQL Agent supports WebSphere Application Server version 4.0 and higher. The following instructions describe how to configure SQL Agent to work with WebSphere using either a JDBC Driver or a JDBC DataSource. You should be aware that:
You should not attempt to use the ProbeBuilder twice on the same JDBC

DataSource or driver.
If other applications use a JDBC DataSource or driver that has been

Introscope-enabled, you will need to add the Agent.jar to the classpaths for those applications. Once this is done, the applications may show up as Unknown Processes in the Workstation. However, this will not affect the functionality or performance of your application and can be ignored.

Step 1: Configure the JDBC datasource or Driver in WebSphere

Configure your JDBC DataSource or driver to work with SQL Agent and WebSphere.

Step 2: Introscope-enable the JDBC DataSource or Driver

The instructions you follow in this section depend on which ProbeBuilder method you use:
JVM AutoProbeno further configuration is required. Application Server AutoProbe or Manual ProbeBuilderuse the instructions in

Use manual or application server ProbeBuilding to instrument the JDBC DataSource or Driver on page 134.

Use manual or application server ProbeBuilding to instrument the JDBC DataSource or Driver
The following instructions assume that:
Introscope, and specifically, ProbeBuilder are already installed you have write permission in the directory that contains your driver file you are able to use the ProbeBuilder Wizard application either on Windows or

via X-windows on UNIX. For instructions on command-line use of ProbeBuilder, see the Using ProbeBuilder from the command line on page 72. To manually Introscope-enable the JDBC DataSource or Driver: 1 Shut down WebSphere.

134 Configuring Introscope SQL Agent

Java Agent Guide

2 Locate the sqlagent.pbd file from an agent installation, and copy it to the <Introscope home>\config\custompbd directory. 3 Locate the file containing the Java classes that implement your JDBC DataSource. Often this is an archive file specified in one of the CLASSPATH settings for your application server. In Websphere, the list of driver classes can be found in <WebSphere home>\bin\setupCmdLine.bat in the DBDRIVER_JARS property. 4 Run the ProbeBuilder Wizard application located in the Introscope directory:
Introscope ProbeBuilder Wizard.exe on Windows IntroscopeProbeBuilderWizard on UNIX

5 At the Welcome screen, click Next. 6 In the Select Original Java Bytecode screen, browse to and select the file containing the JDBC DataSource to Introscope-enable. (For example, if using the file containing an Oracle JDBC DataSource, the name of the file would be classes12.zip.) Note Before Introscope-enabling any file, save a backup copy. 7 In the Destination Location screen, click Next to name the resulting Introscopeenabled file. (For example, if using the file containing an Oracle JDBC DataSource, the name of the resulting file would be classes12.isc.zip.) Accept the default save location for the resulting file. 8 In the System Directives screen, select Websphere SYSTEM directives. Click Next. 9 In the Custom Directives screen, select the sqlagent.pbd (along with any other custom .pbds used in your deployment). 10 Click Add Probes. This will create a copy of the file containing the JDBC DataSource (for example, classes12.isc.zip) and Introscope-enable the JDBC DataSource in this copy. 11 When the Finished screen appears, click Exit. 12 In your JDBC driver directory you should find the file containing the Introscopeenabled JDBC DataSource or driver. To use this file to see the JDBC Metrics in Introscope you can either:
set the original file aside and rename the Introscope-enabled one, as in the

following example:
c:\> rename classes12.zip classes12.orig.zip c:\> rename classes12.isc.zip classes12.zip

Note If the file to be renamed is in use, do not attempt to rename it until you first shut down any application or database that is actively using the file containing the JDBC DataSource.
change your application servers CLASSPATH setting to point to this new

Configuring SQL Agent for WebSphere 135

CA Wily Introscope

Introscope-enabled file. Note If other applications use this same JDBC DataSource or driver file, you will need to put the Agent.jar (located in <WebSphere home>\wily) in the classpath for those applications. Otherwise, applications that reference the JDBC DataSource will fail at runtime. 13 Restart your admin server and application server to see the JDBC Metrics.

Configuring SQL Agent for WebLogic Server

SQL Agent works with WebLogic Server version 6.1 and higher. The following instructions describe how to configure SQL Agent to work with WebLogic using either a JDBC Driver or a JDBC DataSource. Also, you should be aware that:
you should not attempt to use the ProbeBuilder twice on the same JDBC

DataSource or driver
If other applications use a JDBC DataSource or driver that has been

Introscope-enabled, those applications may show up as Unknown Processes in the Workstation. However, this will not affect the functionality or performance of your WebLogic application and can be ignored. The instructions you follow in this section depend on which ProbeBuilder method you use:
JVM AutoProbeno further configuration is required. Application Server AutoProbe or Manual ProbeBuilder use the instructions in

Manually Introscope-enabling the JDBC DataSource or Driver on page 136.

Manually Introscope-enabling the JDBC DataSource or Driver

The following instructions assume that:
you have write permission in the directory that contains your driver file you are able to use the ProbeBuilder Wizard application either on Windows or

via X-windows on UNIX. For instructions on command-line use of ProbeBuilder, see the Introscope Configuration and Administration Guide. To manually Introscope-enable the JDBC DataSource or Driver: 1 Shut down WebLogic. 2 Locate the sqlagent.pbd file from an agent installation, and copy it to the <Introscope home>\config\custompbd directory.

136 Configuring Introscope SQL Agent

Java Agent Guide

3 Locate the file containing the Java classes that implement your JDBC DataSource or driver. Often this is an archive file specified in one of the CLASSPATH settings for your application server. For example, for the Oracle XA DataSource, the name of the file would be classes12.zip. 4 Run the ProbeBuilder Wizard application located in the Introscope directory:
Introscope ProbeBuilder Wizard.exe on Windows IntroscopeProbeBuilderWizard on UNIX

5 At the Welcome screen, click Next. 6 In the Select Original Java Bytecode screen, browse to and select the file containing the JDBC DataSource or driver to Introscope-enable. For example, for the Oracle XA DataSource, the name of the file would be classes12.zip. Note Before Introscope-enabling any file, save a backup copy. In the Destination Location screen, click Next to name the resulting Introscope-enabled file. For example, for the Oracle XA DataSource, the name of the resulting file would be classes12.isc.zip. 7 In the System Directives screen, select the SYSTEM directives for WebLogic. Click Next. 8 In the Custom Directives screen, select the sqlagent.pbd (along with any other custom .pbds used in your deployment). 9 Click Add Probes. This will create a copy of the file containing the JDBC DataSource or driver (e.g., classes12.isc.zip) and Introscope-enable the JDBC DataSource or driver in this copy. This new file will be located in the same directory as the original. 10 When the Finished screen appears, click Exit. 11 In your JDBC driver directory you should find the file containing the Introscopeenabled JDBC DataSource or driver. To use this file to see the JDBC Metrics in Introscope you can either:
set the original file aside and rename the Introscope-enabled one, as in the

following example:

c:\> rename classes12.zip classes12.orig.zip c:\> rename classes12.isc.zip classes12.zip

Note If the file to be renamed is in use, do not attempt to rename it until you first shut down any application or database that is actively using the file containing the JDBC DataSource.
change your application servers CLASSPATH setting to point to this new

Introscope-enabled file.

Configuring SQL Agent for WebLogic Server 137

CA Wily Introscope

Note If other applications use this same JDBC DataSource or driver file, you will need to put the Agent.jar (located in <WebLogic home>\wily) in the classpath for those applications. Otherwise, applications that reference the JDBC DataSource will fail at runtime. 12 Restart your application server to see the JDBC Metrics.

Turning off Blame Metrics for SQL Agent

Blame metric data is collected by default. However, to reduce data overhead and reduce the number of metrics generated, you can turn Blame Metric data off. Note: If Blame Metric generation is turned off for SQL Agent, SQL Agent data will not appear in Transaction Traces. To turn off Blame Metric data generation: 1 Open the file, IntroscopeAgent.profile. 2 Locate the property, introscope.agent.sqlagent.useblame , and:
uncomment the property enter a value of false.

3 Restart the managed application.

Turning Off Statement Metrics

Because some applications may generate an extremely large number of unique SQL statements, causing a metric explosion in the SQL Agent, it is now possible to turn off statement metrics in the SQL Agent. You will not lose backend or toplevel JDBC metrics if you turn off statement metrics. To turn off statement metrics: 1 Open the sqlagent.pbd file. 2 Remove {sql} from the trace directives you wish to turn off. 3 Save the sqlagent.pbd file.

SQL Agent Metrics

SQL Agent metrics appear under the Backends node in the Investigator tree of the Workstation. Metric types include:
Connection CountThe number of live Connection objects in memory.

138 Configuring Introscope SQL Agent

Java Agent Guide

A connection is opened when a Drivers connect() method is invoked, and closed when the Connection returned by the above connect() invocation is closed via the close() method. If more than one Introscope-enabled driver class is used in a JVM, this is the global number of open connections across all drivers.
Average Result Processing Time (ms)The average processing time of a

query. This metric represents the average time spent processing a ResultSet from the end of the executeQuery() call to the invocation of the ResultSet's close() method. The following are metrics produced by the SQL Agent. You may also see additional metrics attributed to specific driver classes. Note Introscope-enabled XADataSources may not report commit or rollback metrics. Other Introscope-enabled DataSources may not report commit or rollback metrics unless those metrics contain data. Backends|<backendName>:Average Response Time Backends|<backendName>:Concurrent Invocations Backends|<backendName>:Connection Count Backends|<backendName>:Responses Per Interval Backends|<backendName>:Errors Per Interval Backends|<backendName>:Stall Count Backends|<backendName>|Commits:Average Response Time Backends|<backendName>|Commits:Concurrent Invocations Backends|<backendName>|Commits:Responses Per Interval Backends|<backendName>|Commits:Errors Per Interval Backends|<backendName>|Commits:Stall Count Backends|<backendName>|Rollbacks:Average Response Time Backends|<backendName>|Rollbacks:Concurrent Invocations Backends|<backendName>|Rollbacks:Responses Per Interval Backends|<backendName>|Rollbacks:Errors Per Interval Backends|<backendName>|Rollbacks:Stall Count Backends|<backendName>|SQL|Dynamic|Query|<sql>:Average Response Time Backends|<backendName>|SQL|Dynamic|Query|<sql>:Average Result Processing Time Backends|<backendName>|SQL|Dynamic|Query|<sql>:Concurrent Invocations Backends|<backendName>|SQL|Dynamic|Query|<sql>:Responses Per Interval Backends|<backendName>|SQL|Dynamic|Query|<sql>:Errors Per Interval Backends|<backendName>|SQL|Dynamic|Query|<sql>:Stall Count

SQL Agent Metrics 139

CA Wily Introscope

Backends|<backendName>|SQL|Dynamic|Update|<sql>:Average Response Time Backends|<backendName>|SQL|Dynamic|Update|<sql>:Concurrent Invocations Backends|<backendName>|SQL|Dynamic|Update|<sql>:Responses Per Interval Backends|<backendName>|SQL|Dynamic|Update|<sql>:Errors Per Interval Backends|<backendName>|SQL|Dynamic|Update|<sql>:Stall Count Backends|<backendName>|SQL|Prepared|Query|<sql>:Average Response Time Backends|<backendName>|SQL|Prepared|Query|<sql>:Average Result Processing Time Backends|<backendName>|SQL|Prepared|Query|<sql>:Concurrent Invocations Backends|<backendName>|SQL|Prepared|Query|<sql>:Responses Per Interval Backends|<backendName>|SQL|Prepared|Query|<sql>:Errors Per Interval Backends|<backendName>|SQL|Prepared|Query|<sql>:Stall Count Backends|<backendName>|SQL|Prepared|Update|<sql>:Average Response Time Backends|<backendName>|SQL|Prepared|Update|<sql>:Concurrent Invocations Backends|<backendName>|SQL|Prepared|Update|<sql>:Responses Per Interval Backends|<backendName>|SQL|Prepared|Update|<sql>:Errors Per Interval Backends|<backendName>|SQL|Prepared|Update|<sql>:Stall Count Backends|<backendName>|SQL|Stored Response Time Backends|<backendName>|SQL|Stored Invocations Backends|<backendName>|SQL|Stored Per Interval Backends|<backendName>|SQL|Stored Interval Backends|<backendName>|SQL|Stored Procedures|Query|<sql>:Average Procedures|Query|<sql>:Concurrent Procedures|Query|<sql>:Responses Procedures|Query|<sql>:Errors Per Procedures|Query|<sql>:Stall Count

140 Configuring Introscope SQL Agent

SECTION

16

Enabling JMX Reporting

This chapter contains information about enabling the Agent to report JMX data. Introscope Java Agent JMX support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 142 143 144 145 147

Default JMX Metric conversion process . Managing Metric volume with JMX filters Configuring JMX reporting . . . . . . . Enabling JSR-77 data for WAS 6.x

Using primary key conversion to streamline JMX Metrics

Enabling JMX Reporting 141

CA Wily Introscope

Introscope Java Agent JMX support

Introscope can collect management data that application servers or Java applications expose as JMX-compliant MBeans, and present the JMX data in the Investigator metric tree. Introscope supports any MBean built to the Sun JMX specification. (For more information on the Sun JMX specification, see http://java.sun.com/products/ JavaManagement/.) Introscope converts the JMX data to Introscope metric format and displays it in the Investigator under the following Resource:

<Domain>|<Host>|<Process>|<Agent>|JMX|

Introscope support for WebLogic 9.0 JMX Metrics

WebLogic versions prior to WebLogic 9.0 provided only a single MBeanServer as a source of JMX metrics. WebLogic 9.0 provides three:
RuntimeServiceMBean: per-server runtime metrics, including active effective

configuration
DomainRuntimeServiceMBean: domain-wide runtime metrics EditServiceMBean: allows user to edit persistent configuration

Introscope polls only the RuntimeServiceMBean, because it is the only one that supports local access (an efficiency issue), and because it contains most of the data expected to be relevant.

Default JMX Metric conversion process

This section describes the process Introscope uses, by default, to convert JMX Metrics for display in the Investigator. This method is used to convert an MBean if:
You use WebLogic 9.0, or You have not configured valid primary keys, as described in Using primary key

conversion to streamline JMX Metrics below. Note: If you specify primary keys that no MBeans match, Introscope will use the default conversion method. In the default conversion method, Introscope displays both the name and the value of the attribute, and lists the pairs alphabetically in the metric tree.

Domain>|<Host>|<Process>|<Agent>|JMX|<domain name>| <key1>=<value1>|<key2>=<value2>:<metric>

142 Enabling JMX Reporting

Java Agent Guide

For example, given an WebLogic MBean with these characteristics: Domain name
WebLogic

Key/Value Pairs
category=server, type=jdbc

Metric Names
connections

If no primary keys are specified in

introscope.agent.jmx.name.primarykeys, the MBean attributes in the table

above would be converted to the following Introscope metric:

<Domain>|<Host>|<Process>|<Agent>|JMX|Weblogic|category=server|t ype=jdbc:connections
Note that the key/value pairs are displayed alphabetically in the Introscope metric.

Using primary key conversion to streamline JMX Metrics

You can optionally configure the order in which metrics appear under the JMX node by defining, in the Agent profile, a Primary Keythose parts of an MBeans ObjectName that uniquely identify it. If you do not configure primary key conversion, Introscope converts the JMX data as described in Default JMX Metric conversion process on page 142. With the default conversion, metrics are listed alphabetically under the JMX node in Investigator. This method of converting JMX data to Introscope metrics results in streamlined metric names, and allows you to control order of key/value pair information in the generated metrics. The behavior is configured in the introscope.agent.jmx.name.primarykeys property in the Agent profile. Values in the primarykeys property should specify the parts of an MBeans JMX ObjectName that uniquely identify an MBean. For example, a WebLogic MBeans ObjectName contains a Type key that specifies the kind of MBean, and a Name key that specifies the name of the resource the MBean represents. The key/value pairs in an ObjectName can vary for different types of MBeans. Introscope converts and presents the MBeans identified by value of the

introscope.agent.jmx.primarykeys property according to these rules:

Only the key value information is displayed, not the key name. Values are ordered in the sequence defined in the primarykeys property.

Using primary key conversion to streamline JMX Metrics 143

CA Wily Introscope

Values are case-sensitive.

For example, given a WebLogic MBean with these characteristics: Domain name
WebLogic

MBean ObjectName Key/ Value Pairs

category=server,type=jdbc

JMX Attribute
connections

If you configure:

introscope.agent.jmx.name.primarykeys=type,category
The connections attribute appears in the Investigator tree in this structure:

<IntroscopeDomain>|<Host>|<Process>|<Agent>|JMX|Weblogic|jdbc|se rver:connections
Note WebLogic 9.0 does not have universally available primary keys, so for WebLogic 9.0 Introscope uses the key/value pair metric naming convention found in the Default Conversion Method described in Default JMX Metric conversion process on page 142. As a result, the JMX Metric tree for WebLogic 9.0 will have a different structure than the metric tree for other WebLogic versions.

Managing Metric volume with JMX filters

Defining JMX filters determines what JMX MBean information will be collected and displayed in Introscope. If no filters are set, all JMX MBean information will be reported by the Agent to the Enterprise Manager, increasing system overhead. Filters are set in the introscope.agent.jmx.name.filter property in the Agent profile file, IntroscopeAgent.profile. Filters are keywords, entered as comma-separated strings in the property. As of Introscope 6.1, filter strings can contain the asterisk (*) and question mark (?) wildcard characters. Introscope matches the filter strings to JMX-generated Introscope metrics. If it finds a match, the metrics that match are reported to Introscope. To limit the volume of metrics returned, define filter strings as narrowly as possible. For instance, if you define a filter string that matches an MBean attribute that exists on multiple MBeans, metrics from each of those MBeans will be reported. If you are only interested in an attribute on selected MBeans, you can qualify the attribute name with the MBean name in your filter string. For example, assume you wish to capture the MessagesCurrentCount attribute value for the JMSDestinationRuntime MBean.

144 Enabling JMX Reporting

Java Agent Guide

If the fully qualified metric name for MessagesCurrentCount is:

*SuperDomain*|host-name|Process|Agent-name|JMX|comp-1| JMSDestinationRuntime|comp-2:MessagesCurrentCount
define introscope.agent.jmx.name.filter in the Agent profile as:

JMX|comp-1|JMSDestinationRuntime|comp-2:MessagesCurrentCount

JMX filters for WebLogic

In the IntroscopeAgent.profile file shipped with WebLogic, the following keywords are already defined:
ActiveConnectionsCurrentCount WaitingForConnectionCurrentCount PendingRequestCurrentCount ExecuteThreadCurrentIdleCount OpenSessionsCurrentCount

Configuring JMX reporting

How you configure Introscope to support JMX depends upon the application server you use. This section describes how to configure Introscope to collect and present JMX data from WebLogic Server and WebSphere 5.0. To configure JMX reporting, you must complete the following steps in this order: 1 Enable JMX support in the Agent profile. 2 Define primary keys for JMX data conversion. 3 Define JMX filters. 4 Configure startup class or service. 5 Add permissions to Java 2 security policy (WebSphere 5.0.x). These are the steps. To
Enable JMX support in the Agent profile

Do this 1 Shut down the managed application if it is running. 2 For WebSphere Agents only, in IntroscopeAgent.profile set

introscope.agent.jmx.enable to true. (The default value is false in the WebSphere Agent profile.)

Configuring JMX reporting 145

CA Wily Introscope

To

Do this

Define primary 1 In IntroscopeAgent.profile, configure primary keys. keys for JMX For WebLogic 9.0, comment out: data introscope.agent.jmx.primarykeys conversion For other WebLogic versions, uncomment: introscope.agent.jmx.primarykeys For WebSphere 6.0, uncomment: introscope.agent.jmx.name.primarykeys=J2EEServer, Application, j2eeType,JDBCProvider,name,mbeanIdentifier 2 If you modify the value of the property, values must be casesensitive, and multiple keys must be separated by commas. 3 Continue to the next step, Define JMX filters. Define JMX filters

1 In IntroscopeAgent.profile, make sure the

introscope.agent.jmx.name.filter property is
2 Enter desired strings, separated by commas, in the property.
In order for Introscope to properly match filtered strings, the strings must be spelled exactly and case sensitive 3 Save changes. 4 Restart the managed application. uncommented.

Configure startup class or service

To enable JMX data, you must configure an Introscope startup class in WebLogic Server, or a custom service in WebSphere. For instructions, see Configuring startup class for WebLogic 8.1 or 9.0 on page 112.

Add If you use WebSphere 5.0.x, add these lines to the <WebSphere permissions to home>/properties/server.policy file: Java 2 security // permissions for Introscope JMX support policy grant codeBase "file:${was.install.root}/-" { (WebSphere permission com.tivoli.jmx.MBeanServerPermission "*"; 5.0.x) permission com.tivoli.jmx.MBeanPermission "*"; permission com.tivoli.jmx.AllMBeanPermission "*"; };

146 Enabling JMX Reporting

Java Agent Guide

Enabling JSR-77 data for WAS 6.x

This section provides instructions for configuring Introscope to collect, retain, and report metrics for JSR-77 JMX MBean objects under WebSphere 6.0 and later. JSR-77, the J2EE Management Specification, abstracts the manageable parts of the J2EE architecture and defines an interface for accessing management information. JSR-77 support requires a JVM version 1.4 or later. If the JVM is 1.4, the application server must also support JSR-77. Related Knowledge Base article(s):For more information about viewing JMX metrics on WAS 6.1, please see the Knowledge Base article Viewing JMX Metrics on WAS 6.1. To enable JSR-77 support: 1 Shut down the managed application if it is running. 2 Configure a WebSphere Custom Service, as described in Configuring a custom service in WebSphere 5.0 or 6.0 on page 113. 3 In the Agent profile, verify that:

introscope.agent.jmx.enable=true
4 In the Agent profile, enable JSR-77 by setting:

introscope.agent.jmx.name.jsr77.disable=false
5 Configure the primary keys method of metric conversion by uncommenting this property in the Agent profile:

introscope.agent.jmx.name.primaryKeys=J2EEServer,Application, j2eeType,JDBCProvider,name,mbeanIdentifier
Note Only the Agent profile provided with Introscope for WebSphere contains this property definition. For more information see Using primary key conversion to streamline JMX Metrics on page 143. 6 To specify the JSR-77 Metrics to report, uncomment and set this property to identify desired metrics:

introscope.agent.jmx.name.filter
Although filtering is not required, it is highly recommended. For more information see Managing Metric volume with JMX filters on page 144. 7 To specify specific Mbean attributes to exclude in JSR-77 metrics, uncomment this property, and update as desired to exclude additional attributes:

introscope.agent.jmx.ignore.attributes=server

Enabling JSR-77 data for WAS 6.x 147

CA Wily Introscope

148 Enabling JMX Reporting

SECTION

17

Configuring Platform Monitoring

This chapter has instructions for configuring Introscope Platform Monitors. Understanding platform monitors . Enabling platform monitors on AIX Disabling platform monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 150 150 151 152

Enabling platform monitors on Windows Server 2003

Troubleshooting platform monitoring

Configuring Platform Monitoring 149

CA Wily Introscope

Understanding platform monitors

Platform monitors enable the Agent to report system metrics, including CPU statistics, to the Enterprise Manager. Platform monitors are included with the Introscope Agent installers. Platform monitors on all operating systems except Windows Server 2003 and AIX are automatically enabled upon Agent installation. Windows Server 2003 and AIX platform monitors require a minimal configuration to work. Introscope can monitor these operating systems:
Solaris Windows Server 2003 Windows 2000 Professional/Server/Advanced Server/Datacenter Server Windows XP Professional AIX 4 or 5 RedHat Enterprise Linux 2.1 or 3.0

The platform metrics generated are:

ProcessID Processor Count - the number of CPUs Utilization % (process) - for the Agent process, the percentage of total capacity

of all processors this process is using. Regardless of how many processors there are, this metric generates only one number.
Utilization % (aggregate) - for this processor, its total utilization (as a

percentage) by all processes in the system. Each processor is shown as a Resource in the Investigator tree.

Enabling platform monitors on Windows Server 2003

To run platform monitors on Windows Server 2003, you must have admin privileges.

Enabling platform monitors on AIX

To enable platform monitors on AIX: 1 After Agent installation, make sure these files are installed in the wily/ext directory:
introscopeAIX4Stats.jar

150 Configuring Platform Monitoring

Java Agent Guide

libIntroscopeAIX4Stats.so introscopeAIX5Stats.jar libIntroscopeAIX5Stats.so

2 Install the Perfstat Library.

AIX 5: Install the patch/fix APAR IY30022 from IBM at:

http://www.ibm.com/support/docview.wss?uid=isg1IY30022

AIX 4.3.3 and higher: A Perfstat Library has been created to work with AIX

4.3.3. Install the following packages from ftp://ftp.software.ibm.com/aix/ fixes/v4/os:

bos.perf.libperfstat bos.perf.perfstat

AIX 4: Bring your system up to 4.3.3 and then install the above packages.

Note Restart your machine to ensure the patches have taken effect.

Disabling platform monitors

To disable platform monitors on any platform, move the .jar file from the /wily/ ext directory to another directory. This table shows the location of platform monitor files installed with an Agent installer. For this platform
Solaris Windows Server 2003 all Windows 2000 platforms XP Professional AIX

The platform monitor files are located in

wily/ext/introscopeSolarisStats.jar wily/ext/libIntroscopeSolarisStats.so wily\ext\introscopeWindowsStats.jar wily\ext\introscopeWindowsStats.dll

wily/ext/introscopeAIX4Stats.jar wily/ext/libIntroscopeAIX4Stats.so wily/ext/introscopeAIX5Stats.jar wily/ext/libIntroscopeAIX5Stats.so

RedHat Enterprise Linux

wily/ext/introscopeRedHatStats.jar wily/ext/libIntroscopeRedHatStats.so

Disabling platform monitors 151

CA Wily Introscope

Troubleshooting platform monitoring

In most cases, the platform monitor successfully detects the operating system and runs if the operating system is supported. In rare cases where this does not occur, you can explicitly specify your operating system in the Agent profile to ensure that the platform monitor runs. To specify your operating system in the Agent profile: 1 Open the Agent profile, usually IntroscopeAgent.profile. 2 Under the Platform Monitor Configuration heading, locate the introscope.agent.platform.monitor.system property and enter the value for your operating system. Available values are:
AIX Solaris Windows Linux RedHat 2.1 RedHat 3.0

3 Restart the managed application. On Windows platforms, the Java Agent logfile will sometimes contain an error similar to the following:
11/28/06 08:29:55 AM PST [ERROR] [IntroscopeAgent] An error occurred polling for platform data

If the error is infrequent, it is likely caused by a transient error originating from Windows itself, and is harmless. On platforms other than Windows, or in the case that the error happens all the time, this error indicates something more serious and should be reported to CA support for Introscope.

152 Configuring Platform Monitoring

SECTION

18

Configuring WebSphere PMI

This chapter has instructions for configuring the Introscope Agent to report WebSphere PMI metrics. Agent support for WebSphere PMI Enabling PMI in WebSphere . . . Configuring PMI in Introscope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 154 156 157

Viewing WebSphere Agent PMI data .

Configuring WebSphere PMI 153

CA Wily Introscope

Agent support for WebSphere PMI

Introscope can provide WebSphere performance data by extracting WebSphere Performance Monitoring Infrastructure (PMI) metrics via the PMI interface provided with WebSphere 4.0 and higher. You must first enable PMI data collection in WebSphere before the data will be available to Introscope. In WebSphere, all performance monitor settings are off by default. These PMI metrics can then be displayed as Introscope metrics. Users can filter which metric categories to bring into Introscope, depending on their needs. To enable PMI reporting:
enable PMI in WebSphere enable PMI in the Introscope Agent profile configure an Introscope Custom Service in WebSphere

Enabling PMI in WebSphere

This section describes how to turn on WebSphere Performance Monitor Settings. To enable PMI in WebSphere 6.0/5.0.x: 1 See your WebSphere 6.0/5.0.x documentation for instructions on enabling Performance Monitoring Settings, and enabling Performance Monitoring for each desired metric category. 2 If you are running WebSphere 5.0.x and 6.0, you must modify the Java 2 Security Policy for PMI. Edit the file, <WebSphere home>/properties/server.policy to include the lines:
// permissions for Introscope PMI support grant codeBase "file: <Introscope_Home directory>/-" { permission java.security.AllPermission; };

To enable PMI in WebSphere 4.0 (Distributed) 1 Open the WebSphere Administrative Console. 2 Select the server you'd like to configure.

154 Configuring WebSphere PMI

Java Agent Guide

3 Click the Services tab.

4 Select Performance Monitoring Settings in the list, then click Edit Properties.

5 Check the checkbox for Enable performance counter monitoring. The Counter Settings section will become available. 6 Click a category under the Performance tree, and click a radio button to select a monitoring level. Note A low level usually collects just enough data without creating a lot of system overhead. 7 Repeat to set monitoring levels for each desired metric category. 8 Click OK. 9 On the Services Tab, click Apply. To enable PMI information, you will also need to configure a Custom Service in WebSphere. See Agent support for WebSphere PMI on page 154.

Enabling PMI in WebSphere 155

CA Wily Introscope

Using PMI with Introscope on z/OS

There are several ways to obtain additional WebSphere-specific performance metrics on z/OS. One solution is to use Wilys PowerPack for z/OS WebSphere product, which provides WebSphere-specific PBDs and metrics. This product uses Wily tracer technology and is a low overhead method of obtaining WebSpherespecific metrics, and does not require you to enable PMI in WebSphere for z/OS. In addition, Introscope supports PMI on WebSphere z/OS, but this approach consumes more system resources.

Configuring PMI in Introscope

After you turn on Performance Monitoring Settings in WebSphere, you must enable PMI data collection in Introscope, and enable the metric categories youd like to see reported. To configure PMI collection, use the following steps: 1 Shut down your managed application. 2 Open the Agent profile, usually IntroscopeAgent.profile. 3 Locate the property, introscope.agent.pmi.enable, under the WebSphere PMI Configurations heading, and verify it is set to true.

156 Configuring WebSphere PMI

Java Agent Guide

4 Introscope can report data from the following high-level PMI metric categories. These categories are represented by commented-out properties under the WebSphere PMI Configuration heading. Four categoriesthreadPool, servletSessions, connectionPool, and j2care set to true by default. WebSphere 4.0 PMI Categories
introscope.agent.pmi.enable.threadPool=true introscope.agent.pmi.enable.servletSessions=true introscope.agent.pmi.enable.connectionPool=true introscope.agent.pmi.enable.j2c=true introscope.agent.pmi.enable.bean=false introscope.agent.pmi.enable.transaction=false introscope.agent.pmi.enable.webApp=false introscope.agent.pmi.enable.jvmRuntime=false introscope.agent.pmi.enable.jvmpi=false JVMPI must be turned on in WebSphere.

WebSphere 5.0.x PMI Categories

introscope.agent.pmi.enable.threadPool=true introscope.agent.pmi.enable.servletSessions=true introscope.agent.pmi.enable.connectionPool=true introscope.agent.pmi.enable.j2c=true introscope.agent.pmi.enable.bean=false introscope.agent.pmi.enable.transaction=false introscope.agent.pmi.enable.webApp=false introscope.agent.pmi.enable.jvmRuntime=false introscope.agent.pmi.enable.jvmpi=false introscope.agent.pmi.enable.cache=false introscope.agent.pmi.enable.orbPerf=false introscope.agent.pmi.enable.j2c=false introscope.agent.pmi.enable.webServices=false introscope.agent.pmi.enable.wlm=false

Note: For data to be provided to the jvmpiModule, introscope.agent.pmi.enable.system=false

5 For each high level metric category you want to report, enter a value of true. 6 Save the changes. 7 Restart the managed application.

Viewing WebSphere Agent PMI data

After youve enabled PMI collections in Introscope, available PMI metrics will be displayed in the following location in the Investigator tree:

<Domain>|<Host>|<Process>|<Agent>|WebSpherePMI

Viewing WebSphere Agent PMI data 157

CA Wily Introscope

158 Configuring WebSphere PMI

SECTION

19

Enabling WebLogic Diagnostic Framework

This chapter has instructions for configuring the Introscope Agent to report WebLogic Diagnostic Framework (WLDF) metrics. Agent support for WebLogic Diagnostic Framework (WLDF) Understanding WLDF Metric conversion . Enabling WLDF reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 160 161

Enabling WebLogic Diagnostic Framework 159

CA Wily Introscope

Agent support for WebLogic Diagnostic Framework (WLDF)

The WebLogic Diagnostic Framework (WLDF) is a monitoring and diagnostic framework that defines and implements a set of services that run within the WebLogic Server process and participate in the standard server life cycle. Using WLDF, you can create, collect, analyze, archive and access diagnostic data generated by a running server and the applications deployed within its containers. This data provides insight into the run-time performance of servers and applications and enables you to isolate and diagnose faults when they occur. WLDF is a new feature in WebLogic 9.0. In previous releases of WebLogic Server, access to diagnostic data by monitoring agentswhich were developed by customers or third-party tools vendorswas limited to JMX attributes, and changes to monitoring agents required server shutdown and restart. However, WLDF enables dynamic access to server data through standard interfaces, and the volume of data accessed at any given time can be modified without shutting down and restarting the server. For more information on WLDF, see http://e-docs.bea.com/wls/docs90/ wldf_configuring/index.html.

Understanding WLDF Metric conversion

Introscope WLDF Metric conversion is similar to that in JMX metric conversion. Where JMX MBeans have multiple Attributes (metrics), WLDF has a set of Data Accessors, each with multiple Columns (metrics). Introscope converts WLDF Columns to Introscope metrics. Information in Data Accessors is defined by a domain name and one or more key/ value pairs. Introscope converts this WLDF information into Introscope-specific metric format and displays it in the Investigator under the following Resource:
<Domain>|<Host>|<Process>|<Agent>|WLDF|

Introscope converts Data Accessor Columns using the following method:

key and value information is displayed key/value pairs are placed in alphabetical order in the Introscope-generated

metrics. The following example shows the syntax used:

<Domain>|<Host>|<Process>|<Agent>|WLDF|<domain name>|<key1>=<value1>|<key2>=<value2>:<metric>

160 Enabling WebLogic Diagnostic Framework

Java Agent Guide

For example, this table shows the information for the BYTECOUNT Column from the HTTPAccessLog Data Accessor: Domain name
WebLogic

Key/Value pairs
Name=HTTPAccessLog, Type=WLDFDataAccessRuntime=Accessor, WLDFRuntime=WLDFRuntime

Metric names
BYTECOUNT

The Data Accessor information in the table above would be converted to the following Introscope metric:
<Domain>|<Host>|<Process>|<Agent>|WLDF|Weblogic|Name=HTTPAccessLog |Type=WLDFDataAccessRuntime=Accessor|WLDFRuntime=WLDFRuntime:BYTECOUNT

Note that the key/value pairs are displayed alphabetically in the Introscope metric.

Enabling WLDF reporting

By default, WLDF reporting is not enabled in Introscope. To obtain WLDF data: 1 Shut down the managed application if it is running. 2 Configure a WebLogic Startup Class, as described in Application Server management data and the Agent on page 112. 3 In the Agent profile:
introscope.agent.wldf.enable=true

Enabling WLDF reporting 161

CA Wily Introscope

162 Enabling WebLogic Diagnostic Framework

SECTION

Wily Knowledge Base Articles

This appendix contains information about the Wily Java Agent.

The Wily Java Agent

You can use the Wily Java Agent to find the most up to date information and solutions regarding your day-to-day use of Introscope and related products. The Wily Java Agent contains troubleshooting information, articles on common issues, and white papers on a range of Introscope subjects.

Accessing the Java Agent

To access the Wily Java Agent, you will need a username and password. Contact Wily support for more information about obtaining a Java Agent username and password. The Wily Java Agent is accessed here: http://support.wilytech.com

Searching and Browsing Java Agent Articles

You can search the Java Agent for answers to questions about specific Wily products, or you can browse through different articles. Browsing the Most Popular Answers before searching the Java Agent may address your current needs. The Most Popular Answers are accessed through the Browse button of the Java Agent. Below are the top 10 most popularly searched Java Agent articles:

Top 10 Most Popular Java Agent Articles

Introscope Master Patch List CA-Wily Division FTP Server Introscope Documentation List Purging or archiving Enterprise Manager database tables Introscope 7.0 Documentation Set

Wily Knowledge Base Articles 163

CA Wily Introscope

JDBC Drivers, DataSources, and Introscope and SQL Agent Metrics Setting Database Properties (JDBC Driver Configuration) for the Enterprise

Manager
Introscope 5.0.2 Documentation Set Introscope 7.1 Documentation Set Introscope 6.0 Documentation Set

164 Wily Knowledge Base Articles

SECTION

Agent Properties

This appendix documents the properties in the IntroscopeAgent.profile. This section includes these topics: Configuring IntroscopeAgent.profile location . Agent naming properties Wily CEM integration . . Agent failover properties Error Detector properties Extensions properties JMX properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 167 169 169 169 169 170 170 172 173 174 175 176 177 177 177 177 178 178 180 182 183 183

Agent to Enterprise Manager connection properties .

Transaction tracing properties . LeakHunter properties

Platform monitoring properties WebSphere PMI properties . Socket metrics properties . SQL Agent properties. . . . . . . . . Stall metrics properties . URL grouping properties. WLDF metrics properties AutoProbe properties . Logging properties Blame properties . . . . . .

Agent thread priority properties .

CPU Utilization Properties .

Cross-process tracing in WebLogic Server .

Note In these files, on Windows machines, when adding a path, you must escape a backslash (\) with another backslash (each one doubled), such as C:\\Introscope\\lib\\Agent.jar.

Agent Properties 165

CA Wily Introscope

Configuring IntroscopeAgent.profile location

The Agent refers to properties in the Agent profile IntroscopeAgent.profilefor its basic connection and naming properties. When you install an Agent, the Agent profile is installed in the <AppServerHome>/wily directory. Introscope looks for the Agent profile in these locations, in this sequence:
location defined in the system property

com.wily.introscope.agentProfile
location defined in com.wily.introscope.agentResource <working directory>/wily directory

To locate the Agent profile in a location other than the <AppServerHome>/wily directory: 1 Define the new location using one of these methods:
define a system property on the Java command line with the -D option to

specify the full path to the location of the IntroscopeAgent.profile file:

com.wily.introscope.agentProfile
Make the Agent profile (IntroscopeAgent.profile) available in a resource

on the classpath. Set com.wily.introscope.agentResource to specify the path to the resource containing the Agent profile. Note If you use this option, AutoProbe will not be able to write to AutoProbe.log. See Managing the ProbeBuilder Log on page 99. 2 Move your ProbeBuilder directives (.pbd and .pbl files) to the same location as the Agent profilethey are referenced relative to the profile location. 3 For Sun ONE only, add the new location of the Agent profile to the Sun One server.xml file: a To add Introscope information to startup scripts for Sun ONE 7.0, log in as Administrator or Root. b Open the server.xml file, in SunOneHome/domains/domain1/server1/

config/
c Add a line to the jvm-options stanza in server.xml:
<jvm-options> -Dcom.wily.introscope.agentProfile=SunOneHome/wily/ IntroscopeAgent.profile </jvm-options>

166 Agent Properties

Java Agent Guide

Agent naming properties

The following sections define the properties related to the Agent name. For more information on Agent naming, see Java Agent Naming on page 83. introscope.agent.agentAutoNamingEnabled Usage
Specifies whether Agent Autonaming will be used to obtain Agent name for supported application servers. Requires Startup Class to be specified for WebLogic; requires Custom Service to be specified for WebSphere.

Options Default Example Notes

CO1-Code1 default here
Set to false and commented out in example

IntroscopeAgent.profile
Set to true, and not commented out in Agent profiles shipped with

supported application servers

introscope.agent.agentAutoNamingMaximumConnectionDelayIn Seconds Usage Options Default Example Notes

120 introscope.agent.agentAutoNamingMaximumConnectionDelayIn Seconds=120 Specifies the amount of time in seconds the Agent waits for naming information before connecting to the Enterprise Manager.

introscope.agent.agentAutoRenamingIntervalInMinutes
Specifies the time interval in minutes during which the Agent will check to see if it has been renamed. Default: 10

introscope.agent.disableLogFileAutoNaming
Disables automatic naming of an Agents log filesthe default behavior when an Agent is configured for autonaming.

introscope.agent.agentName
Name of Agent. Default:
In the Agent profile provided with application server-specific Agent installers,

Agent naming properties 167

CA Wily Introscope

the default reflects the application server, for instance WebLogic Agent.
In the Agent profile provided with default Agent installer, the property value is

Unknown Agent, and the line is commented out.

Note For any installation, if the value of this property is invalid or if this property is deleted from the profile, the Agent name will be Unknown Agent.

introscope.agent.agentNameSystemPropertyKey
Specifies which Java system property will contain Agent name. Default: blank

introscope.agent.clonedAgent
Set to true when running identical copies of an Application on the same machine. Default: false

introscope.agent.customProcessName
Name for process being monitored. Default:
In the Agent profile provided with application server-specific Agent installers,

the default reflects the application server, for instance WebLogic.

In the Agent profile provided with default Agent installer, the property is

commented out.

introscope.agent.defaultProcessName
Default name for an Agent if introscope.Agent.customProcessName is not set and the name cannot be inferred from the main class of the application. Default: UnknownProcess; may vary by installer

introscope.agent.disableLogFileAutoNaming
Specifies whether to disable automatic naming of Agent log files when using AutoNaming options. Default: false

168 Agent Properties

Java Agent Guide

Wily CEM integration

For information on configuring Java Agents for Wily CEM Integration, see the CA Wily CEM Integration Guide.

Agent failover properties

introscope.agent.enterprisemanager.connectionorder
The connection order of backup Enterprise Managers the Agent uses if it is disconnected from its default Enterprise Manager. Items in list must be commaseparated.

introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds
Number of seconds between attempts by Agent to reconnect to primary Enterprise Manager. Default: Commented out; 120

Agent to Enterprise Manager connection properties

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT
The host name of machine running the Enterprise Manager. Default: localhost

introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT
The port on the Enterprise Manager machine that listens for the Agent. Default: 5001

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT
Change this property to use a different client socket factory. Default:

com.wily.isengard.postofficehub.link.net.DefaultSocketFactory

Error Detector properties

introscope.agent.errorsnapshots.enable
Enable the Agent to captures transaction details about serious errors.

Wily CEM integration 169

CA Wily Introscope

Note Requires Introscope Error Detector. Default: true

introscope.agent.errorsnapshots.throttle
The maximum number of error snapshots that the Agent can send in a 15-second period. Note Requires Introscope Error Detector. Default: 10, commented out.

introscope.agent.errorsnapshots.ignore.<index>
This indexed property allows you to specify error messages to ignore. Error snapshots will not be generated or sent for errors with messages matching these filters. You may specify as many as you like (using .0, .1, .2 ...). You may use wildcards (*). The following are examples only. Note Requires Introscope Error Detector. Default: Example definitions are provided, and commented out, as shown below.
#introscope.agent.errorsnapshots.ignore.0=*com.company.HarmlessException* #introscope.agent.errorsnapshots.ignore.1=*HTTP Error Code: 404*

Extensions properties
introscope.agent.extensions.directory
Specifies the location of all extensions to be loaded by the Introscope Agent. Nonabsolute names are resolved relative to the location of this properties file Default: ext

Transaction tracing properties

introscope.agent.transactiontracer.parameter.httprequest.headers
Specifies (in comma-separated list) HTTP request header data to capture. Comma separated list. The Agent profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the desired header names.

170 Agent Properties

Java Agent Guide

introscope.agent.transactiontracer.parameter.httprequest.parameters
Specifies (in comma-separated list) HTTP request parameter data to capture. Default: The Agent profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the desired parameter names.

introscope.agent.transactiontracer.parameter.httpsession.attributes
Specifies (in comma-separated list) HTTP session attribute data to capture. Default: The Agent profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the desired attribute name.

introscope.agent.transactiontracer.userid.key
User-defined key string This property statement is commented out in the Agent profile. Uncomment it and supply the correct value if, in your environment, user IDs are accessed using HttpServletRequest.getHeader or HttpServletRequest.getValue For more information, see

introscope.agent.transactiontracer.userid.method, below.

introscope.agent.transactiontracer.userid.method
Specifies the method that returns User IDs. Allowable values are:
HttpServletRequest.getRemoteUser HttpServletRequest.getHeader HttpServletRequest.getValue

The Agent profile includes a commented out property definition for each of the three allowable values. Uncomment the appropriate statement, based on whether user ID is accessed by getRemoteUser, getHeader, or getValue.

Transaction tracing properties 171

CA Wily Introscope

introscope.agent.transactiontrace.componentCountClamp
Introscope now sets a clamp (set by default to 5,000) to limit the number of traces. When this limit is reached, warnings appear in the log, and the trace stops. The problem was an infinitely expanding transactionfor example when a servlet executes hundreds of object interactions and back-end SQL calls. Transaction Tracer viewed this as one transaction, continuing infinitely. There was an upper limit as the engine did eventually, finish but the JVM would run out of memory beforehand.

JMX properties
introscope.agent.jmx.enable
Enables collection of JMX Metrics. Values are true and false. Default: Varies by Agent version.

introscope.agent.jmx.ignore.attributes
Controls which (if any) JMX MBean attributes are to be ignored. A commaseparated list of desired keywords. If an MBean attribute name matches one in this list then it will be ignored. Leave empty to include all MBean attributes. Default: Commented out; server

introscope.agent.jmx.name.filter
Comma-separated list of filter strings that determine what JMX data Introscope collects and displays. Introscope reports JMX-generated metrics that match the a filter string. Filter strings can contain the asterisk (*) and question mark (?) wildcard characters. * matches zero or more characters ? matches a single character. To match a literal * or ?, escape the character with \\. Examples: ab\\*c matches a metric name that contains ab*c ab*c matches a metric name that contains abc, abxc, abxxc etc. ab?c matches a metric name that contains abxc

172 Agent Properties

Java Agent Guide

ab\\?c matches a metric names that contains ab?c where x is any character. Default:
Commented out. For WebLogic:

ActiveConnectionsCurrentCount,WaitingForConnectionCurrentCount ,PendingRequestCurrentCount,ExecuteThreadCurrentIdleCount,Open SessionsCurrentCount,j2eeType

introscope.agent.jmx.name.jsr77.disable
ADDED in Introscope 6.1, controls whether or not Introscope collects and reports full JSR77 data, including complex JMX data. To enable JSR 77 reporting, set this property to false, and set introscope. agent.jmx.enable to true. Default: true

introscope.agent.jmx.name.jsr77.enable
REMOVED in Introscope 6.1.

introscope.agent.jmx.name.primarykeys
User-defined order of MBean information, and simplifies name conversion. Default:
Commented out in default IntroscopeAgent.profile file For WebLogic: Type,Name For WebSphere:

J2EEServer,Application,j2eeType,JDBCProvider,name,mbeanIdentif ier
Note Comment out this property if using WebLogic Server 9.0.

LeakHunter properties
introscope.agent.leakhunter.collectAllocationStackTraces
Specifies whether to collect allocation stack trace information. Turning on this option has the potential to create higher system overhead, in CPU usage and memory. Default: false

LeakHunter properties 173

CA Wily Introscope

introscope.agent.leakhunter.enable
Enables LeakHunter functionality. Default: true

introscope.agent.leakhunter.leakSensitivity
Sensitivity of LeakHunter (higher sensitivity will report more collections as potential leaks). Must be integer value from 1-10. Default: 5

introscope.agent.leakhunter.logfile.append
Specifies whether to replace log file (value of false) or append to existing log file (value of true) on application restart. Default: false

introscope.agent.leakhunter.logfile.location
Location of LeakHunter.log file. Filename is relative to the directory that contains the agent profile. If property is commented out or left blank, no log file will be written. Default: LeakHunter.log

introscope.agent.leakhunter.timeoutInMinutes
Period (in minutes) during which Introscope LeakHunter looks for new potential leaks. Must be non-negative integer. Value of zero means no timeout. Default: 120 minutes

Platform monitoring properties

introscope.agent.platform.monitor.system
Name of operating system to load a platform monitor for. Allowable values: Solaris, RedHat2.1, RedHat3.0, Windows, AIX, Linux Default: Commented out; Solaris

174 Agent Properties

Java Agent Guide

WebSphere PMI properties

introscope.agent.pmi.enable
Enables collection of data from WebSphere PMI. Default: true

introscope.agent.pmi.enable.bean
Enables collection of PMI bean data. Default: false

introscope.agent.pmi.enable.cache
WebSphere 5.0.x only; enables collection of data about the effectiveness of WebSphere caching layers. Default: false

introscope.agent.pmi.enable.connectionPool
Enables collection of PMI connectionPool data. Default: true

introscope.agent.pmi.enable.j2c
WebSphere 5.0.x only; enables collection of J2EE connector data. Default: false

introscope.agent.pmi.enable.jvmpi
Enables collection of PMI jvmpi data. For data to be provided to this module, JVMPI must be turned on in WebSphere. Default: false

introscope.agent.pmi.enable.jvmRuntime
Enables collection of PMI jvm Runtime data. Default: false

introscope.agent.pmi.enable.orbPerf
WebSphere 5.0.x only; enables collection of performance statistics about the embedded Object Request Broker (ORB).

WebSphere PMI properties 175

CA Wily Introscope

Default: false

introscope.agent.pmi.enable.servletSessions
Enables collection of PMI servletSessions data. Default: true

introscope.agent.pmi.enable.system
WebSphere 5.0.x only; enables collection of operating system-level data. Default: false

introscope.agent.pmi.enable.threadPool
Enables collection of PMI threadPool data. Default: true

introscope.agent.pmi.enable.transaction
Enables collection of PMI transaction data. Default: false

introscope.agent.pmi.enable.webApp
Enables collection of PMI webApp data. Default: false

introscope.agent.pmi.enable.webServices
WebSphere 5.0.x only; enables collection of SOAP and web services data. Default: false

introscope.agent.pmi.enable.wlm
WebSphere 5.0.x only, enables collection of Workload Management (WLM) data on load balancing and failover of WebSphere applications. Default: false

Socket metrics properties

introscope.agent.sockets.reportRateMetrics
Enables reporting of individual Socket's Input/Output Bandwidth rate metrics. Default: true

176 Agent Properties

Java Agent Guide

SQL Agent properties

introscope.agent.sqlagent.sql.useblame
Specifies whether SQL Agent will generate blame metric data. Default: Commented out; true

introscope.agent.sqlagent.sql.maxlength
Limits how much of a SQL statement appears in the Investigator tree for SQL Agent metrics, in bytes. Default: 990 Note Does not appear in Agent profile. To change the value, add the property to the Agent profile.

Stall metrics properties

For more information on stall metric properties, see Disabling the capture of stalls as Events on page 128

introscope.agent.stalls.thresholdseconds
Default: 30

introscope.agent.stalls.resolutionseconds
Default: 10

introscope.agent.stalls.enable
Default: true

Agent thread priority properties

introscope.agent.thread.all.priority
Controls the priority of Agent threads. You can set this from 1 (low) to 10 (high). Default: Commented out; And, if unspecified, is level 5.

URL grouping properties

These properties are for configuring URL Groups for frontend metrics. For more information, see Using URL groups on page 116.

SQL Agent properties 177

CA Wily Introscope

introscope.agent.urlgroup.keys
If an URL address belongs to two URL Groups, the order in which you list the keys for the URL Groups in the keys property is important. The URL Group defined by the narrower pattern should precede the URL Group specified by the broader pattern. For example, if the URL Group with key alpha contains a single address, and the URL Group with key beta includes all addresses on the network segment that contains the address in the first URL Group, alpha should precede beta in the keys parameter. Default: default

introscope.agent.urlgroup.group.default.pathprefix
Default: *

introscope.agent.urlgroup.group.default.format
Default: Default

WLDF metrics properties

introscope.agent.wldf.enable
Enables collection of WLDF metrics. Default: false

AutoProbe properties
introscope.autoprobe.directivesFile
Specifies Directives files for AutoProbe. For more information, see ProbeBuilder Directives on page 33. Default: varies by installer

introscope.autoprobe.enable
When option is set to false, this option turns off the automatic insertion of Probes into the applications bytecode, it does not turn off the Agent or Agent reporting. Default: true

introscope.autoprobe.logfile
Name and location of AutoProbe log file.

178 Agent Properties

Java Agent Guide

Default: AutoProbe.log

introscope.autoprobe.dynamicinstrument.enabled
Enables dynamic ProbeBuilding for Agents that run under JDK 1.5, and use AutoProbe. Default: false Note For more information about dynamic instrumentation, see Dynamic ProbeBuilding (JVM 1.5) on page 28.

introscope.autoprobe.dynamicinstrument.instrumentList
For Agents that run under JDK 1.5, use AutoProbe, and use dynamic instrumentation, this property specifies one or more sets of classes to include for dynamic instrumentation, in terms of the package prefixes. Specify all or a portion of a package name, starting with the beginning of the package name. Wildcards are not supported. For example:

introscope.autoprobe.dynamicinstrument.instrumentList=com.xyz, com.foo.ejb.session
To enable all classes to be re-transformed, use the string all. Default: no classes are dynamically instrumented. Note For more information about dynamic instrumentation, see Dynamic ProbeBuilding (JVM 1.5) on page 28.

introscope.autoprobe.dynamicinstrument.avoidClassLoaders
For Agents that run under JDK 1.5, use AutoProbe, and use dynamic instrumentation, this property specifies one or more sets of classes to exclude from dynamic instrumentation, in terms of the classloader that loads the classes. Specify a comma-separated list of classloader class names. All classes loaded by the specified classloader(s) will then be excluded from dynamic instrumentation. Use the value System to specify the system classloader. This setting:
introscope.autoprobe.dynamicinstrument.avoidClassLoaders=System,com.bea. ClassLoader,com.mycl.ClassLoader

excludes classes that are loaded by the following classloaders:

the system classloader

AutoProbe properties 179

CA Wily Introscope

com.bea.ClassLoader com.mycl.ClassLoader

Default: None. Note For more information about dynamic instrumentation, see Dynamic ProbeBuilding (JVM 1.5) on page 28.

autoprobe.dynamicinstrument.pollIntervalMinutes
For Agents that run under JDK 1.5, use AutoProbe, and use dynamic instrumentation, this property determines the frequency with which the Agent polls for new and changed PBDs. Default: 1.

introscope.autoprobe.hierarchysupport.enabled
For Agents that run under JDK 1.5, use AutoProbe, and use dynamic instrumentation, you can use this property to enable instrumentation of classes that extend a supertype or interface. Default: False.

introscope.autoprobe.hierarchysupport.runOnceOnly
If you have enabled instrumentation of classes that extend a supertype or interface, you can use this property to control whether the utility that enables the feature runs only once, or at a specified interval. Default: True. Note Logging properties related to dynamic instrumentation are defined in the following section.

Logging properties
log4j.appender.logfile.File
Specifies the name and location of Introscope Agent logfile. Default: IntroscopeAgent.log

log4j.logger.IntroscopeAgent
Amount of logging detail for the Introscope Agent log. Level of detail value can be: INFO or

VERBOSE#com.wily.util.feedback.Log4JSeverityLevel.

180 Agent Properties

Java Agent Guide

Destination value can be console or logfile, or both. Default setting is INFO, console, logfile.

log4j.logger.IntroscopeAgent.inheritance
Controls log level and destination for log messages about classes that require instrumentation. Default. None. To configure logging of classes that have not been instrumented because they extend a supertype or interface set to: INFO, pbdlog Note For information about inheritance class logging see Controlling directive logging on page 31.

log4j.appender.pbdlog.File
Identifies log file for messages about classes that require instrumentation. Default. None. To configure logging of classes that have not been instrumented because they extend a supertype or interface set to: pbdupdate.log

log4j.appender.pbdlog
Specifies package for logging messages about classes that require instrumentation. Default. None. To configure logging of classes that have not been instrumented because they extend a supertype or interface set to
com.wily.introscope.agent.AutoNamingRollingFileAppender

log4j.appender.pbdlog.layout
Specifies rules for logging messages about classes that require instrumentation. Default. None. To configure logging of classes that have not been instrumented because they extend a supertype or interface set to
com.wily.org.apache.log4j.PatternLayout

log4j.appender.pbdlog.layout.ConversionPattern
Specifies rules for logging messages about classes that require instrumentation. Default. None. To configure logging of classes that have not been instrumented because they extend a supertype or interface set to
%d{M/dd/yy hh:mm:ss a z} [%-3p] [%c] %m%n

Logging properties 181

CA Wily Introscope

log4j.additivity.IntroscopeAgent.inheritance
Causes the direcitves for multiple level inheritance to be logged only in the pbdupdate.log file. Default: True. To configure the logging of multiple level inheritance direcitves in the pbdupdate.log only, add this property to the agent profile and set to false.

Blame properties
introscope.agent.blame.type
Add this property with a value of standard to turn off boundary blame. For information about boundary blame, see the Introscope Workstation User Guide.

Agent HTTP Tunneling Properties

You can configure Agents to send information using tunneling technology, enabling Agents to connect to an Enterprise Manager remotely. To use this feature, the Agent must be configured to connect to the Enterprise Managers embedded Web server, where the HTTP tunneling Web service is hosted. To configure HTTP tunneled communication in IntroscopeAgent.profile as a new Agent connection, specify:
The host name of machine running the Enterprise Managersee

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT on page 169.

The connection port to the Enterprise Manager Web server. This is the value

for the introscope.enterprisemanager.webserver.port property specified in the IntroscopeEnterpriseManager.properties for the Enterprise Manager to which the Agent will connect. See the Introscope Properties Files section of the Introscope Configuration and Administration Guide for information about introscope.enterprisemanager.webserver.port.
The HTTP tunneling socket factory. Specify this client socket factory:

com.wily.isengard.postofficehub.link.net.HttpTunnelingSocketFactory

Agent HTTP Tunneling Proxy Server Properties

Note These properties only apply to Agents configured to tunnel over HTTP.

introscope.agent.enterprisemanager.transport.http.proxy.host
Specify the proxy server host name.

182 Agent Properties

Java Agent Guide

introscope.agent.enterprisemanager.transport.http.proxy.port
Specify the proxy server port.

introscope.agent.enterprisemanager.transport.http.proxy.username
If the proxy server requires the Agent to authenticate it, specify the username for authentication.

introscope.agent.enterprisemanager.transport.http.proxy.password
If the proxy server requires the Agent to authenticate it, specify the password for authentication

CPU Utilization Properties

introscope.agent.disableAggregateCPUUtilization
When multiple Java Agents are deployed on a single machine, and when CPU overhead is high, the above agent propertycan be set to true to reduce overall CPU overhead on the machine.

Cross-process tracing in WebLogic Server

introscope.agent.weblogic.crossjvm
Default: true This property is commented out by default in the IntroscopeAgent.profile.

CPU Utilization Properties 183

CA Wily Introscope

184 Agent Properties

SECTION

Using the Introscope PBD Generator

You can use the Wily PBD Generator tool to instrument custom Java class files for use by Wily agents. This section includes these topics: About the Wily PBD Generator . Configuring the PBD Generator Using the PBD Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 186 187

Using the Introscope PBD Generator 185

CA Wily Introscope

About the Wily PBD Generator

The Wily PBD Generator utility can create a PBD file from Javadoc tags with which you have annotated your Java code, to facilitate the instrumentation of custom Java class files for use by the Wily Agent. The PBD Generator examines a set of Java source files, and instruments the methods in the classes that contain the Javadoc tag @instrument. Using the PBD Generator tool, you can:
automate building of PBD files, to eliminate potential for errors that might be

introduced by creating PBD files manually.

integrate PBD generation into your build systems to create and update PBD

files automatically and incorporate any changes to the Java source. You configure the PBD Generator by integrating it into an Apache Ant target using the WilyPBDGenerator.jar file, then running it as an Ant Javadoc task.

Configuring the PBD Generator

This tool is intended to be incorporated into Ant-based build systems, as a javadoc task in an Ant target. This sample Javadoc task illustrates the use of this tool in Ant:
<javadoc sourcepath="/src/engineering/products/introscope/source" destdir="/src/engineering/products/introscope/source/generatedpbd" maxmemory="512m" packagenames="com.wily.introscope.console.thornhill.ui.util" verbose="false" private="true"> <doclet name="com.wily.util.build.javadoc.PBDInstrumentDoclet" path="/Wily/tools/WilyPBDGenerator.jar"> <param name="-d" value="/src/engineering/products/introscope/source/ generatedpbd"/> </doclet> </javadoc>

Required PBD Generator parameters

These key PBD Generator parameters are required: This parameter
sourcepath destdir

Determines
the root directory of the Java source tree the directory path of the PBD file that will be output from the tool

186 Using the Introscope PBD Generator

Java Agent Guide

This parameter
packagenames doclet path param name="-d"

Determines
a comma-separated list of the Java packages to be examined for instrumentation the path to find the PBD Generator jar file, which contains this tool this must contain the same value as destdir

Using the PBD Generator

Before you can use the PBD Generator, you insert special Javadoc tags into the Java source files to be instrumented. The syntax for the JavaDoc tag is:
@instrument <valid metric prefix> <optional tracer name>

where:

<valid metric prefix> is any valid Introscope metric prefixa string without
a colon character (:). Pipe characters (|) are acceptable.

<optional tracer name> can be BlamePointTracer, FrontendMarker or BackendMarker. The default is BlamePointTracer if the tracer name is missing.

Using the PBD Generator 187

CA Wily Introscope

188 Using the Introscope PBD Generator

I ND E X

Index

A
Agent Basic Implementation 7 cloned Agent naming 92 configuration 5 verbose mode 97 visibility vs. overhead 6 configuring Agent name 18 Custom Process name 18 data collection options 11 JMX and JSR-77 11 Platform Monitoring 11 PMI 11 Socket Metrics 11 SQL Agent 12 Stall Event Reporting 11 Transaction Tracing Behavior 11 URL Groups for Blame Reporting 11 installation packages for application servers 16 installer 16 installer archive 16 logging options 10 name 18 naming options automatic automatically from application server 89 using Java system property 88 using system property key 88 redirecting output to a file 97 resolving Agent naming conflicts 86 Unistall 25 verbose mode 97 virtual 10 Agent failover

Domain/User configuration 108 Agent.jar 8 AIX platform monitors 150 application server installation packages 16 Oracle 10g 13 SAP NetWeaver 13 Sun ONE 12 support 12 WebLogic 12 WebSphere 12 WebSphere on z/OS 12 autonaming 18 AutoProbe 13, 18 Application Server AutoProbe with Oracle 10g 66 with Sun ONE 65 with WebLogic 60 configuring for Sun, IBM, or HP JVM configuring JRockit JVM 24 configuring JVM 19 configuring JVM 5.0 for 25 Create Connector 19 run Connector 19 AutoProbe Connector NetWeaver 04 23 Oracle 10g 24 Other 24 SAP J2EE 6.20 22 SAP J2EE 6.40 23 Sun ONE 23 WebLogic 20 WebSphere 4.0 20 WebSphere 5.0, 5.1, or 6.0 21

19

Index 189

CA Wily Introscope Local Product

B
backend 41, 42 default 42 BackendMarker 42 blame stack 42 Blame Technology 41 blame tracers 42 boundary blame 41, 42 BRTA 42 bytecode 32

dynamic instrumentation

28, 29

E
EJB 40, 41 subclasses 41 EJBs 34 Enterprise JavaBeans. See EJBs. Enterprise Manager 4, 10, 13, 17 alternate communication channel 18 clustering 13 default communication channel 17 ErrorDetector 6 Extensible Markup Language (XML) 34

C
CEM 12 Integration 12 CEMTracer.jar 8 class hierarchy 30, 35 Classes 43 subclasses 30 multiple levels probed 30 uninstrumented subclasses 30, 31 Combined Counter Tracer 48 Command-Line ProbeBuilder commands 73 configuring other Java applications 72 Common Object Request Broker Architecture 34 configuring agent name 18 JRockit JVM for AutoProbe 24 JVM 5.0 for AutoProbe 25 JVM AutoProbe 19 ProbeBuilder options 18 Sun, IBM, or HP JVM for AutoProbe 19 CORBA 34 Counter Tracer 47 CreateAutoProbeConnector.jar 8 Custom Method Tracers 40 Custom Process name 18 Custom Tracers 43 custom Tracers. See ProbeBuilder Directives Customer Experience Manager. See CEM. CustomTracerExample.pbd 43

F
File Systems 34 FrontendMarker 42 frontends 41, 42 default 42

H
HP configuring AutoProbe for HP Hotspot 1.3 13 19

I
IBM configuring AutoProbe for 19 IBM JVM 13 Identify keywords 40 installation archive packages 16 disk space requirement 14 Installation Archives Application Server Oracle 10g 17 Other 17 SAP NetWeaver 17 Sun ONE 16 WebLogic 16 WebSphere 16 Installation Directories 8 wily Directory 8 wilyxt Directory 8, 9 wilyconnectors Directory 8 Instrumentation

D
directive updates disable 31 Domains 10

190 Index

Java Agent Guide

dynamic update of internal directives 30 multi-level subclass 30 polling 30 instrumentation 28, 30 InstrumentPoints 40 Interitance.jar 8 Interval Counter Tracer 47 Introscope IntroscopeAgent.profile 7, 8, 16, 17 PowerPacks 6 introscope.autoprobe.directivesFile 28 IntroscopeAgent.profile. See Introscope. Introscope-enabled code enabling 13 options 75 switching back to original code 75 Investigator 18

Tag Libraries 34 JSR-77 11 JTA 34 JVM 4, 13, 16 JVM 1.5 28, 29, 30, 60 JVM 5.0 30 configuring for AutoProbe JVM AutoProbe 8, 13

25

L
lax files Introscope ProbeBuilder Wizard.lax LeakHunter 6 log file 30 Log4J Agent settings 97 Logging Options 10 76

J
j2ee.pbd 42 Java 1.5 9, 35 Java 5 JVM 13 Java Database Connectivity. See JDBC. Java Message Service 34 Java Naming and Directory Interface 34 Java Server Pages. See JSP. Java Transaction API 34 Java Virtual Machine. See JVM. Java2 Security Policy on WebSphere 5.0 63, 64, 68 JavaI5DynamicInstrumentation.jar 9 JMS 34 JMX 11 JMX support 142146 configuring 145 enabling 145 JMX filters 144 Metric name conversion default method 142 primary keys method 143 primary keys 143 primary keys, defining 146 JNDI 34 JRockit 13 configuring JVM for 24 JSP DB Tag Libraries 34 IO Tag Libraries 34

M
method 35, 43 methods method signature 48 metrics 35, 41 and URL groups 118

N
Network Sockets 34

O
Oracle 43 databases 43 Oracle 10g 13, 17, 24, 66 10.0.3 66 AutoProbe 66 AutoProbe Connector 24 Oracle JDBC 34 oraclejdbc.pbd 43

P
PBD 28 PBD Generator, configuring and using PBD. See ProbeBuilder Directives. PBL full version 28 typical version 28 platform monitor 150 troubleshooting 152 pre-production environments 6 185

Index 191

CA Wily Introscope Local Product

private methods 11 ProbeBuilder Command-line ProbeBuilder 72 customizing Directives 33 dynamic for JVM 1.5 28 list files 35 log 99 options 18 ProbeBuilder Directives 8, 18, 28, 33, 34, 35 Agent Initialization 40 applying changes to 57 components tracked 34 components tracked in 34 custom Tracers 43 combining 51 creating 43 examples template 47 exceptions 40 files custom 36, 71 updating Introscope with 56 system 36, 71 keywords 51 action 52 IdentifyClassAs 52 IdentifyCorbaAs 53 IdentifyInheritedAs 53 InstrumentPoint 53 TraceAllMethodsIfInherits 53 TraceAllMethodsOfClass 53 TraceOneMethodOfClass 53 TraceOneMethodWithParameterIfInherits 54 TraceOneMethodWithParametersOfClass 54 TurnOn 54 skip 52 SkipClass 52 SkipPackage 52 Threshold action 55 parameters 55 TraceAllMethodsWithThresholdOfClass 55 TraceOneMethodWithThresholdOfClass 55 Threshold Tracers OverThresholdPerIntervalCounter 56 StalledMethodTracer 56 UnderThresholdPerIntervalCounter

56 modifying 37 EJB subclass tracing 40 Tracer Groups adding classes to 40 Tracer groups turning on or off 39 only defined methods traced 30, 31, 34 parameters 51 Tracer Groups adding classes to 40 turning on or off 40 Tracer groups turning on or off 39 updating probes 56 ProbeBuilder List 28, 35 ProbeBuilder Wizard 7071 ProbeBuilder.jar 9 BasicDirectiveLoader.jar 9 SignedJARDirectiveLoader.jar 9 UnifiedDirectiveLoader.jar 9 ProbeBuilding 16 dynamic ProbeBuilding 29 production environments 6 properties files 165 properties files Introscope ProbeBuilder Wizard.lax 76

R
RedHat platform monitor 150 redirect output Agent 97 re-instrumenting 28 Remote Method Invocation. See RMI.

S
SAP J2EE 6.20 22 AutoProbe Connector 22 SAP J2EE 6.40 23 AutoProbe Connector 23 SAP NetWeaver 13, 17, 23 SAP NetWeaver 04 23 AutoProbe Connector 23 SAP NetWeaver 6.40 23, 58 ServletHeaderDecorator.jar 8 Servlets 34

192 Index

Java Agent Guide

single-metric Tracers 48 Skip directives 28 Skips 40, 43 Socket Metrics 11 SQL Agent 12, 43 SQL Metrics Average Query Roundtrip Time (ms) 139 connection count 138 SQLAgent.jar 9 supported JDBC Drivers 133 SQLAgent.jar 9 sqlagent.pbd 42 Stall Event Reporting 11 Struts 34 subclass hierarchy 32 Sun configuring AutoProbe for 19 Sun ONE 16, 23, 34 AutoProbe Connector 23 SuperDomain 10 Supportability-Agent.jar 9 System Logs 34

Rate Tracer 47 FrontendMarker 42 MethodCPUTimer 45 MethodRateTracer 46 MethodTimer 46 MethodTraceDecrementor 46 MethodTraceIncrementor 46 names 45 PerIntervalCounter 46 switching between full and typical syntax 44 toggle files 37

28

U
UDP 34 URL groups using the URLGrouper 121 URL Groups for Blame Reporting User Datagram Protocol 34

11

V
verbose mode running Agent in Virtual Agent 10 97

T
Threads 34 tracer 35 tracing methods 35 Tracer Group 35 Tracer Groups 28, 30, 40, 41 adding classes to 40 Tracers advanced custom creating 48 advanced single-metric 48 BackendMarker 42 Blame Technology and 48 components 44 ConcurrentInvocationCounter 45 Custom 43 sample file 43 default Tracer Groups 37 DumpStackTraceTracer 46 example template 47 examples 47 Average Tracer 47 Combined Counter Tracer 48 Counter Tracer 47 Interval Counter Tracer 47

W
WebAppSupport.jar 8 WebLogic 16, 20, 34 AutoProbe Connector 20 bootstrap classpath 20 WebSphere 16, 34 Introscope-enabling JDBC DataSource 134 WebSphere 4.0 20 AutoProbe Connector 20 WebSphere 5.0, 5.1, or 6.0 21 AutoProbe Connector 21 Websphere 6.0 62 WebSphere Application Server configuring custom service for 113 disabling automatic Agent naming for 92 PMI data 154156 viewing 157

Index 193