Spotlight on SQLServer 8.

0
Reporting and Trending Guide

 2011 Quest Software, Inc.

ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in
this guide is furnished under a software license or nondisclosure agreement. This software may be
used or copied only in accordance with the terms of the applicable agreement. No part of this
guide may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording for any purpose other than the purchasers personal use
without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license,
express or implied, by estoppel or otherwise, to any intellectual property right is granted by this
document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN
QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS
INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR
INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect
to the accuracy or completeness of the contents of this document and reserves the right to make
changes to specifications and product descriptions at any time without notice. Quest does not
make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
email: [email protected]
Refer to our Web site (www.quest.com) for regional and international office information.
Patents
Protected by U.S. Patent #6,901,582. Additional patents pending.
Trademarks
Quest, Quest Software, the Quest Software logo and Spotlight are trademarks and registered
trademarks of Quest Software, Inc in the United States of America and other countries. For a
complete list of Quest Softwares trademarks, please see
http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered
trademarks are property of their respective owners.
Spotlight on SQL Server 8.0
Reporting and Trending Guide
Monday, 16 May 2011

Table of Contents
About Spotlight Reporting and Trending

Views

Work with Views

Reports
Work with Reports
Sample Reports

7
9
9
10

Database Growth

10

Deadlock List

10

Server Alarms

10

SQL Server Health

11

Windows Server Health

11

Create Reports
Web Reports

11
13

Work with Web Reports

13

Specify the Location of Your Web Reports Server

13

Deploy Web Reports

13

Spotlight Statistics Repository

15

Schema of the Spotlight Statistics Repository

15

How the Spotlight Statistics Repository works

17

Store Data in the Spotlight Statistics Repository

20

Query the Spotlight Statistics Repository

21

Query Custom Counters

26

Query Alarm Data

27

Table Definitions

29

spotlight_datasources

29

Spotlight on SQL Server Reporting and Trending Guide

Table of Contents

spotlight_domains

29

spotlight_monitored_objects

29

spotlight_perfdata

30

spotlight_stat_classes

31

spotlight_stat_keys

31

spotlight_stat_names

31

spotlight_technologies

32

spotlight_timestamps

32

Stored Procedures
Example stored procedures

Specifications

33
36

36

spotlight_get_datasources

36

spotlight_get_domains

37

spotlight_get_monitored_objects

37

spotlight_get_table_columns

38

spotlight_get_table_data_instance

39

spotlight_get_table_keys

39

spotlight_get_table_range

40

spotlight_get_table_span

40

spotlight_get_tables

41

spotlight_get_technologies

41

spotlight_sqlanalysis_results

42

spotlight_ssr_version

42

Maintenance Plan for Spotlight Statistics Repository and Playback Database

43

Database Configuration

43

Fragmentation and Index Performance

43

Database Backup

44

Spotlight on SQL Server Reporting and Trending Guide

Table of Contents

Appendix: Contact Quest

45

Contact Quest Support

45

Spotlight Support

45

Contact Quest Software

46

About Quest Software

46

Index

47

1
About Spotlight Reporting and Trending
In Spotlight Reporting and Trending, you can access views and reports to assist with long-term
trend analysis and associated reporting. By configuring the Diagnostic Server to store statistics in
the Spotlight Statistics Repository, you can retrieve the data via views and reports
Views display data stored in the Spotlight Statistics Repository in the form of charts and tables.
The charts and tables are grouped into logical categories that help you analyze the collected
data. Views are different to drilldowns in that they show long-term data collected over time, not
real-time data.
Reports also display data stored in the Spotlight Statistics Repository and are displayed in the
Spotlight Report Viewer. These reports are designed to provide you with a starting point for
meeting your reporting requirements and can be modified to suit your needs.
To view the data collected by Spotlight in the Reporting and Trending portal, you must
configure the Spotlight Statistics Repository. See the Spotlight on SQL Server Deployment
Guide for more information.
Note: It is important to implement a good maintenance routine to ensure the Spotlight Statistics
Repository is optimized and free of inconsistencies. See "Maintenance Plan for Spotlight
Statistics Repository and Playback Database" (page 43) for more information.

2
Views
Work with Views
Use Views to analyze and investigate long-term SQL Server data for planning purposes, to
identify trends, or to determine potential performance issues. Views display data stored in the
Spotlight Statistics Repository in the form of charts and tables. The charts and tables are grouped
into logical categories, called Views, that help you analyze the collected data. Views are
different to drilldowns in that they show long-term data collected over time, not real-time data.
Note: You have some control over how the data is collected. See the online help for more
information.
To open a view
1. Select Reporting and Trending | Views from the Spotlight Browser.
2. Select from the view categories.
3. Select the view.
4. Click Select Connections. Select the connection(s) to view.

5. Click Refresh.

Click

To do this...
Select a different Spotlight Statistics Repository.

Select
Repository
Time

Select the time frame for the view.

Range
Filter
Data
Export

Filter the data in the view.

Export the view. Grids are exported as .csv files. Charts are exported as .bmp
files.

Spotlight on SQL Server Reporting and Trending Guide

Views

Click

To do this...
You cannot change the name of the exported files so if you have previously
exported a view and you don't want to overwrite the files, either rename the
exported files (via Windows Explorer) or export the new files to a different
location.

Configure
|Custom
Counters

Create the data for the Custom Counters view.

Configure
|
Diagnostic
Server

Configure the diagnostic server. See Configure the Spotlight Statistics

Repository for more information.

For descriptions of the views available, see the online help.

3
Reports
Work with Reports
In Spotlight, you can configure the Diagnostic Server to store the statistics it collects in a
repository for reporting purposes. You can then retrieve this data in the form of views and
reports. You can use the example reports provided or build on them to create your own reports
by querying the data yourself using stored procedures. See "Stored Procedures" (page 33) for
more information.
Spotlight reports are displayed in an external viewer. When you launch the Spotlight Report
Viewer it connects to the Spotlight Statistics Repository.
To view a Spotlight report
1. Select Reporting and Trending | Reports from the Spotlight Browser.
2. Select a report.
3. If SQL Server authentication is required, enter the user and password details.
4. In the Spotlight Report Viewer:
a. Select the start date.
b. Select the end date.
c. Select the Diagnostic Server.
d. Select the connection.
e. Click View Report.
Tips:
l

Change reports by selecting a different report from the Report list at the top of the
Spotlight Report Viewer. Complete the Diagnostic Server and Connection Name fields
and then click View Report.

If you change the report criteria (time frame, Diagnostic Server or Connection Name),
click View Report to see an updated version of the report.

Save reports in PDF, or as an .xls (Microsoft Excel) file using the Save icon on the Report
toolbar.

Print, search, and refresh reports using the Report toolbar.

You can create your own reports for use in Spotlight.

Spotlight on SQL Server Reporting and Trending Guide

10

Reports

Sample Reports
Spotlight Reporting and Trending includes a set of example reports that demonstrate how stored
procedures are used to query the Spotlight Statistics Repository.
Example reports are located on the Spotlight client in the following folder:
l

C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\Trending\Reports

Example reports compatible with SQLServer 2008 Reporting Services are located in the
following folder:
l

C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\Trending\Reports\Web

Reports 2008

The input parameters are the same for all example reports. The Input parameters are:
l

Time start

Time end

Diagnostic Server

Server name (monitored object)

Database Growth
The Database growth report contains the following charts:
Data Growth Trend Shows the following statistics for each database on the specified server:
l

Data size

Data allocated

Log Growth Trend Shows the following statistics for the transaction log for each database on
the specified server:
l

Log size

Log allocated

Deadlock List
The Deadlock List report shows all lock requests resulting in a deadlock on the specified server
for a set time period.

Server Alarms
The Server Alarms report shows all alarms that have occurred on the specified server for a set
time period.

Spotlight on SQL Server Reporting and Trending Guide

11

Reports

SQL Server Health

The Server Health report shows the following charts for the specified server for a set time period:
l

Number of Logged on Users

Batches, Compiles and Recompiles

Buffer Cache Hit Ratio

Procedure Cache Hit Ratio

Page Life Expectancy

Server Memory

Windows Server Health

The Windows Server Health report shows the following charts for the specified server for a set
time period:
l

Total CPU Utilization chart

Context Switches/Sec

Paging Rate

Network Bandwidth Utilization

Disk Queue Length

See "Deploy Reports" in this section for information on how to deploy the example reports.

Create Reports
The Spotlight Report Viewer displays reports using SQL Server 2005 or 2008 Report Definition
Language files (.rdl). You can build on the example reports or create your own reports by
querying the data from the Spotlight Statistics Repository yourself.
Example reports are located on the Spotlight client in the following folder:
l

C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\Reports

Example reports compatible with SQLServer 2008 Reporting Services are located in the
following folder:
l

C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\Reports\Web

Reports 2008

To use your own report definition files

Set the location of your .rdl files in Spotlight.

a. Click Reporting and Trending in the Spotlight browser.

Spotlight on SQL Server Reporting and Trending Guide

12

Reports

b. Right-click Reports.
c. Select Settings.
d. Navigate to the directory containing the rdl files you want to use.
e. Click OK.
or

Copy your .rdl files into C:\Program Files\Quest Software\Spotlight on SQL

Server\Plug-ins\Trending\Reports.

Note: If you do not have write access to the Program Files folders listed above, you can create
the following folder and use it as an alternative:
l

Windows XP - My Documents\Spotlight Reports

Vista - <user>\Documents\Spotlight Reports

4
Web Reports
Work with Web Reports
View web-based reports within Spotlight by configuring the Web Reports portal to point to your
web-based reports server.
Review the related topics for information on how to use the SQL Server Reporting Services web
interface to deploy the sample reports included with Spotlight Reporting and Trending:
To open web reports in Spotlight

Select Reporting and Trending | Web Reports from the Spotlight Browser.

Specify the Location of Your Web Reports Server

You can view web-based reports within Spotlight by configuring the Web Reports portal to
point to your web-based reports server.
To specify the location of your Web reports server
1. Select Configure | Diagnostic Server.

2. Click Specify the location of your Web Reports server.

3. Enter the URL of your web reports server.

Deploy Web Reports

This topic outlines how to use the SQL Server 2005 and 2008 Reporting Services web interface
to deploy the example reports included with Spotlight Reporting and Trending. See "Sample
Reports" (page 10) for more information.
To complete the following steps, you need report server administrator privileges.
1. In the Reporting Services web interface, create a shared data source by clicking New
Data Source.
2. Complete the fields on the New Data Source page. You will need the following
information:

Spotlight on SQL Server Reporting and Trending Guide

14

Web Reports

Data Source name

The example reports included with Spotlight Reporting and Trending use a shared
data source called Spotlight Statistics Repository.

Connection type
Select Microsoft SQL Server.

Connection string
Enter the connection string for the SQL Server instance that you installed the
Spotlight Statistics Repository on.
For example:
Data Source=YourServerHostname;Initial Catalog=Spotlight
Statistics Repository

Connection credentials
Including user name and password, if required.

3. Upload the report definition (.rdl) files.

To do this:
a. In the Reporting Services web interface, click Upload File.
b. Click Browse to navigate to the .rdl file. Spotlight report definition files are
located on the Spotlight client in the following folder:
C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\Reports
Report definition files compatible with SQLServer 2008 Reporting Services are
located in:
C:\Program Files\Quest Software\Spotlight on SQLServer\Plugins\SoSSE\Reports\Web Reports 2008
c. Click OK.
You can view web-based reports within Spotlight by configuring the Web Reports portal to
point to your web-based reports server.

5
Spotlight Statistics Repository
Schema of the Spotlight Statistics Repository
The Spotlight Statistics Repository is based on a star schema because of the simplicity it offers
for data storage and retrieval. Star schemas are a simple schema consisting of a centralized fact
table connected to multiple dimension tables.

Fact tables hold factual data and the dimension tables hold descriptive data. One of the benefits
of a star schema is the speed of data retrieval. With star schemas, you can use relatively simple
SQL queries to return particular information about, in this case, a SQL Server instance. Later in
this chapter well show you how you can query the Spotlight Statistics Repository using
Spotlight API stored procedures and T-SQL.
In the Spotlight Statistics Repository, the fact table (spotlight_perfdata) stores facts related to
SQL Server performance, and the dimension tables hold definition data such as SQL Server
instance information and attributes.

Spotlight on SQL Server Reporting and Trending Guide

16

Spotlight Statistics Repository

The schema of the Spotlight Statistics Repository is as follows:

The main dimension tables in the Spotlight Statistics Repository are as follows:
l

Spotlight_datasources
Contains data used to categorize the collections that are performed by Spotlight. There is
one row in this table for each category of data collected. Examples of categories are SQL
Server, Windows, and Diagnostic Server.

Spotlight_domains
Contains data about the Diagnostic Servers feeding the Spotlight Statistics Repository. In
the case of multiple Diagnostic Servers feeding the Spotlight Statistics Repository, there
is one row per Diagnostic Server. The Domain_description column contains the
Diagnostic Server host name.

Spotlight_monitored_objects
Contains data about each of the SQL Server or Windows servers that a Diagnostic Server
is monitoring. There is one row in this table for each server being monitored by each
Diagnostic Server.

Spotlight on SQL Server Reporting and Trending Guide

17

Spotlight Statistics Repository

Spotlight_techonologies
Contains data that categorizes the collections that are performed. There is one row in this
table for each category of data collected.

The remaining dimension tables contain data about the collections being stored in the Spotlight
Statistics Repository. See "Table Definitions" (page 29) for more information on each of the
tables in the Spotlight Statistics Repository.

How the Spotlight Statistics Repository works

The Diagnostic Server collects data about the SQL Server instance and then writes it to the
Spotlight Statistics Repository. How often this data is written to the Spotlight Statistics
Repository depends on the scheduling options set for each data collection in Spotlight.
The data is inserted into the relevant dimension tables and then the fact is written into the fact
table (spotlight_perfdata).
Lets use the example of a Diagnostic Server writing data about the sqlblockinglist
collection to the Spotlight Statistics Repository. In the following chart, the Diagnostic
Server, DS123, is collecting data about SQL Server instance, SQLServer789, running on
Windows host, Windows01.

Spotlight on SQL Server Reporting and Trending Guide

Spotlight Statistics Repository

The following chart shows how the data flows into the Spotlight Statistics Repository.

18

Spotlight on SQL Server Reporting and Trending Guide

Spotlight Statistics Repository

19

Spotlight on SQL Server Reporting and Trending Guide

20

Spotlight Statistics Repository

Store Data in the Spotlight Statistics Repository

There are a number of options that must be configured for Spotlight to store data in the Spotlight
Statistics Repository.
To configure the Spotlight Statistics Repository
1. In Spotlight, select Configure | Diagnostic Server.

Spotlight on SQL Server Reporting and Trending Guide

21

Spotlight Statistics Repository

2. Click Configure the Spotlight Statistics Repository.

3. Select Enable use of the Spotlight Statistics Repository.
4. Complete the remaining options.
See Configure the Spotlight Statistics Repository in the Spotlight on SQL Server
Deployment Guide for more information.
To store data in the Spotlight Statistics Repository
1. Click Configure |Scheduling.

2. From here, you can configure how often data is stored in the Spotlight Statistics
Repository for each collection. See Configure Scheduling in the online help for more
information.

Query the Spotlight Statistics Repository

Before querying the Spotlight Statistics Repository we need to know how the repository is
structured and what data it contains. You can use the Spotlight API stored procedures to browse
the dimension tables in the repository. You can then use additional stored procedures or custom
T-SQL to retrieve data.

Scenario
Your manager has asked you to provide statistics on database performance for the last
quarter. You decide to query the Spotlight Statistics Repository to return file I/O statistics
for the databases your manager is interested in.
To retrieve this data, you first browse the dimension tables to become familiar with the data
in the Spotlight Statistics Repository. Once familiar, you write queries using the Spotlight
stored procedures to get data from the fact table. You then refine the queries using custom TSQL to retrieve more specific data from the Spotlight Statistics Repository.
First up, were going to use some of the Spotlight API stored procedures to browse the dimension
tables and get the range of values available for use in our queries.
The Spotlight stored procedures used below are explained in detail in this guide.
l

See "Stored Procedures" (page 33) for more information.

See "Specifications" (page 36) for more information.

In SQL Server Management Studio, create a new query against the Spotlight Statistics
Repository. We want to know the names of the available data sources and the names and
domains of the monitored objects.

Spotlight on SQL Server Reporting and Trending Guide

22

Spotlight Statistics Repository

Lets start exploring the Spotlight Statistics Repository by looking at datasources. Datasources
are broad categories of data.
To get a list of data sources in the repository, run the following query:
exec spotlight_get_datasources

This returns the following results:

We can see here that we have data from Windows, SQL Server, the Diagnostic Server and alarms
(availability).
Now we want to know the name of the Diagnostic Server storing data in the Spotlight Statistics
Repository. We can determine this by running:
exec spotlight_get_domains

To get the following result:

We can see here that there is one Diagnostic Server writing data to this repository. There can be
multiple Diagnostic Servers, each one resulting in a row.
Finally, we want to know information about the servers being monitored.
We can do this by running the following query:
exec spotlight_get_monitored_objects

To get the following results:

A common question is what is the range of timestamps available for a monitored server? The
following query returns the time ranges for all monitored servers:
select

Spotlight on SQL Server Reporting and Trending Guide

23

Spotlight Statistics Repository

so.monitored_object_name,
min(st.timecollected) as 'start',
max(st.timecollected) as 'end'
from
spotlight_timestamps st
join spotlight_monitored_objects so on st.monitored_
object_id = so.monitored_object_id
group by
so.monitored_object_name

Now that we have some information about the Diagnostic Server, monitored servers and
categories of collections in the Spotlight Statistics Repository, we want to know what statistics
are available to us.
The spotlight_get_tables stored procedure returns a list of tables in the repository for a given
data source. Since we are looking for SQL Server data, from our first query above, we know
that the datasource is sqlserver_spotlight, so we will use that as the parameter in the
following query:
exec spotlight_get_tables 'sqlserver_spotlight'

The following results are returned.

Were going to look at virtual file stats as this will give us the file IO statistics we need
for our report.
To find out what columns are available to us in the virtual file stats table use the following
query. With this query we need to specify the datasource and table name.

Spotlight on SQL Server Reporting and Trending Guide

24

Spotlight Statistics Repository

exec spotlight_get_table_columns 'sqlserver_

spotlight','virtualfilestats'

We need to produce the report over a time range, but for curiositys sake, we would like to find
out the time range of all stored data for a particular domain name, monitored object, and table
combination.
To do this, use the spotlight_get_table_span stored procedure and specify the domain name,
monitored object, and table name.
exec spotlight_get_table_span 'DS123:3843','Windows01_
SQLServer789_sqlserver','virtualfilestats'

Now we can put it all together and query the fact table. Well use the spotlight_get_table_range
stored procedure and specify the following:
l

start date

end date

domain name

monitored object

table name

Spotlight_get_table_range returns data for the requested time range for a table, for a particular
monitored object and domain.
This gives us the following query:
exec spotlight_get_table_range '2009-03-26 11:15:16.153','200907-26 17:15:17.113', 'DS123:3843','Windows01_SQLServer789_
sqlserver', 'virtualfilestats'

Which returns all columns for the table virtualfilestats:

Although there is a lot of useful data in the query results, its not exactly what we are looking
for. We need to refine the data returned from the Spotlight Statistics Repository. Following on

Spotlight on SQL Server Reporting and Trending Guide

25

Spotlight Statistics Repository

from the example above, if we want to retrieve only certain columns from the virtualfilestats
table, we can use custom T-SQL to return specific data.
To return only the iorate, disk, readsrate, and writesrate columns from the virtualfilestats
table, we can use the following custom T-SQL:
select
sp.timecollected,
max(case when sn.statistic_name = 'dbname' then
sp.raw_value end) as 'dbname',
max(case when sn.statistic_name = 'dbfilename' then
sp.raw_value end) as 'dbfilename',
max(case when sn.statistic_name = 'disk' then sp.raw_
value end) as 'disk',
max(case when sn.statistic_name = 'iorate' then
sp.raw_value end) as 'iorate',
max(case when sn.statistic_name = 'readsrate' then
sp.raw_value end) as 'readsrate',
max(case when sn.statistic_name = 'writesrate' then
sp.raw_value end) as 'writesrate'
from
spotlight_perfdata sp
join spotlight_stat_classes sc on sp.statistic_class_
id = sc.statistic_class_id
join spotlight_stat_names sn on sp.statistic_name_id =
sn.statistic_name_id
join spotlight_monitored_objects so on sp.monitored_
object_id = so.monitored_object_id
where
sc.statistic_class_name = 'virtualfilestats'
and so.monitored_object_name = 'Windows01_
SQLServer789_sqlserver'
and sp.timecollected between '2009-09-01' and
'2009-09-30'
group by
sp.timecollected, sp.statistic_key_id
order by

Spotlight on SQL Server Reporting and Trending Guide

26

Spotlight Statistics Repository

sp.timecollected

This gives us the following results:

Query Custom Counters

Scenario
You have created a custom counter in Spotlight on SQL Server and now want to create a
report that contains data from that counter.
Querying custom counters is exactly the same as querying other data in the Spotlight Statistics
Repository. SQL Server custom counters are stored in the statistic class sqlcustomcounters and
Windows custom counters are stored in the class windowscustomcounters. You can use the API
stored procedures to retrieve date ranges and column names with these like any other data
collection.
The following example of T-SQL retrieves SQL Server custom counter values:
select
sp.timecollected,
max(case when sn.statistic_name = 'countername' then sp.raw_value
end) as 'countername',
max(case when sn.statistic_name = 'countervalue' then sp.raw_
value end) as 'countervalue'
from
spotlight_perfdata sp
join spotlight_stat_classes sc on sp.statistic_class_id =
sc.statistic_class_id

Spotlight on SQL Server Reporting and Trending Guide

27

Spotlight Statistics Repository

join spotlight_stat_names sn on sp.statistic_name_id =

sn.statistic_name_id
join spotlight_monitored_objects so on sp.monitored_object_id =
so.monitored_object_id
where
sc.statistic_class_name = 'sqlcustomcounters'
and so.monitored_object_name = 'Windows01_SQLServer789_sqlserver'
and sp.timecollected between '2009-09-01' and '2009-09-30'
group by
sp.timecollected, sp.statistic_key_id
order by
sp.timecollected

The following shows the results returned for the custom counter Pages Allocated.

Query Alarm Data

Alarm data can be queried from the Spotlight Statistics Repository. Querying alarm data is the
same as querying other data stored in the repository. Alarm data is stored in a statistic class
called alarms.
The following is an example of a T-SQL statement that can be used to retrieve alarm data. You
need to specify the monitored object name and start and end times as parameters.
select
sp.timecollected,
max(case when sn.statistic_name = 'severity' then sp.raw_value
end) as 'severity',
max(case when sn.statistic_name = 'text' then sp.raw_value end)
as 'text',
max(case when sn.statistic_name = 'rule' then sp.raw_value end)
as 'rule',

Spotlight on SQL Server Reporting and Trending Guide

Spotlight Statistics Repository

max(case when sn.statistic_name = 'action' then sp.raw_value end)

as 'action',
max(case when sn.statistic_name = 'key' then sp.raw_value
end) as
'key',
max(case when sn.statistic_name = 'raw text' then sp.raw_value
end) as 'raw text'
from
spotlight_perfdata sp
join spotlight_stat_classes sc on sp.statistic_class_id =
sc.statistic_class_id
join spotlight_stat_names sn on sp.statistic_name_id =
sn.statistic_name_id
join spotlight_monitored_objects so on sp.monitored_object_id =
so.monitored_object_id
where
sc.statistic_class_name = 'alarms'
and so.monitored_object_name = 'Windows01_SQLServer789_sqlserver'
and sp.timecollected between '2009-09-08' and '2009-09-17'
group by
sp.timecollected, sp.statistic_key_id
order by
sp.timecollected

The above T-SQL statement returns the following results:

28

Spotlight on SQL Server Reporting and Trending Guide

29

Spotlight Statistics Repository

Table Definitions
spotlight_datasources
spotlight_datasources contains data used to categorize the collections that are performed. There is
one row in this table for each category of data collected.
Column

Description

datasource_id

Generated ID.

datasource_name

Name of the data source.

spotlight_domains
spotlight_domains contains data about Diagnostic Servers. If you have multiple Diagnostic
Servers feeding the Spotlight Statistics Repository, there is one row per Diagnostic Server
in this table.
Column

Description

domain_id

Generated ID.

domain_
description

The name of the domain (analogous to Diagnostic Server host name means
that this Spotlight Statistics Repository can hold data from multiple
diagnostic servers).

spotlight_monitored_objects
spotlight_monitored_objects contains data about the SQL Servers and Windows hosts that a
Diagnostic Server is monitoring. There is one row in this table for each SQL Server or Windows
host being monitored by each Diagnostic Server.
Column

Description

monitored_
object_id

Generated ID.

monitored_
object_
name

Name of monitored object (Windows host name or SQL Server Instance

Name etc).

monitored_
object_
display_
name

A user-friendly interpretation of the encoded name that the Diagnostic

Server uses to uniquely identify a monitored object. This name is suitable
for display on reports or other user-visible outputs.

Spotlight on SQL Server Reporting and Trending Guide

30

Spotlight Statistics Repository

Column

Description

parent_
monitored_
object_id

The monitored_object_id value of another monitored object upon which

this one depends, or null if no such dependency exists. For example, a SQL
Server databases parent will be the Windows host upon which it runs.

technology_
id

The technology of this monitored object.

domain_id

The domain (that is, Diagnostic Server) this monitored object belongs to.

spotlight_perfdata
spotlight_perdata is the fact table into which all of the data is stored. Each fact is effectively
matrixed by the hierarchy of attributes that denote which Diagnostic Server, monitored object,
collection and collection column the datum originally came from.
Column

Description

perfdata_id

Generated ID.

monitored_
object_id

The monitored object that this fact came from.

statistic_
class_id

The collection that this fact came from.

statistic_
name_id

The column in the collection that this fact came from.

statistic_
key_id

The key column of the collection that this fact came from.

timecollected

The time this fact was collected from the monitored object.

raw_value

The raw value returned by the collection for that column.

num_
observations

Internal use only.

sum_of_
values

Internal use only.

max_value

Internal use only.

min_value

Internal use only.

sum_of_
squares

Internal use only.

Spotlight on SQL Server Reporting and Trending Guide

31

Spotlight Statistics Repository

spotlight_stat_classes
spotlight_stat_classes contains the names of collections being stored in the Spotlight Statistics
Repository. There is one row for each table/data source combination.
Column

Description

statistic_class_id

Generated ID.

datasource_id

The data source that this collection belongs to.

statistic_class_name

The name of the collection.

spotlight_stat_keys
spotlight_stat_keys contains key values for all of the collections stored in the Spotlight Statistics
Repository. Each key value/table/data source combination has a row in this table. The purpose of
this table is to allow queries to get data from the fact table based on a key value without having
to scan values in the fact table itself.
Column

Description

statistic_
key_id

Generated ID.

statistic_
class_id

The statistic_class (Collection) this column belongs to.

datasource_
id

The data source of the collection that this column belongs to.

statistic_
name_id

Generated ID.

statistic_
key_name

The name of the key column.

spotlight_stat_names
spotlight_stat_names contains data about the columns belonging to collections. There is one row
in this column for each column/table/data source combination.
Column

Description

statistic_
name_id

Generated ID.

Spotlight on SQL Server Reporting and Trending Guide

32

Spotlight Statistics Repository

Column

Description

statistic_
class_id

The statistic_class (collection) this column belongs to.

datasource_
id

The data source of the collection that this column belongs to.

statistic_
name

The name of the column.

datatype_
guide

This column provides a broad guide to the type of data that can be expected
in facts for this named statistic. Values can be string, numeric or date.

derived

Whether this is a derived column or not.

spotlight_technologies
spotlight_technologies contains data used to categorize the collections that are performed. There
is one row in this table for each category of data collected.
Column

Description

technology_id

Generated ID.

technology_name

The name of the technology.

spotlight_timestamps
spotlight_timestamps contains a summary of times for which data was collected, qualified by
monitored entity. It allows Spotlight to quickly determine the times for which historical data
is available for individual monitored entities, without the overhead of scanning the
potentially large fact table spotlight_perfdata. There is one row in this table for each
category of data collected.
Column

Description

timestamp_id

Generated ID.

timecollected

The time this fact was collected from the monitored object.

domain_id

The domain that this fact belongs to.

monitored_
object_id

Generated ID.

statistic_
class_id

The collection that this fact came from.

Spotlight on SQL Server Reporting and Trending Guide

33

Spotlight Statistics Repository

Stored Procedures
The following stored procedures return either metadata about the Spotlight Statistics Repository
or data from within the Spotlight Statistics Repository.
Stored procedure

Description

spotlight_get_datasources

Returns a list of all data sources in the

repository (effectively the contents of
spotlight_datasources).

spotlight_get_domains

Returns a list of all the diagnostic servers

putting data into this repository.

spotlight_get_monitored_objects
@domain_name, @technology_name

Returns a list of names of all the monitored

objects that have data in this repository for a
given domain_name. If domain_name is NULL
all monitored objects are returned. Also returns
the monitored object display name, domain
description, and technology name.

spotlight_get_table_columns
@datasource_name, @table_name

Returns a list of columns for a table belonging

to a data source (effectively the contents of
spotlight_stat_names).

spotlight_get_table_data_instance
@date_time, @domain_name,
@monitored_object_name, @table_name

Returns the rows and columns from a table with

a timestamp that most closely matches @date_
time.

spotlight_get_table_keys @datasource_
name, @table_name

Returns a list of keys for a table belonging to a

data source (effectively the contents of
spotlight_stat_keys).

spotlight_get_table_range @start_date,
@end_date, @domain_name,
@monitored_object_name, @table_name

Returns the rows and columns from a table

between @start_date and @end_date.

spotlight_get_table_span @domain_
name,@monitored_object_name,
@table_name

Returns the time span of all stored data for a

particular domain_name, monitored object, and
table combination.

spotlight_get_tables @datasource_name

Returns a list of tables in the repository for a

given data source (effectively the contents of
spotlight_stat_classes).

spotlight_get_technologies

Returns a list of technologies in the repository

(effectively the contents of spotlight_
technologies).

Spotlight on SQL Server Reporting and Trending Guide

34

Spotlight Statistics Repository

Stored procedure

Description

spotlight_rt_get_aggregated_workload_
analysis @start_date, @end_date,
@domain_name, @monitoredobject

Returns data for the requested time range for

the Workload view.

spotlight_rt_get_batch_data @start_date,
@end_date, @domain_name,
@monitoredobject_list, @table_name,
@column_list

Returns data for the requested time range for a

table and associated columns for multiple
monitored objects in a domain.

spotlight_rt_get_fileio_stats @start_date,
@end_date, @domain_name,
@monitoredobject

Returns data for the Database File IO Statistics

view.

spotlight_rt_get_point_data @start_date,
@end_date, @domain_name_list,
@monitoredobject_list, @table_name,
@column_name

Returns the same data as spotlight_rt_get_

batch_data but for a particular point in time.

spotlight_rt_get_point_data_top @start_
date, @end_date, @domain_name_list,
@monitoredobject_list, @table_name,
@column_list,@sortdesc_column_list,
@maxrows

Returns the top n rows of spotlight_rt_get_

point_data based on the column sort order
passed in.

spotlight_rt_get_waitstats @start_date,
@end_date, @domain_name,
@monitoredobject

Returns data for the view Wait Statistics view.

spotlight_sqlanalysis_results @domain_
name, @monitored_object_name,
@date_time_from, @date_time_to,
@maxrows

Returns the worst performing SQL recorded in

the Spotlight Statistics Repository over the
supplied time span for a particular SQL Server
instance.

spotlight_ssr_version

Returns the version number of the Spotlight

Statistics Repository.

The following stored procedures are used for the example reports:
Stored procedure

Description

spotlight_rep_alarms

Returns a list of alarms for a chosen monitored object

from the repository.
Used in the Server Alarms report. This is a further
example of how to query the Spotlight Statistics
Repository to return tabular information.

Spotlight on SQL Server Reporting and Trending Guide

35

Spotlight Statistics Repository

Stored procedure

Description

spotlight_rep_cache_hit_ratios

Returns hit rates for the buffer and procedure caches for
all monitored objects in the repository.
Used in the SQL Server Health report. This is an example
of querying two columns from a table over time from the
Spotlight Statistics Repository.

spotlight_rep_cpu_utilization

Returns the amount of CPU being used by the specified

server against the total amount being used by all
Windows processes.
Used in the Windows Server Health report. This is an
example of querying one column from a table over time
from the Spotlight Statistics Repository.

spotlight_rep_disk_growth

Returns information on disk space usage over time.

spotlight_rep_disk_statistics

Returns disk statistics (percentage busy and disk queue

length) for the specified server.
Used in the Windows Server Health report. This is an
example of querying multiple columns with multiple
rows from a table over time from the Spotlight Statistics
Repository.

spotlight_rep_growth_data_
files

Returns the following statistics for the specified server:

l

Data size

Data allocated

Index size

Table size

Log size

Log allocated

Used in the Database Growth report. This is an example

of querying multiple columns with multiple rows from a
table over time from the repository.
spotlight_rep_sql_batches_rate

Returns the rate at which batches of SQL statements are

being submitted to SQL Server for execution.

Spotlight on SQL Server Reporting and Trending Guide

Spotlight Statistics Repository

Example stored procedures

The following stored procedures can be used as examples of how to retrieve data from the
Spotlight Statistics Repository.
l

spotlight_rt_get_allsqlagentjobs

spotlight_rt_get_custom_counter

spotlight_rt_get_failedsqlagentjobs

spotlight_rt_get_mostcommonblock

spotlight_rt_get_sqlblockedmost

spotlight_rt_get_sqlblockingmost

Specifications
The following section contains specifications for each of the stored procedures.

spotlight_get_datasources
Returns a list of all data sources in the repository (effectively the contents of spotlight_
datasources).
Syntax
spotlight_get_datasources
Input parameters
There are no input parameters for this stored procedure.
Returned values
ColumnName

ColumnType

Description

datasource_id

int

ID of data source

datasource_name

nvarchar(255)

Name of data source

36

Spotlight on SQL Server Reporting and Trending Guide

37

Spotlight Statistics Repository

spotlight_get_domains
Returns a list of all the diagnostic servers putting data into this repository.
Syntax
spotlight_get_domains
Input parameters
There are no input parameters for this stored procedure.
Returned values
ColumnName

ColumnType

Description

domain_id

int

ID of domain

domain_description

nvarchar(255)

Descriptive name of domain

spotlight_get_monitored_objects
Returns a list of names of all the monitored objects that have data in this repository for a given
domain_name. If domain_name is NULL all monitored objects are returned.
Syntax
spotlight_get_monitored_objects @domain_name, @technology_name
Input parameters
@domain_name
@technology_name

Spotlight on SQL Server Reporting and Trending Guide

38

Spotlight Statistics Repository

Returned values
ColumnName

ColumnType

Description

monitored_object_name

nvarchar(255)

Name of monitored object

monitored_object_
display_name

nvarchar(255)

A user-friendly interpretation of the encoded

name that the Diagnostic Server uses to
uniquely identify a monitored object. This
name is suitable for display on reports or
other user-visible outputs.

monitored_object_
display_technology_
name

nvarchar(255)

Combines the columns monitored_object_

display_name and technology_name to
display a user-friendly name that includes
the monitored object name and the
technology.

domain_description

nvarchar(255)

Descriptive name of domain

technology_name

nvarchar(255)

Descriptive name of technology

spotlight_get_table_columns
Returns a list of columns for a table belonging to a data source (effectively the contents of
spotlight_stat_names).
Syntax
spotlight_get_table_columns @datasource_name, @table_name
Input parameters
@datasource_name
@table_name
Returned values
ColumnName

ColumnType

Description

statistic_name

nvarchar(255)

Name of table column

Spotlight on SQL Server Reporting and Trending Guide

39

Spotlight Statistics Repository

spotlight_get_table_data_instance
Returns the rows and columns from a table with a timestamp that most closely matches
@date_time.
Syntax
spotlight_get_table_data_instance @date_time, @domain_name, @monitored_object_name,
@table_name
Input parameters
@date_time
@domain_name
@monitored_object_name
@table_name
Returned values
The returned values are dependent on the columns implicit in the table represented by the
@table_name parameter.

spotlight_get_table_keys
Returns a list of keys for a table belonging to a data source (effectively the contents of spotlight_
stat_keys).
Syntax
spotlight_get_table_keys @datasource_name, @table_name
Input parameters
@datasource_name
@table_name
Returned values
ColumnName

ColumnType

Description

statistic_key_value

nvarchar(255)

Key value for procedure

Spotlight on SQL Server Reporting and Trending Guide

40

Spotlight Statistics Repository

spotlight_get_table_range
Returns the rows and columns from a table between @start_date and @end_date.
Syntax
spotlight_get_table_range @start_date, @end_date, @domain_name, @monitored_object_name,
@table_name
Input parameters
@start_date
@end_date
@domain_name
@monitored_object_name
@table_name
Returned values
The returned values are dependent on the columns implicit in the table represented by the
@table_name parameter.

spotlight_get_table_span
Returns the time span of all stored data for a particular domain_name, monitored object, and
table combination.
Syntax
spotlight_get_table_span @domain_name, @monitored_object_name, @table_name
Input parameters
@domain_name
@monitored_object_name
@table_name
Returned values
The returned values are dependent on the columns implicit in the table represented by the
@table_name parameter.

Spotlight on SQL Server Reporting and Trending Guide

Spotlight Statistics Repository

spotlight_get_tables
Returns a list of tables in the repository for a given data source (effectively the contents of
spotlight_stat_classes).
Syntax
spotlight_get_tables @datasource_name
Input parameters
@datasource_name
Returned values
ColumnName

ColumnType

Description

statistic_class_name

nvarchar(255)

Name of table

spotlight_get_technologies
Returns a list of technologies in the repository (effectively the contents of spotlight_
technologies).
Syntax
spotlight_get_technologies
Input parameters
There are no input parameters for this stored procedure.
Returned values
ColumnName

ColumnType

Description

technology_id

int

ID of technology

technology_name

nvarchar(255)

Descriptive name of technology

41

Spotlight on SQL Server Reporting and Trending Guide

Spotlight Statistics Repository

spotlight_sqlanalysis_results
Returns the worst performing SQL recorded in the Spotlight Statistics Repository over the
supplied time span for a particular SQL Server instance.
Syntax
spotlight_sqlanalysis_results @domain_name, @monitored_object_name, @date_time_from,
@date_time_to, @maxrows
Input parameters
@domain_name
@monitored_object_name
@date_time_from
@date_time_to
@maxrows
Returned values
Returns information about the worst performing SQL recorded in the Spotlight Statistics
Repository.

spotlight_ssr_version
Returns the version number of the Spotlight Statistics Repository.
Syntax
spotlight_ssr_version
Input parameters
There are no input parameters for this stored procedure.
Returned values
Returns the version number.

42

Spotlight on SQL Server Reporting and Trending Guide

43

Spotlight Statistics Repository

Maintenance Plan for Spotlight Statistics

Repository and Playback Database
It is recommended that a maintenance plan for both the Spotlight Statistics Repository and the
playback database is implemented.
Both the Spotlight Statistics Repository and playback database support a large number of
monitored servers and automatically maintain the age of data available. Due to the nature of the
automatic purging of old data, the data may become fragmented and this may decrease the
efficiency of data retrieval operations over time.
This topic provides guidance on how to configure and automate the maintenance of the
Spotlight Statistics Repository and the playback database.

Database Configuration
The Spotlight Statistics Repository and the playback database do not require the database to be
run under the Full Recovery model. However, since Spotlight uses the model system database to
create the repository, your Spotlight repository may inherit the Full Recovery setting. Unless you
are required to use the Full Recovery model and are prepared to perform regularly scheduled log
file backups, we recommend you change the repository recovery model to Simple (in SQL Server
2000, this option is called Truncate Log on Checkpoint). Doing so will help maintain a
considerably smaller transaction log file.
To change the repository database to Simple recovery, run the following command:
SQL Server 2000
EXEC [dbo].[sp_dboption] 'SpotlightStatisticsRepository','trunc.
log on chkpt.','true'

SQL Server 2005 and 2008

ALTER DATABASE [SpotlightStatisticsRepository] SET RECOVERY
SIMPLE;

Fragmentation and Index Performance

The Spotlight Statistics Repository implements a star schema because of the data warehousingstyle simplicity it offers for data storage and retrieval. The star schema implements a primary fact
table that references a number of dimension tables. The fact table holds collections and alarm
data, and the dimension tables hold definition information on everything from instance
information to collection names. Spotlight maintains the age of the data in the repository, but the
continuous INSERT and DELETE operations performed for data insertion and purging can lead
to fragmentation. Fragmentation will eventually lead to increased CPU and I/O resource
consumption.

Spotlight on SQL Server Reporting and Trending Guide

44

Spotlight Statistics Repository

Although the playback database uses a simpler schema, its data will also become fragmented
over time as new data is inserted and old data purged.
To best address performance concerns, fragmentation should be minimized and index statistics
should be kept up-to-date. Spotlight's scheduled maintenance feature does this automatically. By
default, the Diagnostic Server runs maintenance procedures daily at 3am for the playback
database and Spotlight Statistics Repository. The maintenance schedule can be changed in
Spotlight Options. See the online help for more information.

Database Backup
The Spotlight Statistics Repository and playback database should be included in the list of
important databases which have a disaster recovery plan associated with them. The
implementation of this task is dependent on the policies and infrastructure of your organization.

Appendix: Contact Quest

Contact Quest Support
Quest Support is available to customers who have a trial version of a Quest product or who have
purchased a Quest product and have a valid maintenance contract. Quest Support provides
unlimited 24x7 access to SupportLink, our self-service portal. Visit SupportLink at
http://support.quest.com.
From SupportLink, you can do the following:
l

Retrieve thousands of solutions from our online Knowledgebase

Download the latest releases and service packs

Create, update and review Support cases

View the Global Support Guide for a detailed explanation of support programs, online services,
contact information, policies and procedures. The guide is available at: http://support.quest.com.

Spotlight Support
When contacting Quest Support about a Spotlight product, please have the version and build
number handy. You can obtain both by choosing the About option from the Help menu in the
relevant Spotlight application. If your question is about an error message, write the message
down and have it available for the technical support representative.
Whenever you have a question about Spotlight, click Support Bundle or Contact Support on
the Help menu. This creates a file called SpotlightSupport.zip. This file contains a snapshot of
your Spotlight installation. Send this file and your request for assistance to Quest Software
([email protected]).
Note: You can find SpotlightSupport.zip in different locations under different operating systems.
l

Windows XP: C:\Documents and Settings\<user>\Application Data\Quest

Software\Spotlight\Default\Support

Vista: C:\Users\<user>\AppData\Roaming\Quest Software\Spotlight\Default\Support

If you find that information in the Help system or printed user guides is incorrect, unclear, or
incomplete, please report the problem to Quest Support.

Spotlight on SQL Server Reporting and Trending Guide

46

Appendix: Contact Quest

Contact Quest Software

Email

[email protected]
Quest Software, Inc.
World Headquarters

Mail

5 Polaris Way
Aliso Viejo, CA 92656
USA

Web site

www.quest.com

See our web site for regional and international office information.

About Quest Software

Now more than ever, organizations need to work smart and improve efficiency. Quest Software
creates and supports smart systems management productshelping our customers solve everyday
IT challenges faster and easier. Visit www.quest.com more information.

reports

Index
A
about Quest Software

46

alarm data

27
C

about

create

11

customize

11

Database Growth

10

definition files

11

deploy

13
10

Contact Quest Software

46

examples

custom counters

26

open

Customer support

45

Server Alarms

10

SQLServer Health

10

web

13

Windows Server Health

10

D
definition files

11
E
S

example reports

10
schema
M

maintenance plan

Spotlight Reporting
43

Q
query
alarm data

27

custom counters

26

Spotlight Statistics Repository

21

Quest Software

46

Quest support

45
R

reporting

15
6

Spotlight Statistics Repository

alarm data

27

custom counters

26

how it works

17

maintenance plan

43

query

21

schema

15

specifications

36

store data

20

stored procedures

33

table definitions

29

store data

17, 20

Spotlight on SQL Server Reporting and Trending Guide

Index

stored procedures

33

Support

45
T

table definitions

29

Technical support

45

trending

6
V

views
copy

default

display

open

save

user

7
W

web reports
server location

13
13

48