Customization Guide
Customization Guide
Customization Guide
TABLE OF CONTENTS
Chapter 1. Introduction.........................................................................................1
Chapter 2. Metric Sections..................................................................................... 2
2.1. Section Files............................................................................................... 2
2.2. Section Definition.........................................................................................5
2.3. Metric Options and Filters.............................................................................. 5
2.4. Missing Sections........................................................................................... 6
2.5. Derived Metrics........................................................................................... 7
Chapter 3. Rule System.........................................................................................9
3.1. Writing Rules.............................................................................................. 9
3.2. Integration................................................................................................. 9
3.3. Rule System Architecture.............................................................................. 10
3.4. NvRules API............................................................................................... 11
3.5. Rule File API............................................................................................. 11
3.6. Rule Examples........................................................................................... 12
Chapter 4. Python Report Interface........................................................................ 13
4.1. Basic Usage............................................................................................... 13
4.2. High-Level Interface.................................................................................... 15
4.3. Metric attributes........................................................................................ 15
4.4. NVTX Support............................................................................................ 16
4.5. Sample Script............................................................................................ 17
Chapter 5. Source Counters.................................................................................. 18
Chapter 6. Report File Format.............................................................................. 20
6.1. Version 7 Format........................................................................................ 20
www.nvidia.com
Nsight Compute v2023.2.0 | ii
LIST OF TABLES
www.nvidia.com
Nsight Compute v2023.2.0 | iii
www.nvidia.com
Nsight Compute v2023.2.0 | iv
Chapter 1.
INTRODUCTION
The goal of NVIDIA Nsight Compute is to design a profiling tool that can be easily
extended and customized by expert users. While we provide useful defaults, this allows
adapting the reports to a specific use case or to design new ways to investigate collected
data. All the following is data driven and does not require the tools to be recompiled.
While working with section files or rules files it is recommended to open the Metric
Selection tool window from the Profile menu item. This tool window lists all sections
and rules that were loaded. Rules are grouped as children of their associated section or
grouped in the [Independent Rules] entry. For files that failed to load, the table shows the
error message. Use the Reload button to reload rule files from disk.
www.nvidia.com
Nsight Compute v2023.2.0 | 1
Chapter 2.
METRIC SECTIONS
The Details page consists of sections that focus on a specific part of the kernel analysis
each. Every section is defined by a corresponding section file that specifies the data to be
collected as well as the visualization used in the UI or CLI output for this data. Simply
modify a deployed section file to add or modify what is collected.
www.nvidia.com
Nsight Compute v2023.2.0 | 2
Metric Sections
Identifier: "SampleSection"
DisplayName: "Sample Section"
Description: "This sample section shows information on active warps and cycles."
Header {
Metrics {
Label: "Active Warps"
Name: "smsp__active_warps_avg"
}
Metrics {
Label: "Active Cycles"
Name: "smsp__active_cycles_avg"
}
}
On data collection, this section will cause the two PerfWorks metrics
smsp__active_warps_avg and smsp__active_cycles_avg to be collected.
More advanced elements can be used in the body of a section. See the
ProfilerSection.proto file for which elements are available. The following example
shows how to use these in a slightly more complex example. The usage of regexes is
allowed in tables and charts in the section Body only and follows the format regex:
followed by the actual regex to match PerfWorks metric names.
The supported list of metrics that can be used in sections can be queried using the
command line interface with the --query-metrics option. Each of these metrics can
be used in any section and will be automatically collected if they appear in any enabled
section. Note that even if a metric is used in multiple sections, it will only be collected
once. Look at all the shipping sections to see how they are implemented.
www.nvidia.com
Nsight Compute v2023.2.0 | 3
Metric Sections
Identifier: "SampleSection"
DisplayName: "Sample Section"
Description: "This sample section shows various metrics."
Header {
Metrics {
Label: "Active Warps"
Name: "smsp__active_warps_avg"
}
Metrics {
Label: "Active Cycles"
Name: "smsp__active_cycles_avg"
}
}
Body {
Items {
Table {
Label: "Example Table"
Rows: 2
Columns: 1
Metrics {
Label: "Avg. Issued Instructions Per Scheduler"
Name: "smsp__inst_issued_avg"
}
Metrics {
Label: "Avg. Executed Instructions Per Scheduler"
Name: "smsp__inst_executed_avg"
}
}
}
Items {
Table {
Label: "Metrics Table"
Columns: 2
Order: ColumnMajor
Metrics {
Name: "regex:.*__elapsed_cycles_sum"
}
}
}
Items {
BarChart {
Label: "Metrics Chart"
CategoryAxis {
Label: "Units"
}
ValueAxis {
Label: "Cycles"
}
Metrics {
Name: "regex:.*__elapsed_cycles_sum"
}
}
}
}
www.nvidia.com
Nsight Compute v2023.2.0 | 4
Metric Sections
www.nvidia.com
Nsight Compute v2023.2.0 | 5
Metric Sections
Metrics {
Label: "DRAM Frequency"
Name: "dram__cycles_elapsed.avg.per_second"
Filter {
MaxArch: CC_70
}
Options {
Name: "dram__cycles_elapsed.avg.per_second"
Filter {
MinArch: CC_75
MaxArch: CC_86
}
}
}
In the next example, the metric in the section header is only collected for launch-based
collection scopes (i.e. kernel- and application replay for CUDA kernels or CUDA Graph
nodes), but not in range-based scopes.
Header {
Metrics {
Label: "Theoretical Occupancy"
Name: "sm__maximum_warps_per_active_cycle_pct"
Filter {
CollectionFilter {
CollectionScopes: CollectionScope_Launch
}
}
}
}
Filters can be applied to an entire section instead of or in addition to being set for
individual metrics. If both types of filters are specified, they are combined.
www.nvidia.com
Nsight Compute v2023.2.0 | 6
Metric Sections
MetricDefinitions {
MetricDefinitions {
Name: "derived_metric_name"
Expression: "derived_metric_expr"
}
MetricDefinitions {
...
}
...
}
Since metrics can contain regular values and/or instanced values, elements are combined
as below. Constants are treated as metrics with only a regular value.
www.nvidia.com
Nsight Compute v2023.2.0 | 7
Metric Sections
2. If both metrics have no correlation ids, the first N values are operator-
combined, where N is the minimum of the number of elements in both metrics.
a1 + b1
a2 + b2
a3
a4
3. Else if both metrics have correlation ids, the sets of correlation ids from
both metrics are joined and then operator-combined as applicable.
a1 + b1
a2
b3
a4 + b4
b5
4. Else if only the left-hand side metric has correlation ids, the right-hand
side regular metric value is operator-combined with every element of the left-
hand side metric.
a1 + b
a2 + b
a3 + b
5. Else if only the right-hand side metric has correlation ids, the right-hand
side element values are operator-combined with the regular metric value of the
left-hand side metric.
a + b1 + b2 + b3
In all operations, the value kind of the left-hand side operand is used. If the right-hand
side operand has a different value kind, it is converted. If the left-hand side operand is a
string-kind, it is returned unchanged.
Examples for derived metrics are derived__avg_thread_executed, which
provides a hint on the number of threads executed on average at each instruction, and
derived__uncoalesced_l2_transactions_global, which indicates the ratio of
actual L2 transactions vs. ideal L2 transactions at each applicable instruction.
MetricDefinitions {
MetricDefinitions {
Name: "derived__avg_thread_executed"
Expression: "thread_inst_executed_true / inst_executed"
}
MetricDefinitions {
Name: "derived__uncoalesced_l2_transactions_global"
Expression: "memory_l2_transactions_global /
memory_ideal_l2_transactions_global"
}
MetricDefinitions {
Name: "sm__sass_thread_inst_executed_op_ffma_pred_on_x2"
Expression:
"sm__sass_thread_inst_executed_op_ffma_pred_on.sum.peak_sustained * 2"
}
}
www.nvidia.com
Nsight Compute v2023.2.0 | 8
Chapter 3.
RULE SYSTEM
NVIDIA Nsight Compute features a new Python-based rule system. It is designed as the
successor to the Expert System (un)guided analysis in NVIDIA Visual Profiler, but meant
to be more flexible and more easily extensible to different use cases and APIs.
3.2. Integration
The rule system is integrated into NVIDIA Nsight Compute as part of the profile report
view. When you profile a kernel, available rules will be shown in the report's Details
page. You can either select to apply all available rules at once by clicking Apply Rules at
the top of the page, or apply rules individually. Once applied, the rule results will be
added to the current report. By default, all rules are applied automatically.
www.nvidia.com
Nsight Compute v2023.2.0 | 9
Rule System
www.nvidia.com
Nsight Compute v2023.2.0 | 10
Rule System
API. In addition, some functionality is provided directly by the NvRules module, e.g.
for global error reporting. Finally, since rules are valid Python code, they can use regular
libraries and language functionality that ship with Python as well.
From the rule Context, multiple further objects can be accessed, e.g. the Frontend,
Ranges and Actions. It should be noted that those are only interfaces, i.e. the actual
implementation can vary from tool to tool that decides to implement this functionality.
Naming of these interfaces is chosen to be as API-independent as possible, i.e. not to
imply CUDA-specific semantics. However, since many compute and graphics APIs
map to similar concepts, it can easily be mapped to CUDA terminology, too. A Range
refers to a CUDA stream, an Action refers to a single CUDA kernel instance. Each action
references several Metrics that have been collected during profiling (e.g. instructions
executed) or are statically available (e.g. the launch configuration). Metrics are accessed
via their names from the Action.
Each CUDA stream can contain any number of kernel (or other device activity) instances
and so each Range can reference one or more Actions. However, currently only a single
Action per Range will be available, as only a single CUDA kernel can be profiled at once.
The Frontend provides an interface to manipulate the tool UI by adding messages,
graphical elements such as line and bar charts or tables, as well as speedup estimations,
focus metrics and source markers. The most common use case is for a rule to
show at least one message, stating the result to the user, as illustrated in extras/
RuleTemplates/BasicRuleTemplate.py This could be as simple as "No issues
have been detected," or contain direct hints as to how the user could improve the
code, e.g. "Memory is more heavily utilized than Compute. Consider whether it is
possible for the kernel to do more compute work." For more advanced use cases, such as
adding speedup estimates, key performance indicators (a.k.a. focus metrics) or source
markers to annotate individual lines of code to your rule, see the templates in extras/
RuleTemplates.
www.nvidia.com
Nsight Compute v2023.2.0 | 11
Rule System
‣ apply(handle): Apply this rule to the rule context provided by handle. Use
NvRules.get_context(handle) to obtain the Context interface from handle.
‣ get_name(): Return the user-consumable display name of this rule.
‣ get_description(): Return the user-consumable description of this rule.
Optional Functions
‣ get_section_identifier(): Return the unique section identifier that maps this
rule to a section. Section-mapped rules will only be available if the corresponding
section was collected. They implicitly assume that the metrics requested by the
section are collected when the rule is applied.
‣ evaluate(handle):
Declare required metrics and rules that are necessary for this rule to be applied. Use
NvRules.require_metrics(handle, [...]) to declare the list of metrics that
must be collected prior to applying this rule.
Use e.g. NvRules.require_rules(handle, [...]) to declare the list of other
rules that must be available before applying this rule. Those are the only rules that
can be safely proposed by the Controller interface.
import NvRules
def get_identifier():
return "GpuArch"
def apply(handle):
ctx = NvRules.get_context(handle)
action = ctx.range_by_idx(0).action_by_idx(0)
ccMajor =
action.metric_by_name("device__attribute_compute_capability_major").as_uint64()
ctx.frontend().message("Running on major compute capability " + str(ccMajor))
www.nvidia.com
Nsight Compute v2023.2.0 | 12
Chapter 4.
PYTHON REPORT INTERFACE
Importing a report
Once the module is imported, you can load a report file by calling the load_report
function with the path to the file. This function returns an object of type IContext
which holds all the information concerning that report.
Querying ranges
1
On Linux machines you will also need a GNU-compatible libc and libgcc_s.so.
www.nvidia.com
Nsight Compute v2023.2.0 | 13
Python Report Interface
When working with the Python module, kernel profiling results are grouped into
ranges which are represented by IRange objects. You can inspect the number of ranges
contained in the loaded report by calling the num_ranges() member function of an
IContext object and retrieve a range by its index using range_by_idx(index).
>>> my_context.num_ranges()
1
>>> my_range = my_context.range_by_idx(0)
Querying actions
Inside a range, kernel profiling results are called actions. You can query the number of
actions contained in a given range by using the num_actions method of an IRange
object.
>>> my_range.num_actions()
2
In the same way ranges can be obtained from an IContext object by using the
range_by_idx(index) method, individual actions can be obtained from IRange objects
by using the action_by_idx(index) method. The resulting actions are represented by
the IAction class.
>>> my_action.name()
MyKernel
Querying metrics
To get a tuple of all metric names contained within an action you can use the
metric_names() method. It is meant to be combined with the metric_by_name()
method which returns an IMetric object. However, for the same task you may also use
the [] operator, as explained in the High-Level Interface section below.
The metric names displayed here are the same as the ones you can use with the --
metrics flag of NVIDIA Nsight Compute. Once you have extracted a metric from an
action, you can obtain its value by using one of the following three methods:
‣ as_string() to obtain its value as a Python str
‣ as_uint64() to obtain its value as a Python int
‣ as_double() to obtain its value as a Python float
For example, to print the display name of the GPU on which the kernel was profiled you
can query the device__attribute_display_name metric.
www.nvidia.com
Nsight Compute v2023.2.0 | 14
Python Report Interface
>>> display_name_metric =
my_action.metric_by_name('device__attribute_display_name')
>>> display_name_metric.as_string()
'NVIDIA GeForce RTX 3060 Ti'
Note that accessing a metric with the wrong type can lead to unexpected (conversion)
results.
>>> display_name_metric.as_double()
0.0
There is also a convenience method IMetric.value() which allows you to query the
value of a metric object without knowledge of its type:
>>> display_name_metric.value()
'NVIDIA GeForce RTX 3060 Ti'
All the available methods of a class, as well as their associated Python docstrings, can be
looked up interactively via
>>> help(ncu_report.IMetric)
or similarly for other classes and methods. In your code, you can access the docstrings
via the __doc__ attribute, i.e. ncu_report.IMetric.value.__doc__.
www.nvidia.com
Nsight Compute v2023.2.0 | 15
Python Report Interface
‣ metric_subtype()
‣ rollup_operation()
‣ unit()
‣ description()
The first method metric_type() returns one out of three enum values
(IMetric.MetricType_COUNTER, IMetric.MetricType_RATIO,
IMetric.MetricType_THROUGHPUT) if the metric is a hardware metric, or
IMetric.MetricType_OTHER otherwise (e.g. for launch or device attributes).
The method metric_subtype() returns an enum value representing the
subtype of a metric (e.g. IMetric.MetricSubtype_PEAK_SUSTAINED,
IMetric.MetricSubtype_PER_CYCLE_ACTIVE). In case a metric does not
have a subtype, None is returned. All available values (without the necessary
IMetric.MetricSubtype_ prefix) may be found in the NvRules API documentation,
or may be looked up interactively by executing help(ncu_report.IMetric).
IMetric.rollup_operation() returns the operation which is used to accumulate
different values of the same metric and can be one of IMetric.RollupOperation_AVG,
IMetric.RollupOperation_MAX, IMetric.RollupOperation_MIN or
IMetric.RollupOperation_SUM for averaging, maximum, minimum or summation,
respectively. If the metric in question does not specify a rollup operation None will be
returned.
Lastly, unit() and description() return a (possibly empty) string of the metric's unit
and a short textual description for hardware metrics, respectively.
The above methods can be combined to filter through all metrics of a report, given
certain criteria:
www.nvidia.com
Nsight Compute v2023.2.0 | 16
Python Report Interface
There is also a actions_by_nvtx member function in the IRange class which allows
you to get a tuple of actions matching the NVTX state described in its parameter.
The parameters for the actions_by_nvtx function are two lists of strings representing
the state for which we want to query the actions. The first parameter describes the NVTX
states to include while the second one describes the NVTX states to exclude. These
strings are in the same format as the ones used with the --nvtx-include and --nvtx-
exclude options.
#!/usr/bin/env python3
import sys
import ncu_report
if len(sys.argv) != 2:
print("usage: {} report_file".format(sys.argv[0]), file=sys.stderr)
sys.exit(1)
report = ncu_report.load_report(sys.argv[1])
www.nvidia.com
Nsight Compute v2023.2.0 | 17
Chapter 5.
SOURCE COUNTERS
The Source page provides correlation of various metrics with CUDA-C, PTX and SASS
source of the application, depending on availability.
Which Source Counter metrics are collected and the order in which they are displayed
in this page is controlled using section files, specifically using the ProfilerSectionMetrics
message type. Each ProfilerSectionMetrics defines one ordered group of metrics, and
can be assigned an optional Order value. This value defines the ordering among those
groups in the Source page. This allows, for example, you to define a group of memory-
related source counters in one and a group of instruction-related counters in another
section file.
Identifier: "SourceMetrics"
DisplayName: "Custom Source Metrics"
Metrics {
Order: 2
Metrics {
Label: "Instructions Executed"
Name: "inst_executed"
}
Metrics {
Label: ""
Name: "collected_but_not_shown"
}
}
If a Source Counter metric is given an empty label attribute in the section file, it will be
collected but not shown on the page.
www.nvidia.com
Nsight Compute v2023.2.0 | 18
Source Counters
www.nvidia.com
Nsight Compute v2023.2.0 | 19
Chapter 6.
REPORT FILE FORMAT
This section documents the internals of the profiler report files (reports in the following)
as created by NVIDIA Nsight Compute. The file format is subject to change in future
releases without prior notice.
www.nvidia.com
Nsight Compute v2023.2.0 | 20
Report File Format
www.nvidia.com
Nsight Compute v2023.2.0 | 21
Notice
ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS,
DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY,
"MATERIALS") ARE BEING PROVIDED "AS IS." NVIDIA MAKES NO WARRANTIES,
EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE
MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF
NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR
PURPOSE.
Information furnished is believed to be accurate and reliable. However, NVIDIA
Corporation assumes no responsibility for the consequences of use of such
information or for any infringement of patents or other rights of third parties
that may result from its use. No license is granted by implication of otherwise
under any patent rights of NVIDIA Corporation. Specifications mentioned in this
publication are subject to change without notice. This publication supersedes and
replaces all other information previously supplied. NVIDIA Corporation products
are not authorized as critical components in life support devices or systems
without express written approval of NVIDIA Corporation.
Trademarks
NVIDIA and the NVIDIA logo are trademarks or registered trademarks of NVIDIA
Corporation in the U.S. and other countries. Other company and product names
may be trademarks of the respective companies with which they are associated.
Copyright
© 2018-2023 NVIDIA Corporation and affiliates. All rights reserved.
This product includes software developed by the Syncro Soft SRL (http://
www.sync.ro/).
www.nvidia.com