Debugger Markup

Starting with the 6.6.07 version of the debugger we have included a new mechanism for
enhancing output from the debugger and extensions: debugger markup language (DML). As
with HTML the debugger’s markup support allows output to include directives and extra non-
display information in the form of tags. Debugger user interfaces can parse out the extra
information to provide new behaviors.

DML is primarily intended to address two issues:

 Linking of related information. One of the DML tags is a link tag which lets an output
producer indicate that information related to a piece of output can be accessed via the
link’s stated action. As with HTML links this allows user to navigate hyperlinked
information in an intuitive way.

 Discoverability of debugger and extension functionality. The debugger and its

extensions contain a huge body of functionality but very little of it is apparent. Users
must simply know what is available, meaning that often users are unaware of
commands which could help them. DML links let arbitrary commands be wrapped in
alternate presentations, such as descriptive text, menu systems or linked help.

Another key thing to understand about DML is what it is not.

 DML is not HTML. DML is deliberately very simple and has only a handful of tags.
Moving forward this simplicity will be maintained as DML is not intended to grow into a
full presentation language. As there is a huge body of established debugger code based
on an output stream of plain text a goal of DML is to allow simple translation between
DML and plain text to support existing interfaces. This requires limiting DML to tags that
are not critical to presentation. Effects such as colors can easily be supported since
removing them does not remove the text carrying the actual information. On the other
hand, formatting tags such as positioning would not be supported as there is no
mapping to a stream of plain text.

 DML is not XML. DML does not attempt to carry semantic nor structured information.
As mentioned above, there must be a simple mapping between DML and plain text, thus
DML tags are all discardable. DML is not extensible; all tags must be defined by the
debugger team.
As DML is text markup that can flow through the usual text handling channels in dbgeng and
dbgeng remoting it is fully compatible with remoting. Just as with a web browser and server a
DML “browser” will render content returned from the server, thus all DML content is inherently
remotable. An extension or command on the server can produce rich or plain text on a client
depending on the client user interface support for DML.

DML Content Specification

DML content roughly follows the XML/HTML rules for special characters. The characters &, <, >
and “ are special and cannot be used in plain text. The equivalent escaped versions are &amp;,
&lt;, &gt; and &quot;.

Example:

The text

“Alice & Bob think 3 < 4”

would be converted to the DML

&quot;Alice &amp; Bob think 3 &lt; 4&quot;

A significant departure from XML/HTML rules is that DML text can include stream-style
formatting characters such as \b, \t, \r and \n. This is for compatibility with existing debugger
text production and consumption.

DML tags are given as a starting <tagname [args]> and a following </tagname>. Currently all
tags come in begin/end pairs but eventually DML might include XML-like singletons.

DML Tag Reference

<link [name=”text”] [cmd=”debugger_command”] [section=”name”]>link text</link>
The link tag is the basic hyperlinking mechanism in DML. It directs user interfaces which support
DML presentation to display the link text as a clickable link. When a link with a cmd
specification is clicked the debugger command is executed and its output should replace the
current output.

The name and section arguments allow for navigation between named links, similar to HTML’s
<a name> and #name support. When a link that has a section argument is clicked on the UI will
scan for a link named with a matching name and will scroll that into view. This allows links to
point to different sections of the same page (or a particular section of a new page). DML’s
section name is separate to avoid having to define a new syntax which would allow a section
name at the end of the command string.

Conversion to plain text drops the tags.

<exec cmd=”debugger_command”>descriptive text</exec>

An exec tag is similar to a link tag in that the descriptive text should be presented as a clickable
item. However, in the exec case the given command is executed without replacing the current
output, thus this gives a way to simply have commands executed with a click, such as from a
menu.

Conversion to plain text drops the tags.

<b>text</b>, <i>text</i>, <u>text</u>

These tags request bold, italic and underlined text, respectively. They can be nested to have a
mix of the properties.

Conversion to plain text drops the tags.

<col fg="name" bg="name">text</col>

Request foreground and background colors for the text. The colors are given as names of
known colors instead of absolute values as that allows customers to control what kind of color
they see. Current color names (defaults only apply to windbg):

 wbg and wfg – Default window background and foreground colors. Default to system
colors for window and window text.

 clbg and clfg – Current line background and foreground colors. Default to system colors
for highlight and highlight text.

 changed – Used for data that has changed since a previous stop point, such as changed
registers in windbg. Defaults to red.

 srcnum, srcchar, srcstr, srcid, srckw, srcpair, srccmnt, srcdrct, srcspid, srcannot – Source
element colors. Defaults can be seen in windbg.

 empbg and emphfg – Emphasized text. Defaults to light blue.

 subbg and subfg – Subdued text. Default to system color for inactive caption text and
inactive captions.

 normbg, normfg, warnbg, warnfg, errbg, errfg, verbbg, verbfg – Output level colors.
Defaults can be seen in windbg.

Conversion to plain text drops the tag.

DML Additions to the dbgeng Interface

DML ultimately is just text with embedded tags and some rules about special characters.
dbgeng already has a set of text handling input methods and output interfaces, thus the
addition of DML only requires specification of the type of content carried in input and output
text.

Providing DML Content to dbgeng

A new output control flag, DEBUG_OUTCTL_DML, indicates that the text generated by a dbgeng
method should be handled as DML content. If this flag is not given the text is treated as plain
text context, just as before. DEBUG_OUTCTL_DML can be used with ControlledOutput,
ControlledOutputVaList, ControlledOutputWide and ControlledOutputVaListWide. Text given
must follow the DML rules for valid characters.

An additional control flag, DEBUG_OUTCTL_AMBIENT_DML, allows specification of DML context

text without modifying any out output control attributes. DEBUG_OUTCTL_AMBIENT_TEXT has
been added also as a more-descriptive alias for the previously-existing
DEBUG_OUTCTL_AMBIENT.

All output routines have been enhanced to allow a new format specifier %[h|w]Y{t}. This format
specifier has a string pointer as an argument and indicates that the given text is plain text and
should be converted to DML format during output processing. This gives callers a simple way of
including plain text in DML content without having to pre-convert to DML format themselves.
The h and w qualifiers indicate ANSI or Unicode text, as with %s.

Providing DML Content From a Debuggee

dbgeng has been enhanced to scan debuggee output for a special marker – <?dml?> – that
indicates the remaining text in a piece of debuggee output should be treated as DML. The mode
change only applies to a single piece of debuggee output, such as a single OutputDebugString
string, and is not a global mode switch.

Example:

OutputDebugString(“This is plain text\n<?dml?>This is <col

fg=\”emphfg\”>DML</col> text\n”);

This piece of output will have a line of plain text followed by a line of DML where the acronym
DML is done in a different color.

IDebugOutputCallbacks2
IDebugOutputCallbacks2 allows dbgeng interface clients to receive full DML content for
presentation. IDebugOutputCallbacks2 is an extension of IDebugOutputCallbacks (not
IDebugOutputCallbacksWide) so that it can be passed in to the existing SetOutputCallbacks
method. The engine will do a QueryInterface for IDebugOutputCallbacks2 to see which
interface the incoming output callback object supports. If the object supports
IDebugOutputCallbacks2 all output will be sent through the extended IDebugOutputCallbacks2
methods; the basic IDebugOutputCallbacks::Output method will not be used. The new methods
are:

 GetInterestMask – Allows the callback object to describe which kinds of output

notifications it wants to receive. The basic choice is between plain text content
 (DEBUG_OUTCBI_TEXT) and DML content (DEBUG_OUTCBI_DML). In addition the
callback object can also request notification of explicit flushes
(DEBUG_OUTCBI_EXPLICIT_FLUSH).

 Output2 – All IDebugOutputCallbacks2 notifications come through Output2. The Which

parameter indicates what kind of notification is coming in while the Flags, Arg and Text
parameters carry the notification payload. Current notifications are:

o DEBUG_OUTCB_TEXT – Plain text output. Flags are from DEBUG_OUTCBF_*,

Arg is the output mask and Text is the plain text. This will only be received if
DEBUG_OUTCBI_TEXT was given in the interest mask.

o DEBUG_OUTCB_DML – DML content output. Flags are from DEBUG_OUTCBF_*,

Arg is the output mask and Text is the DML content. This will only be received if
DEBUG_OUTCBI_DML was given in the interest mask.

o DEBUG_OUTCB_EXPLICIT_FLUSH – A caller has called FlushCallbacks with no

buffered text. Normally when buffered text is flushed the
DEBUG_OUTCBF_COMBINED_EXPLICIT_FLUSH flag will be set, folding the two
notifications into one. If no text is buffered a flush-only notification is sent.

Note that an output object can register for both text and DML content if it can handle them
both. During output processing of the callback the engine will pick the format that reduces
conversions, thus supporting both may reduce conversions in the engine. It is not necessary,
though, and supporting only one format is the expected mode of operation.

Automatic Conversions
dbgeng will automatically convert between plain text and DML as necessary. For example, if a
caller sends DML content to the engine the engine will convert it to plain text for all output
clients which only accept plain text. Alternately, the engine will convert plain text to DML for all
output callbacks which only accept DML.

New Use of DML

Several engine commands have been enhanced and a few have been added during validation of
the initial DML implementation. These commands are not intended to be comprehensive, but
they do show some of what can be done with DML.

New Commands
.dml_flow <start> <target>

.dml_flow allows for interactive exploration of code flow for a function. It builds a code flow
graph for the function starting at the given start address (similar to uf). It then shows the basic
block given the target address plus links to referring blocks and blocks referred to by the current
block. The intent of the DML version command is to show how simple use of links enables easy
navigation of something that would be very tedious to do with typed commands.

.dml_start [<filename>]

.dml_start is intended to be a “start page” for DML, allowing exploration of available commands.
The default implementation is a simple set of links that allow navigation of some debuggee
content, core commands, available extensions and dot commands. Users can customize
.dml_start by explicitly providing a filename to read or by setting the DBGENG_START_FILE
environment variable. If .dml_start is given a file it reads DML from the file and displays it.

!dml_proc

!dml_proc is a new extension which displays current processes and allows drilling into processes
for more information. The top-level process links go to more-detailed process information, such
as a thread list. Thread list entries link to thread information and potentially stack information
down to individual frames. Kernel-mode process displays allow setting and resetting user-mode
state – embedded .process exec tags – and full !process 0 7 output. !dml_proc works for both
user- and kernel-mode, auto-selecting what information to display.

.prefer_dml [0|1]

.prefer_dml controls a global setting as to whether DML-enhanced commands should default to

DML mode or not.

Enhanced Commands
.help /D

.help has a new DML mode where a top bar of links is given, allowing display of commands that
start with a particular letter. This gives more convenient browsing than having to read the full
list. Linked to by .dml_start.

.chain /D

.chain has a new DML mode where extensions are linked to a .extmatch command displaying
the commands for an extension DLL. Linked to by .dml_start.
.extmatch /D

.extmatch has a new DML mode where extension commands are links to help for the command.
This is only supported when the extension advertises that it has per-command help, such as
uexts. Any dbgeng-style extension can participate by returning the new
DEBUG_EXTINIT_HAS_COMMAND_HELP flag from initialization and having their !help
implementation take a command name as an argument.

lmD

lm has a new DML mode where module names link to an lmv command giving details on a
particular module. The column headers are also active links to allow for selection of sorting by
name or by start address.

kM

k has a new DML mode where frame numbers link to a .frame/dv command which displays
locals for the frame.

.printf /D

The /D option for .printf indicates that the string produced by the .printf is DML content. This
allows a script or command to produce DML-enhanced output. .printf is not affected by
.prefer_dml.

Command Window/Console Enhancements

All of the Windows debuggers now have command output areas which support DML parsing. In
windbg the command window supports all DML behavior and will show colors, font styles and
links. The console debuggers – ntsd, cdb and kd – only support the color attributes of DML, and
the only when running in a true console with color mode enabled. Debuggers with redirected
I/O, ntsd –d or remote.exe sessions will not display any colors (and in fact will not even use
IDebugOutputCallbacks2).

All of the Windows debuggers have dual I/O paths so that they fall back on using
IDebugOutputCallbacks[Wide] when running on or connected to dbgeng.dll versions that do not
support IDebugOutputCallbacks2.
windbg’s Command Browser Window
windbg has a new user interface element which parses and displays DML: the command
browser window. The command browser window works similarly to the existing command
window, with a place for output display and a text entry bar. However, the command browser
window is intended to collect the full output of a single command for display, similar to a web
page. All tags – link, exec and appearance modifications – are fully supported.

The command browser window deliberately mimics the behavior of a web browser, with a drop-
down history and forward/back buttons (app commands for forward and back are supported so
extra mouse buttons work). The history drop-down only displays the last twenty commands but
full history is kept so by going back in the commands you can get the drop-down to display older
history.

Command browser windows can be either auto-refresh or manual-refresh. Auto-refresh

browsers will automatically re-run their command on debugger state changes. This keeps the
output “live” but at the cost of executing the command on all changes (the debugger has no way
of knowing exactly what state the command relies on and thus cannot optimize refresh). Auto-
refresh is on by default. If the browser does not need to be live the window’s context menu can
be used to disable auto-refresh. Command windows will also automatically disable auto-refresh
if they detect that a command is causing debugger changes, because in that case the window
would refresh continuously.

A “recent commands” sub-window has been added to the View menu to hold commands of
interest. Selecting a recent command opens a new browser with the given command. There is a
menu item on the browser window’s context menu that adds the window’s current command to
the list of recent commands. The list of recent commands is persisted in workspaces.

The View menu has a “Set Browser Start Command” option which allows a user to set a
preferred command for new browser windows to start with, such as .dml_start. This command
is saved in workspaces.

Links have a right-click context menu similar to the right-click context menu in a web browser.
Links can be followed or followed in a new browser window. A link’s command can be copied to
the clipboard for use. Links do not currently have hover popups that display link commands.
Due to a richedit limitation clicking too far to the start of the first character of a link may not be
recognized. If it isn’t, try clicking a little farther away from the beginning.

.browse <command> in the command window will open a new command browser window and
execute the given command.

You can have as many command windows open at once as you like. Command windows persist
in workspaces but only save the current command; the history is not kept.

Command browser windows can run any debugger command, it does not have to be a
command that produces DML. You can use browser windows to have an arbitrary set of
commands active for inspection regardless of DML. Command browser window commands are
executed by the engine, not by the user interface. This means that user-interface specific
commands, such as .cls, cannot be used in command browser windows. It also means that
when the user interface is a remote client the command will be executed by the server, not by
the client, and will show server state.

The command browser window executes the command synchronously and so does not display
output until the command has completed. There is currently no way to have output produced
as the command runs; long-running commands will not show anything until they have finished.

Command completion in the input bar is supported.

A new command browser window can be opened with Ctrl+N.

Console Debugger Color Mode

As mentioned previously, ntsd, cdb and kd now have the ability to display colored output when
running in a true console. This is not the default, it requires color mode to be explicitly enabled
via tools.ini. The new col_mode <true|false> token in tools.ini controls the color mode setting.
When color mode is enabled the debugger can produce colored output. By default most colors
are not set and instead default to the current console colors. The new col <name> <colspec>
token in tools.ini allows users to define whatever colors they want for particular settings. The
color names are the same color names discussed above under the “col” tag. colspec is a three-
letter RGB indicator of the form [rR-][gG-][bB-]. A lower-case letter indicates darker, an upper-
case letter indicates brighter and a dash indicates no color component contribution. Due to
console color limitations bright is not actually per-component, but applies to all components if
any request bright. In other words, rgB is the same as RGB. For this reason it is recommended
that all caps be used if any caps are going to be used. Examples:

 R--: Bright red.

 -g-: Dark green.

 -gb: Dark cyan.

 RG-: Bright yellow (same as Rg-, but RG- is the preferred form).

 ---: Black.

 RGB: White.