Roll Your Own: Create a Language Compiler for the .

NET Framework Page 1 of 26

Roll Your Own

Create a Language Compiler for the .NET Framework

Joel Pobar

This article discusses:

l Language definition
This article uses the following technologies:
l The phases of a compiler
l The CLR abstract stack
.NET Framework
l Tools for getting your IL right
Code download available at: CompilerWriting2008_02.exe (158KB)

Contents
Language Definition
High-Level Architecture
The Scanner
The Parser
Targeting the .NET Framework
Tools for Getting Your IL Right
The Code Generator
Wrapping Up ... Almost
Dynamic Method Invocation
Using LCG to Perform Quick Late-Binding
The Dynamic Language Runtime

C
ompiler hackers are celebrities in the world of computer science. I've seen Anders Hejlsberg deliver a presentation at the Professional Developers Conference and then walk
off stage to a herd of men and women asking him to sign books and pose for photographs. There's a certain intellectual mystique about individuals who dedicate their time to
learning and understanding the ins and outs of lambda expressions, type systems, and assembly languages. Now, you too can share some of this glory by writing your own
compiler for the Microsoft .NET Framework.

There are hundreds of compilers for dozens of languages that target the .NET Framework. The .NET CLR wrangles these languages into the same sandpit to play and interact
together peacefully. A ninja developer can take advantage of this when building large software systems, adding a bit of C# and a dabble of Python. Sure, these developers are
impressive, but they don't compare to the true mastersthe compiler hackers, for it is they who have a deep understanding of virtual machines, language design, and the nuts
and bolts of these languages and compilers.

In this article, I will walk you through the code for a compiler written in C# (aptly called the "Good for Nothing" compiler), and along the way I will introduce you to the
high-level architecture, theory, and .NET Framework APIs that are required to build your own .NET compiler. I will start with a language definition, explore compiler
architecture, and then walk you through the code generation subsystem that spits out a .NET assembly. The goal is for you to understand the foundations of compiler
development and get a firm, high-level understanding of how languages target the CLR efficiently. I won't be developing the equivalent of a C# 4.0 or an IronRuby, but this
discussion will still offer enough meat to ignite your passion for the art of compiler development.

Language Definition

Software languages begin with a specific purpose. This purpose can be anything from expressiveness (such as Visual Basic), to productivity (such as Python, which aims to
get the most out of every line of code), to specialization (such as Verilog, which is a hardware description language used by processor manufacturers), to simply satisfying the
author's personal preferences. (The creator of Boo, for instance, likes the .NET Framework but is not happy with any of the available languages.)

Once you've stated the purpose, you can design the languagethink of this as the blueprint for the language. Computer languages must be very precise so that the
programmer can accurately express exactly what is required and so that the compiler can accurately understand and generate executable code for exactly what's expressed.
The blueprint of a language must be specified to remove ambiguity during a compiler's implementation. For this, you use a metasyntax, which is a syntax used to describe the
syntax of languages. There are quite a few metasyntaxes around, so you can choose one according to your personal taste. I will specify the Good for Nothing language using a
metasyntax called EBNF (Extended Backus-Naur Form).

It's worth mentioning that EBNF has very reputable roots: it's linked to John Backus, winner of the Turing Award and lead developer on FORTRAN. A deep discussion of
EBNF is beyond the scope of the article, but I can explain the basic concepts.

The language definition for Good for Nothing is shown in Figure 1. According to my language definition, Statement (stmt) can be variable declarations, assignments, for
loops, reading of integers from the command line, or printing to the screenand they can be specified many times, separated by semicolons. Expressions (expr) can be
strings, integers, arithmetic expressions, or identifiers. Identifiers (ident) can be named using an alphabetic character as the first letter, followed by characters or numbers. And
so on. Quite simply, I've defined a language syntax that provides for basic arithmetic capabilities, a small type system, and simple, console-based user interaction.

You might have noticed that this language definition is short on specificity. I haven't specified how big the number can be (such as if it can be bigger than a 32-bit integer) or
even if the number can be negative. A true EBNF definition would precisely define these details, but, for the sake of conciseness, I will keep my example here simple.

Here is a sample Good for Nothing language program:

var ntimes = 0;
print "How much do you love this company? (1-10) ";
read_int ntimes;
var x = 0;
for x = 0 to ntimes do
print "Developers!";
end;
print "Who said sit down?!!!!!";

You can compare this simple program with the language definition to get a better understanding of how the grammar works. And with that, the language definition is done.

High-Level Architecture

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 2 of 26

A compiler's job is to translate high-level tasks created by the programmer into tasks that a computer processor can understand and execute. In other words, it will take a
program written in the Good for Nothing language and translate it into something that the .NET CLR can execute. A compiler achieves this through a series of translation
steps, breaking down the language into parts that we care about and throwing away the rest. Compilers follow common software design principlesloosely coupled
components, called phases, plugged together to perform translation steps. Figure 2 illustrates the components that perform the phases of a compiler: the scanner, parser, and
code generator. In each phase, the language is broken down further, and that information about the program's intention is served to the next phase.

Figure 2 The Compiler's Phases

Figure 2 The Compiler's Phases (Click the image for a larger

view)

Compiler geeks often abstractly group the phases into front end and back end. The front end consists of scanning and parsing, while the back end typically consists of code
generation. The front end's job is to discover the syntactic structure of a program and translate that from text into a high-level in-memory representation called an Abstract
Syntax Tree (AST), which I will elaborate on shortly. The back end has the task of taking the AST and converting it into something that can be executed by a machine.

The three phases are usually divided into a front end and a back end because scanners and parsers are typically coupled together, while the code generator is usually tightly
coupled to a target platform. This design allows a developer to replace the code generator for different platforms if the language needs to be cross-platform.

I've made the code for the Good for Nothing compiler available in the code download accompanying this article. You can follow along as I walk through the components of
each phase and explore the implementation details.

The Scanner

A scanner's primary job is to break up text (a stream of characters in the source file) into chunks (called tokens) that the parser can consume. The scanner determines which
tokens are ultimately sent to the parser and, therefore, is able to throw out things that are not defined in the grammar, like comments. As for my Good for Nothing language,
the scanner cares about characters (A-Z and the usual symbols), numbers (0-9), characters that define operations (such as +, -, *, and /), quotation marks for string
encapsulation, and semicolons.

A scanner groups together streams of related characters into tokens for the parser. For example, the stream of characters " h e l l o w o r l d ! " would be grouped into one
token: "hello world!".

The Good for Nothing scanner is incredibly simple, requiring only a System.IO.TextReader on instantiation. This kicks off the scanning process, as shown here:

public Scanner(TextReader input)

{
this.result = new Collections.List<object>();
this.Scan(input);
}

Figure 3 illustrates the Scan method, which has a simple while loop that walks over every character in the text stream and finds recognizable characters declared in the
language definition. Each time a recognizable character or chunk of characters is found, the scanner creates a token and adds it to a List<object>. (In this case, I type it as
object. However, I could have created a Token class or something similar to encapsulate more information about the token, such as line and column numbers.)

You can see that when the code encounters a " character, it assumes this will encapsulate a string token; therefore, I consume the string and wrap it up in a StringBuilder
instance and add it to the list. After scan builds the token list, the tokens go to the parser class through a property called Tokens.

The Parser

The parser is the heart of the compiler, and it comes in many shapes and sizes. The Good for Nothing parser has a number of jobs: it ensures that the source program conforms
to the language definition, and it handles error output if there is a failure. It also creates the in-memory representation of the program syntax, which is consumed by the code
generator, and, finally, the Good for Nothing parser figures out what runtime types to use.

The first thing I need to do is take a look at the in-memory representation of the program syntax, the AST. Then I'll take a look at the code that creates this tree from the
scanner tokens. The AST format is quick, efficient, easy to code, and can be traversed many times over by the code generator. The AST for the Good for Nothing compiler is
shown in Figure 4.

A quick glance at the Good for Nothing language definition shows that the AST loosely matches the language definition nodes from the EBNF grammar. It's best to think of
the language definition as encapsulating the syntax, while the abstract syntax tree captures the structure of those elements.

There are many algorithms available for parsing, and exploring all of them is beyond the scope of this article. In general, they differ in how they walk the stream of tokens to
create the AST. In my Good for Nothing compiler, I use what's called an LL (Left-to-right, Left-most derivation) top-down parser. This simply means that it reads text from
left to right, constructing the AST based on the next available token of input.

The constructor for my parser class simply takes a list of tokens that was created by the scanner:

public Parser(IList<object> tokens)

{
this.tokens = tokens;
this.index = 0;
this.result = this.ParseStmt();

if (this.index != this.tokens.Count)
throw new Exception("expected EOF");
}

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 3 of 26

The core of the parsing work is done by the ParseStmt method, as shown in Figure 5. It returns a Stmt node, which serves as the root node of the tree and matches the
language syntax definition's top-level node. The parser traverses the list of tokens using an index as the current position while identifying tokens that are subservient to the
Stmt node in the language syntax (variable declarations and assignments, for loops, read_ints, and prints). If a token can't be identified, an exception is thrown.

When a token is identified, an AST node is created and any further parsing required by the node is performed. The code required for creating the print AST node is as follows:

// <stmt> := print <expr>

if (this.tokens[this.index].Equals("print"))
{
this.index++;
Print print = new Print();
print.Expr = this.ParseExpr();
result = print;
}

Two things happen here. The print token is discarded by incrementing the index counter, and a call to the ParseExpr method is made to obtain an Expr node, since the
language definition requires that the print token be followed by an expression.

Figure 6 shows the ParseExpr code. It traverses the list of tokens from the current index, identifying tokens that satisfy the language definition of an expression. In this case,
the method simply looks for strings, integers, and variables (which were created by the scanner instance) and returns the appropriate AST nodes representing these
expressions.

For string statements that satisfy the "<stmt> ; <stmt>" language syntax definition, the sequence AST node is used. This sequence node contains two pointers to stmt nodes
and forms the basis of the AST tree structure. The following details the code used to deal with the sequence case:

if (this.index < this.tokens.Count && this.tokens[this.index] ==

Scanner.Semi)
{
this.index++;

if (this.index < this.tokens.Count &&

!this.tokens[this.index].Equals("end"))
{
Sequence sequence = new Sequence();
sequence.First = result;
sequence.Second = this.ParseStmt();
result = sequence;
}
}

The AST tree shown in Figure 7 is the result of the following snippet of Good for Nothing code:

Figure 7 helloworld.gfn AST Tree and High-Level Trace

Figure 7 helloworld.gfn AST Tree and High-Level Trace

(Click the image for a larger view)

var x = "hello world!";

print x;

Targeting the .NET Framework

Before I get into the code that performs code generation, I should first step back and discuss my target. So here I will describe the compiler services that the .NET CLR
provides, including the stack-based virtual machine, the type system, and the libraries used for .NET assembly creation. I'll also briefly touch on the tools that are required to
identify and diagnose errors in compiler output.

The CLR is a virtual machine, meaning it is a piece of software that emulates a computer system. Like any computer, the CLR has a set of low-level operations it can perform,
a set of memory services, and an assembly language to define executable programs. The CLR uses an abstract stack-based data structure to model code execution, and an
assembly language, called Intermediate Language (IL), to define the operations you can perform on the stack.

When a computer program defined in IL is executed, the CLR simply simulates the operations specified against a stack, pushing and popping data to be executed by an
instruction. Suppose you want to add two numbers using IL. Here's the code used to perform 10 + 20:

ldc.i4 10
ldc.i4 20
add

The first line (ldc.i4 10) pushes the integer 10 onto the stack. Then the second line (ldc.i4 20) pushes the integer 20 onto the stack. The third line (add) pops the two integers
off the stack, adds them, and pushes the result onto the stack.

Simulation of the stack machine occurs by translating IL and the stack semantics to the underlying processor's machine language, either at run time through just-in-time (JIT)
compilation or before hand through services like Native Image Generator (Ngen).

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 4 of 26

There are lots of IL instructions available for building your programsthey range from basic arithmetic to flow control to a variety of calling conventions. Details about all
the IL instructions can be found in Partition III of the European Computer Manufacturers Association (ECMA) specification (available at msdn2.microsoft.com/aa569283).

The CLR's abstract stack performs operations on more than just integers. It has a rich type system, including strings, integers, booleans, floats, doubles, and so on. In order for
my language to run safely on the CLR and interoperate with other .NET-compliant languages, I incorporate some of the CLR type system into my own program. Specifically,
the Good for Nothing language defines two typesnumbers and stringswhich I map to System.Int32 and System.String.

The Good for Nothing compiler makes use of a Base Class Library (BCL) component called System.Reflection.Emit to deal with IL code generation and .NET assembly
creation and packaging. It's a low-level library, which sticks close to the bare metal by providing simple code-generation abstractions over the IL language. The library is also
used in other well-known BCL APIs, including System.Xml.XmlSerializer.

The high-level classes that are required to create a .NET assembly (shown in Figure 8) somewhat follow the builder software design pattern, with builder APIs for each
logical .NET metadata abstraction. The AssemblyBuilder class is used to create the PE file and set up the necessary .NET assembly metadata elements like the manifest. The
ModuleBuilder class is used to create modules within the assembly. TypeBuilder is used to create Types and their associated metadata. MethodBuilder and LocalBuilder deal
with adding methods to types and locals to methods, respectively. The ILGenerator class is used to generate the IL code for methods, utilizing the OpCodes class, which is a
big enumeration containing all possible IL instructions. All of these Reflection.Emit classes are used in the Good for Nothing code generator.

Figure 8 Reflection.Emit Libraries Used to Build a .NET Assembly

Figure 8 Reflection.Emit Libraries Used to Build a .NET

Assembly (Click the image for a larger view)

Tools for Getting Your IL Right

Even the most seasoned compiler hackers make mistakes at the code-generation level. The most common bug is bad IL code, which causes unbalance in the stack. The CLR
will typically throw an exception when bad IL is found (either when the assembly is loaded or when the IL is JITed, depending on the trust level of the assembly). Diagnosing
and repairing these errors is simple with an SDK tool called peverify.exe. It performs a verification of the IL, making sure the code is correct and safe to execute.

For example, here is some IL code that attempts to add the number 10 to the string "bad":

ldc.i4 10
ldstr "bad"
add

Running peverify over an assembly that contains this bad IL will result in the following error:

[IL]: Error: [C:\MSDNMagazine\Sample.exe : Sample::Main][offset 0x0000002][found ref 'System

.String'] Expected numeric type on the stack.

In this example, peverify reports that the add instruction expected two numeric types where it instead found an integer and a string.

ILASM (IL assembler) and ILDASM (IL disassembler) are two SDK tools you can use to compile text IL into .NET assemblies and decompile assemblies out to IL,
respectively. ILASM allows for quick and easy testing of IL instruction streams that will be the basis of the compiler output. You simply create the test IL code in a text editor
and feed it in to ILASM. Meanwhile, the ILDASM tool can quickly peek at the IL a compiler has generated for a particular code path. This includes the IL that commercial
compilers emit, such as the C# compiler. It offers a great way to see the IL code for statements that are similar between languages; in other words, the IL flow control code
generated for a C# for loop could be reused by other compilers that have similar constructs.

The Code Generator

The code generator for the Good for Nothing compiler relies heavily on the Reflection.Emit library to produce an executable .NET assembly. I will describe and analyze the
important parts of the class; the other bits are left for you to peruse at your leisure.

The CodeGen constructor, which is shown in Figure 9, sets up the Reflection.Emit infrastructure, which is required before I can start emitting code. I begin by defining the
assembly name and passing that to the assembly builder. In this example, I use the source file name as the assembly name. Next is the ModuleBuilder definitionfor one
module definition, this uses the same name as the assembly. I then define a TypeBuilder on the ModuleBuilder to hold the only type in the assembly. There are no types
defined as first class citizens of the Good for Nothing language definition, but at least one type is necessary to hold the method, which will run on startup. The MethodBuilder
defines a Main method to hold the IL that will be generated for the Good for Nothing code. I have to call SetEntryPoint on this MethodBuilder so it will run on startup when a
user runs the executable. And I create the global ILGenerator (il) from the MethodBuilder using the GetILGenerator method.

Once the Reflection.Emit infrastructure is set up, the code generator calls the GenStmt method, which is used to walk the AST. This generates the necessary IL code through
the global ILGenerator. Figure 10 shows a subset of the GenStmt method, which, at first invocation, starts with a Sequence node and proceeds to walk the AST switching on
the current AST node type.

The code for the DeclareVar (declaring a variable) AST node is as follows:

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 5 of 26

else if (stmt is DeclareVar)

{
// declare a local
DeclareVar declare = (DeclareVar)stmt;
this.symbolTable[declare.Ident] =
this.il.DeclareLocal(this.TypeOfExpr(declare.Expr));

// set the initial value

Assign assign = new Assign();
assign.Ident = declare.Ident;
assign.Expr = declare.Expr;
this.GenStmt(assign);
}

The first thing I need to accomplish here is to add the variable to a symbol table. The symbol table is a core compiler data structure that is used to associate a symbolic
identifier (in this case, the string-based variable name) with its type, location, and scope within a program. The Good for Nothing symbol table is simple, as all variable
declarations are local to the Main method. So I associate a symbol with a LocalBuilder using a simple Dictionary<string, LocalBuilder>.

After adding the symbol to the symbol table, I translate the DeclareVar AST node to an Assign node to assign the variable declaration expression to the variable. I use the
following code to generate Assignment statements:

else if (stmt is Assign)

{
Assign assign = (Assign)stmt;
this.GenExpr(assign.Expr, this.TypeOfExpr(assign.Expr));
this.Store(assign.Ident, this.TypeOfExpr(assign.Expr));
}

Doing this generates the IL code to load an expression onto the stack and then emits IL to store the expression in the appropriate LocalBuilder.

The GenExpr code shown in Figure 11 takes an Expr AST node and emits the IL required to load the expression onto the stack machine. StringLiteral and IntLiteral are
similar in that they both have direct IL instructions that load the respective strings and integers onto the stack: ldstr and ldc.i4.

Variable expressions simply load a method's local variable onto the stack by calling ldloc and passing in the respective LocalBuilder. The last section of code shown in Figure
11 deals with converting the expression type to the expected type (called type coercion). For instance, a type may need conversion in a call to the print method where an
integer needs to be converted to a string so that print can be successful.

Figure 12 demonstrates how variables are assigned expressions in the Store method. The name is looked up via the symbol table and the respective LocalBuilder is then
passed to the stloc IL instruction. This simply pops the current expression from the stack and assigns it to the local variable.

The code that is utilized to generate the IL for the Print AST node is interesting because it calls in to a BCL method. The expression is generated onto the stack, and the IL call
instruction is used to call the System.Console.WriteLine method. Reflection is used to obtain the WriteLine method handle that is needed to pass to the call instruction:

else if (stmt is Print)

{
this.GenExpr(((Print)stmt).Expr, typeof(string));
this.il.Emit(Emit.OpCodes.Call,
typeof(System.Console).GetMethod("WriteLine",
new Type[] { typeof(string) }));
}

When a call to a method is performed, method arguments are popped off the stack last on, first in. In other words, the first argument of the method is the top stack item, the
second argument the next item, and so on.

The most complex code here is the code that generates IL for my Good for Nothing for loops (see Figure 13). It is quite similar to how commercial compilers would generate
this kind of code. However, the best way to explain the for loop code is to look at the IL that is generated, which is shown in Figure 14.

The IL code starts with the initial for loop counter assignment and immediately jumps to the for loop test using the IL instruction br (branch). Labels like the ones listed to the
left of the IL code are used to let the runtime know where to branch to the next instruction. The test code checks to see if the variable x is less than 100 using the blt (branch if
less than) instruction. If this is true, the loop body is executed, the x variable is incremented, and the test is run again.

The for loop code in Figure 13 generates the code required to perform the assignment and increment operations on the counter variable. It also uses the MarkLabel method on
ILGenerator to generate labels that the branch instructions can branch to.

Wrapping Up ... Almost

I've walked through the code base to a simple .NET compiler and explored some of the underlying theory. This article is intended to provide you with a foundation to the
mysterious world of compiler construction. While you'll find valuable information online, there are some books you should also check out. I recommend picking up copies of:
Compiling for the .NET Common Language Runtime by John Gough (Prentice Hall, 2001), Inside Microsoft IL Assembler by Serge Lidin (Microsoft Press, 2002),
Programming Language Pragmatics by Michael L. Scott (Morgan Kaufmann, 2000), and Compilers: Principles, Techniques, and Tools by Alfred V. Oho, Monica S. Lam,
Ravi Sethi, and Jeffrey Ullman (Addison Wesley, 2006).

That pretty well covers the core of what you need to understand for writing your own language compiler; however, I'm not quite finished with this discussion. For the
seriously hardcore, I'd like to look at some advanced topics for taking your efforts even further.

Dynamic Method Invocation

Method calls are the cornerstone of any computer language, but there's a spectrum of calls you can make. Newer languages, such as Python, delay the binding of a method and
the invocation until the absolute last minutethis is called dynamic invocation. Popular dynamic languages, such as Ruby, JavaScript, Lua, and even Visual Basic, all share
this pattern. In order for a compiler to emit code to perform a method invocation, the compiler must treat the method name as a symbol, passing it to a runtime library that will
perform the binding and invocation operations as per the language semantics.

Suppose you turn off Option Strict in the Visual Basic 8.0 compiler. Method calls become late bound and the Visual Basic runtime will perform the binding and invocation at
run time.

Rather than the Visual Basic compiler emitting an IL call instruction to the Method1 method, it instead emits a call instruction to the Visual Basic runtime method called

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 6 of 26

CompilerServices.NewLateBinding.LateCall. In doing so, it passes in an object (obj) and the symbolic name of the method (Method1), along with any method arguments. The
Visual Basic LateCall method then looks up the Method1 method on the object using Reflection and, if it is found, performs a Reflection-based method invocation:

Option Strict Off

Dim obj
obj.Method1()

IL_0001: ldloc.0
IL_0003: ldstr "Method1"
...
IL_0012: call object CompilerServices.NewLateBinding::LateCall(object, ... , string, ...)

Using LCG to Perform Quick Late-Binding

Reflection-based method invocation can be notoriously slow (see my article "Reflection: Dodge Common Performance Pitfalls to Craft Speedy Applications" at
msdn.microsoft.com/msdnmag/issues/05/07/Reflection). Both the binding of the method and the method invocation are many orders of magnitude slower than a simple IL call
instruction. The .NET Framework 2.0 CLR includes a feature called Lightweight Code Generation (LCG), which can be used to dynamically create code on the fly to bridge
the call site to the method using the faster IL call instruction. This significantly speeds up method invocation. The lookup of the method on an object is still required, but once
it's found, a DynamicMethod bridge can be created and cached for each repeating call.

Figure 15 shows a very simple version of a late binder that performs dynamic code generation of a bridge method. It first looks up the cache and checks to see if the call site
has been seen before. If the call site is being run for the first time, it generates a DynamicMethod that returns an object and takes an object array as an argument. The object
array argument contains the instance object and the arguments that are to be used for the final call to the method. IL code is generated to unpack the object array on to the
stack, starting with the instance object and then the arguments. A call instruction is then emitted and the result of that call is returned to the callee.

The call to the LCG bridge method is performed through a delegate, which is very quick. I simply wrap the bridge method arguments up into an object array and then call.
The first time this happens, the JIT compiler compiles the dynamic method and performs the execution of the IL, which, in turn, calls the final method.

This code is essentially what an early bound static compiler would generate. It pushes an instance object on the stack, then the arguments, and then calls the method using the
call instruction. This is a slick way of delaying that semantic to the last possible minute, satisfying the late-bound calling semantics found in most dynamic languages.

The Dynamic Language Runtime

If you're serious about implementing a dynamic language on the CLR, then you must check out the Dynamic Language Runtime (DLR), which was announced by the CLR
team in late April. It encompasses the tools and libraries you need to build great performing dynamic languages that interoperate with both the .NET Framework and the
ecosystem of other .NET-compliant languages. The libraries provide everything that someone creating a dynamic language will want, including a high-performing
implementation for common dynamic language abstractions (lightning-fast late-bound method calls, type system interoperability, and so on), a dynamic type system, a shared
AST, support for Read Eval Print Loop (REPL), and more.

A deep look into the DLR is well beyond the scope of this article, but I do recommend you explore it on your own to see the services it provides to dynamic languages. More
information about the DLR can be found on Jim Hugunin's blog (blogs.msdn.com/hugunin).

Joel Pobar is a former Program Manager on the CLR team at Microsoft. He now hangs out at the Gold Coast in Australia, hacking away on compilers, languages, and other
fun stuff. Check out his latest .NET ramblings at callvirt.net/blog.

2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.
WinUnit

Simplified Unit Testing for Native C++ Applications

Maria Blees

This article discusses:

l Unit testing philosophy
This article uses the following technologies:
l Setting up WinUnit
l Building test fixtures
C++, Visual Studio
l Implementation and customization
Code download available at: WinUnit2008_02.exe (1438KB)

Contents
Start Testing Today
Getting Started with WinUnit
Fixtures: Setup and Teardown
Running WinUnit
Implementation Details
Going Further...

T
hese days it can be hard not to feel downright oppressed as a native code developerit seems like the developers using the Microsoft .NET Framework get all the cool
tools!

I've always been interested in good engineering practices, but I've been frustrated by so-called "engineering experts" who extol the virtues of unit testing, yet can offer little
more than hand waving when asked for tool recommendations for native code. What I really wanted, and what I thought would be the easiest to integrate into an automated
build system, was the equivalent of NUnit for native code. That is, I wanted to be able to make a DLL with only tests in it, and have an external test-runner that would run
those tests and take care of the reporting and logging. I also wanted to be able to declare each test only once, and to have a minimum of extra code in my test DLL.

So I built a native code unit testing tool I call WinUnit. I'll go into more detail later, but here's a little preview of how easy it is to create and run a test DLL using WinUnit. To

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 7 of 26

start, create a single CPP filelet's call it DummyTest.cpp:

#include "WinUnit.h"

BEGIN_TEST(DummyTest)
{
WIN_ASSERT_TRUE(3 > 4, _T("You should see this error message."));
}
END_TEST

Build it into a DLL with this command:

cl /EHsc /LD DummyTest.cpp

Then execute WinUnit on it:

>WinUnit DummyTest.dll
Processing [DummyTest.dll]...
(DummyTest)
DummyTest.cpp(5): error : WIN_ASSERT_TRUE failed: "3 > 4". You should see this error message.
FAILED: DummyTest.
[DummyTest.dll] FAILED. Tests run: 1; Failures: 1.

There were errors.

Tests run: 1; Failed: 1.

Changing 3 > 4 to the true expression 4 > 3 will of course get rid of the failure.

WinUnit will accept any number of DLLs or directories, process them, report the total results, and return with an exit code. By the way, if you haven't guessed, WinUnit only
works on Windows.

I concede that the world was not entirely devoid of options for C++ unit testing before I wrote my own tool. The problem I found with existing offerings was that they
generally placed a higher priority on portability than on being easy to understand and use out of the box. CppUnitLite is my favorite among the portable set, but because it's
designed to be very customizable, there was more overhead than I would have liked in getting it set up. The line between test-runner and tests is blurry due to the fact that it's
all done within the same binary, and the code that actually invokes the test-running functionality needs to be written and put somewhere. In addition, the tricky macro
implementation was hard to explain to my coworkers and thus made adoption difficult. However, if you require portability to other operating systems, CppUnitLite is
definitely worth taking a look at.

Other options that have been available for C++ unit testing for quite some time are the same options available for testing .NET code, most notably NUnit and the Visual
Studio Team Test unit testing framework. Since both will execute any properly attributed .NET assembly, you can use these tools with tests written in managed C++ (now
C++/CLI). The main inconvenience I find with this approach is that the tools are highly geared toward .NET development and not toward native code. This should not be a
surprise. But it is hard to use most of the provided assert methods with native C++ for they expect .NET-style objects, and as a developer you will need to have at least a
passing familiarity with C++/CLI (or Managed Extensions for C++) to write the tests. You'll find more details about limitations with this approach in the footnote on the
Visual Studio 2005 Product Feature Comparisons page (msdn2.microsoft.com/vstudio/aa700921.aspx). As mentioned previously, I find the "ease of convincing my coworkers
to use it" factor important, and using the managed tools provides an extra hurdle in this area.

Start Testing Today

Code Coverage and Convenience
Before getting into WinUnit usage, I want to discuss my philosophy about unit testing in general and unit
testing native code in particular. If you're like me, you may have noticed that the clean, elegant, and modular
classes shown in unit testing examples seem to have nothing in common with the blob of legacy code you I'd like to briefly address a feature of Visual Studio Team
typically have to work with. If you think that you must rework your entire code base before you can unit test, I System (VSTS) that deserves a mention for its relevance to
can tell you those tests are going to be a long time coming. The good news is that a testable unit doesn't have to unit testing: its set of code coverage tools. While the exact
be a clean, elegant class. It's simply the smallest unit of code that can be tested in isolation. If that's your whole significance of code coverage numbers is debatable,
application, well, there's no better time to start than now! reviewing your coverage after you've carefully constructed
thoughtful tests can be, if nothing else, an excellent sanity
check.
Seriously, if your application consists of exactly one executable and its only interface is through a GUI, you
may have to do a bit of refactoring to make the non-GUI part automatable. However, it can certainly be done
incrementally. I recommend reading the whitepaper "The Humble Dialog Box," by Michael Feathers, for a Unfortunately, VSTS does not make it easy to use (or even
sensible approach to making a GUI application more testable to know about) the code coverage tools for native code.
(objectmentor.com/resources/articles/TheHumbleDialogBox.pdf). Along with the code for this project, I've included a set of
macros that will make it easy for you to do code coverage
on native projectsin conjunction with WinUnit as well as
Your ultimate goal should be to have automated developer tests running as part of your build process, making
with any other scenarios you have for exercising your code.
it as easy as possible for developers to add tests as they fix bugs or write new code. Once the infrastructure is
set up, you can work on refactoring existing code to allow better test coverage in problem areas. The book
Working Effectively with Legacy Code, also by Michael Feathers (Prentice Hall, 2004), is an excellent To use these macros, simply load WinUnit.vsmacros into
resource for guiding you through this process (and many of the examples are in C++). Macro Explorer in Visual Studio. In the _Variables module
you'll need to set variables for your particular local machine,
which includes the path to the Visual Studio performance
If your application is already split up into static libraries with clearly defined interfaces, you can link these
tools (including the code coverage tools) and the paths to
directly into test DLLs and you'll be able to get at the individual classes to test them. Consider having one test
WinUnit.exe and WinUnit.h. You'll find more details in
DLL per library to maintain clear interface boundaries.
readme.txt as well as the _Readme module in the macro
project.
If your application contains separate DLLs, or is itself a library provided in DLL form, there are two ways you
can access the exports to exercise them in tests. If you use import libraries, you can link those into test DLLs
The CodeCoverage module includes macros for starting and
and have transparent access to the production functions for testing. If you do not use import libraries, use
stopping coverage data collection, instrumenting binaries
GetProcAddress in conjunction with LoadLibrary and FreeLibrary to get the DLL exports to call directly in
(you can specify a list of binaries to be instrumented for
your tests.
each project), and launching a .coverage file with the results.
Note that if your DLL is a COM DLL, it doesn't need to be registered to be tested. Windows XP and later offer
The RunningTests module includes macros for running the
a mechanism called "Registration-Free COM," which you could use to get around the usual registration
test containing the cursor in the currently opened document,
requirement for COM DLLs. You'll find details at msdn2.microsoft.com/ms973913. As a bonus, once you've
all tests in the currently opened file, and all tests in the
gotten it working for tests, you can use the feature in your production code.
selected project. Other macros provide shortcuts for
changing relevant project settings and for running different
If your application is a monolithic executable, you probably want to split it up so you have at least one library.
sets of tests with code coverage turned on.
But you might also want to test the executable in its entirety to check for proper exit codes, for example. In my
unit tests for WinUnit itself, I included some tests that exercised the entire executable. You can find these within the TestWinUnit projectin the MainTest.cpp filein the

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 8 of 26

accompanying code download.

At risk of coming across as the style police, there's one point about physically organizing production code that I would be remiss if I did not mention. In my years at Microsoft
working on large C++ projects, I've found that one thing that made modularity (and testability) exceedingly difficult was having multiple classes jumbled together across
arbitrary .cpp and .h files. It's up to you how strictly you adhere to this practice in your production code, but I find associating one and only one class with a similarly named
pair of .cpp and .h files to be helpful. As a bonus, it simplifies keeping your tests organized, with one matching test file for each class. You can see some examples in my
projects WinUnitLib and TestWinUnit.

There is one other book I recommend reading if the whole concept of unit testing has just passed you by, or if you're a developer in need of a refresher on how to think like a
tester: Pragmatic Unit Testing with C# in NUnit, by Andrew Hunt and David Thomas, (Pragmatic Bookshelf, 2006) is an excellent primer. (There's also a Java counterpart.)
The meat of the book is actually language-independent, despite the language-specific title. A cute mnemonic from the book is that unit tests should be A-TRIP: Automatic,
Thorough, Repeatable, Independent, and Professional. Repeatable means each test produces the same results any time it is run. Independent means tests should be able to be
run in any order, with no dependencies on each other. This point will be important to remember later.

Getting Started with WinUnit

The first thing you'll need to do is build WinUnit and place WinUnit.exe and WinUnit.h in known locations on your machine. You can refer to the readme.txt included with
the code for more information. Next, to get started writing tests, create a test project. As I demonstrated earlier, this is just a regular C++ DLL. For information on creating a
native C++ DLL project in Visual Studio, see the walkthrough at msdn2.microsoft.com/ms235636.

As with other unit testing frameworks, several assert macros are provided with WinUnit for your test-writing convenience, such as the WIN_ASSERT_TRUE you saw earlier.
(I'll discuss the other WIN_ASSERT macros you can use with WinUnit later in this section.) The asserts perform their magic using C++ exceptions, so you should keep the
Visual Studio 2005 default compiler option of /EHsc when building your test DLL. However, there's no requirement for your production code to use C++ exception handling,
even if it's linked into your test DLL. (Note that the Visual C++ 2005 toolset or later is required to use the provided assert macros.) You'll also want to add WinUnit's
Include directory to your test project's include path as well (see Figure 1), or to your global include paths.

Figure 1 Setting Project Include Path

Figure 1 Setting Project Include Path (Click the image for a

larger view)

I designed WinUnit to work well from the command line, which means you can set up Visual Studio to run WinUnit on your test project after every build. To do so, go to the
project Properties of your test project, and under Configuration Properties | Build Events, select Post-Build Event (see Figure 2). For Command Line, type the full path to
where you copied WinUnit.exe followed by "$(TargetPath)" (include the double quotes). For Description, type "Running WinUnit..." (without the quotes). If you add the
folder containing WinUnit.exe to your global executables path, you don't need to specify the full path here.

Figure 2 Configuring WinUnit as a Post-Build Event

Figure 2 Configuring WinUnit as a Post-Build Event (Click

the image for a larger view)

Alternatively (or in addition), you may wish to add WinUnit.exe to your Tools menu in order to run your tests. To accomplish this, go to Tools | External Tools and click Add.
For Title, type "&WinUnit". For Command, browse to WinUnit.exe. For Arguments, type "$(TargetPath)" (include the double quotes). For Initial Directory, type
$(TargetDir). Uncheck Close on Exit and check Use Output Window (see Figure 3). At this point in the process you can select WinUnit from the Tools menu, and it will
execute on whichever project is currently selected.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 9 of 26

Figure 3 Adding WinUnit to External Tools

Figure 3 Adding WinUnit to External Tools

Finally, to set up your project for debugging, go to project Properties | Configuration Properties | Debugging, and type the full path to WinUnit.exe in the Command textbox.
For Command Arguments, type "$(TargetPath)" (include the double quotes).

To verify everything in your project is set up correctly, you can add DummyTest.cpp to your project and build. The project should build, but if you've typed in the false
assertion line as presented, you'll clearly see a test failure on that line.

Now that the setup is out of the way, it's time to look at the different ways of using WinUnit features to write tests. You may want to switch to WinUnitComplete.sln for the
moment to follow along with my examples. The projects containing the examples are SampleLib and TestSampleLib. SampleLib is a static library that happens to contain
exactly one class, BinaryNumber. TestSampleLib is a test DLL that links with SampleLib.lib and includes tests for the BinaryNumber class along with a few other examples.

General WinUnit test functions always begin with BEGIN_TEST(TestName) and end with END_TEST. Within each test, one or more WIN_ASSERT macros are used in
order to verify various bits of functionality.

My example BinaryNumber class has two constructors. One takes an unsigned short, and one takes a string comprising the characters 1 and 0. I intend for the constructors to
result in equivalent objects when passed equivalent values. So I may have a test like this:

BEGIN_TEST(BinaryNumberConstructorsShouldBeEquivalent)
{
unsigned short numericValue = 7;
BinaryNumber bn1(numericValue);
BinaryNumber bn2("111");
WIN_ASSERT_EQUAL(bn1.NumericValue, bn2.NumericValue,
_T("Both values should be %u."), numericValue);
WIN_ASSERT_STRING_EQUAL(bn1.StringValue, bn2.StringValue);
}
END_TEST

This test will fail if either of the two assert lines is false. Notice that the WIN_ASSERT_EQUAL macro in this example is passed the two values being compared, plus two
extra arguments. These comprise an informational message that will be shown if the assert fails. All WIN_ASSERT macros take an optional printf-style format string and
arguments for this purpose.

Since I implemented operator '==' for the BinaryNumber class, I can also use the following construction (which is what I have in my sample file, BinaryNumberTest.cpp):

BEGIN_TEST(
BinaryNumberConstructorsShouldBeEquivalent)
{
BinaryNumber bn1(7);
BinaryNumber bn2("111");
WIN_ASSERT_EQUAL(bn1, bn2);
}
END_TEST

Here's a different type of assert, WIN_ASSERT_THROWS. Suppose I use exceptions for my error handling in my production code, and I want to force an error condition in
my test to ensure that the proper exception is thrown. I might make a test like this:

BEGIN_TEST(
BinaryNumberPlusRecognizesIntegerOverflow)
{
unsigned short s = USHRT_MAX / 2 + 1;

BinaryNumber bn1(s);
BinaryNumber bn2(s);

WIN_ASSERT_THROWS(bn1 + bn2,
BinaryNumber::IntegerOverflowException)
}
END_TEST

I know that my BinaryNumber class only holds an unsigned short. I also know that the operator '+' should detect when I'm trying to add two numbers together that are too big.
WIN_ASSERT_THROWS takes an expression that should throw a C++ exception, along with the exception type itself. The test fails if the exception is not thrown.

Figure 4 lists the WIN_ASSERT macros available in WinUnit.h, as well as one non-assert, WIN_TRACE, that can be used to provide tracing through the OutputDebugString
API function in your tests.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 10 of 26

Note that for either WIN_ ASSERT_EQUAL or WIN_ASSERT_NOT_EQUAL, if a numeric literal is passed in as one of the values and an unsigned number as the other,
you'll get a signed/unsigned mismatch, as numeric literal integers always template-match to int. To get around this, postfix numeric literals with 'U' so they will match as
unsigned.

All asserts take an optional printf-style format string, plus arguments. If _UNICODE is defined, these message strings will be wchar_t*, so use the _T ("") macro around the
format strings, or L"" if you're building Unicode-only.

You may have noticed that my test names begin with the name of the class being tested, followed by the name or description of the method being tested, followed by a partial
sentence describing what the test is supposed to show. This convention serves two purposes. First, it is an easy way to make clear exactly what you're trying to test, which can
be useful when looking at test output. Second, it's a way to group tests to be run together at varying levels of granularity. WinUnit does not inherently have the concept of
groups of tests, as do its .NET Framework-based equivalents. The way to tell WinUnit to run a subset of tests in a project is with the -p (prefix) option. You specify a prefix
string, and WinUnit will run all the tests whose names start with that string. By naming your tests with words ordered least to most specific, you will be able to easily run
related groups of tests from the command line.

Fixtures: Setup and Teardown

In real life, the things you're trying to test are often not as simple as the examples I've shown so far. There may be cases where several setup steps are required to get to the
point where you can actually execute the functionality you want to verify. For example, if I were testing a function that deletes all the files in a directory, I might first need to
create a directory and put files in it. Or I might want to set an environment variable required for the functionality I'm testing.

To maintain the independence and repeatability of tests, it's important that whatever setup work you do at the beginning of a test is undone at the end of the test. WinUnit
supports the concepts of single-test fixtures. These are setup and teardown function pairs that will be executed at the start and finish of each test associated with them. This is
especially helpful if you have several tests that require the same setup and cleanup.

Figure 5 shows an example of a fixture, with setup and teardown functions implemented and two tests associated with it. Note that this example also shows that you can also
use the various WIN_ASSERT macros even in fixtures so you can report failure if the fixture you define does not work correctly.

In this example, I'm pretending I was the writer of the Windows DeleteFile function and I'm writing some test functions for it. In my setup function, I first create a temporary
file. In my test functions, I exercise the DeleteFile functionality by trying to delete the temporary file. Although I expect the file to have been deleted by the end of one of the
tests, I still check whether the file exists and delete it in the teardown function. The teardown should undo whatever the setup did and should not count on the tests succeeding.
This ensures that your tests are independent and repeatable.

As you can see, there's a slightly different syntax for tests that use fixtures. Instead of the BEGIN_TEST and END_TEST construction, BEGIN_TESTF and END_TESTF are
used, with the fixture name following the name of the test in BEGIN_TESTF. The setup and teardown functions are indicated by the all-caps SETUP and TEARDOWN, with
the name of the fixture in parentheses.

You'll notice I've used the WIN_ASSERT_WINAPI_SUCCESS macro a few times in this example. As indicated in Figure 4, this macro is to be used in conjunction with
Windows functions whose documentation explicitly says to call GetLastError for more information in the case of failure. The first parameter to the assert should be an
expected true statement related to the function call. (It can be the function call itself, if it returns a Boolean.) If the function itself is not part of the expression, it's a good idea
to include a message indicating the name of the function that has failed if this assert fires, to make it easier to see at a glance from the output exactly what happened. The
assert implementation calls GetLastError and retrieves the message string associated with the error code and adds that to the test results.

To see a fixture used in a different way, see BinaryNumberTest.cpp in the TestSampleLib project. There I used it to open and close a data provider from which I read rows of
test data to be used in my tests. In this case, the data provider is just a text file, and each row is a line, but it could just as easily be an XML file or even a database table. You
should consider using a similar approach if you are testing functionality that can benefit from large amounts of data being run through it.

One difference between WinUnit fixtures and those of other unit test frameworks is that they are never automatically associated with teststhe association must be explicit.
This also means that it is possible to have more than one fixture in a single file.

Running WinUnit

Now let's take a look at how WinUnit works. WinUnit uses one of the only reflection-like features available for native code on Windows: discovering the exports of a DLL.
There was a great two-part article by Matt Pietrek in MSDN Magazine in February and March 2002 ("Inside Windows: An In-Depth Look into the Win32 Portable
Executable File Format"), which I referred to extensively in figuring out how to do this. It works for both 32-bit and 64-bit executables, but be sure to use a 64-bit build of
WinUnit to run 64-bit test binaries (32-bit executables can't run 64-bit ones).

The command line syntax for WinUnit is:

WinUnit [options] {dllName | directoryName}+

The tool takes one or more DLLs or directories over which all contained DLLs will be enumerated. It discovers the exports and executes the ones whose namewhich must
be undecoratedstarts with TEST_. This restriction is intended to ensure that only functions you meant to run as tests will be run, since the function prototype is assumed.
The expected function prototype is as follows:

bool __cdecl TEST_TestName(wchar_t* errorBuffer, size_t bufferSize);

The errorBuffer parameter receives error output from the test; bufferSize is the size of the buffer in wide characters. If the test function returns false (or throws a structured
exception handling (SEH) exception), it's considered a failed test, and whatever output was put in the errorBuffer is displayed.

Besides DLLs and directory names, WinUnit takes several optional command-line arguments, which are shown in Figure 6. By default the output of WinUnit goes to the
console. Informational messages go to stdout; error messages go to stderr. After WinUnit has processed the specified DLLs, it exits with a code of 0 for success and non-zero
otherwise (see Figure 7).

The BEGIN_TEST macros used in the examples prepend TEST_ to the test names and use 'extern "C" __declspec(dllexport)' to export the function names undecorated. This
is the equivalent of putting them in a .def file for the DLL.

The WIN_ASSERT macros work via C++ exceptions; however, all exceptions are caught within the test functions themselves, so WinUnit doesn't care whether exceptions
were used or not. I used exceptions because they make it easy to exit out of any block at any point and still ensure that proper cleanup occurs. You could implement your own
test functions that would work just as well with WinUnit entirely without macros by using the function prototype shown earlier, ensuring that the names started with TEST_,
and exporting them in undecorated form from a DLL.

The BEGIN_TEST and END_TEST macros declare the function and set up a try/catch block. The WIN_ASSERT macros throw an AssertException class exception on

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 11 of 26

failure, which holds a message string describing the error. If a thrown AssertException is caught, the message is copied into the buffer and the function returns false.

Implementation Details

I initially thought it would be elegant to have no parameters to the test functions, and instead just throw exceptions directly out of the functions and catch and process them in
the test runner. This would have meant there was no need for an END_TEST macro. Unfortunately, stack unwinding did not work properly when exceptions were thrown
across the DLL boundary, so it ended up working out better to have the exceptions as an implementation detail and catch them within the tests themselves.

Implementing the fixture concept was another challenge. My first idea was to rely on C++ automatic storage and declare an object at the top of the test function whose
constructor and destructor would do setup and teardown. However, I soon discovered firsthand why you're not supposed to throw exceptions in destructors. Part of the C++
specification states that when an exception is thrown by a destructor function during stack unwind, the terminate function is called. This state could occur if I implemented
fixtures as described and had a fixture object with exception-throwing asserts in its destructor. If the asserts in the fixture object's destructor fired during the stack unwind due
to an exception thrown in the body of the function, terminate would be called.

The remedy for this was to ensure that the fixture object's destructor would never be called during a stack unwind by putting it alone in its own try/catch block outside the
main try/catch for the function. This necessitated the special fixture syntax using the BEGIN_TESTF and END_TESTF pair to house two try/catch blocks and a fixture object
declaration.

Running across the unpleasant terminate behavior made me think of something else I also wanted to make sure to handle: I wanted to provide the option of running in non-
interactive mode (currently the n command line option). WinUnit generally runs non-interactively, but I noticed that when the terminate function was called, it put up a
dialog box. I thought dialog boxes like this would be inconvenient in automation scenarios because they would potentially block further automated tasks.

You can see what I did to disable such error messages in ErrorHandler.cpp in the WinUnit project. Interestingly, some of the dialogs that can come up in error conditions are
associated with the C Runtime (CRT). This means that setting a global variable in the CRT to turn them off isn't going to do any good if the variable is set in the test-running
tool (WinUnit) and the test binary is using a different CRT. If you want to ensure that CRT-specific error dialogs will be turned off, you will need to be sure you are using the
same Runtime Library option for building both WinUnit and your test binaries, and that it's one of the DLL options. This option can be found in project Properties |
Configuration Properties | C/C++ | Code Generation. By default, it is /MD for release builds and /MDd for debug builds. If you build both WinUnit and your test binaries in
Visual Studio 2005 with the same option here, you will have the same CRT instance.

I wanted to have some trace functionality to aid in debugging tests. I decided to take advantage of the fact that my tool is meant to run on Windows and that I know the PE
format. I therefore hook (override) OutputDebugStringA and OutputDebugStringW in the test DLL being executed, sending the output wherever I want. I borrowed this part
heavily from the code that comes with the book Debugging Applications for Microsoft .NET and Microsoft Windows (Microsoft Press, 2003), by John Robbins, who kindly
gave me permission to use it in this project. I then provided a WIN_TRACE macro which funnels its arguments to OutputDebugString.

I also wanted to have some way to pass information from the command line into the tests. I thought this might be useful if, for example, you were using the tool as part of an
automated build, and there was some known location that contained test data files, and you wanted the tests to know about that location. I decided on environment variables.
Arbitrary variables can be set for the process using the "--" command-line option; they can be retrieved within the tests using WinUnit::Environment::GetString (a wrapper on
GetEnvironmentVariable), GetEnvironmentVariable itself, or getenv_s/_wgetenv_s.

Finally, I wanted to offer the ability for a user to write a custom logger. Internally, a logger chain is used, so you can opt to send output to console, to OutputDebugString, to a
file, or to any combination of those. A fourth option is to provide your own custom logger, which I'll talk about next.

Going Further...

WinUnit should solve your basic native C++ unit testing needs, but there are a few things you may want to consider adding to it.

You may have noticed the l command-line option in Figure 4 for passing in a custom logger. You might want to use this feature to send WinUnit output to an XML file or
even to a database for use in a reporting system. Also, the default loggers do not write Unicode; the o option creates an ANSI text file. If you want your output in Unicode,
you will need to write a custom logger for that. To implement a custom logger feature, simply create a DLL that implements one or more of a set of ten logger-related
functions, and then pass the path to the DLL to WinUnit via the l option. You can see my project SampleLogger.cpp for more details.

You might also wish to add your own custom asserts, using the existing ones as examples. You could make an additional header file or a static library to be linked into your
test DLLs. It might be interesting to implement a more in-depth set of string asserts, similar to the StringAssert class that comes with the Visual Studio unit testing framework
for .NET-based code, which includes string comparisons with regular expressions. A FileAssert class could be useful as wellcomparing entire files or attributes of files.
You'll notice that I used an Assert class with static methods for the implementation of the asserts, and the WIN_ASSERT macros are thin wrappers around them. I would have
liked not to have used macros at all, but I wanted to be able to easily include the file and line number. Of course, macros pollute the global namespace, so if you're going to
make your own it would be a good idea to start them with a different prefix.

Another handy Visual Studio add-in would be one that allowed you to run tests individually (with or without code coverage) by right-clicking on them, similar to how
TestDriven.NET works for unit tests in the .NET Framework. See the sidebar "Code Coverage and Convenience" for a short discussion of code coverage tools.

My goal has been to show that unit testing native C++ can be easy, fun, and you can get started right now. To give you an idea of the power of WinUnit, take a look through
the TestWinUnit project, where I used WinUnit to test itself. Those examples are completely real world and will show you advanced usage you can apply to your own unit
tests. If you've been struggling with your native C++ unit testing, WinUnit makes it easyand any time you can make testing easy, you're far more likely to actually do it.

Maria Blees has been a developer at Microsoft for 10 years and has a special fondness for native code and engineering excellence. She can be reached at the address listed in
the code and welcomes bug reports and suggestions for this tool.

2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.
Silverlight

Create a Custom Expression Encoder Publishing Plug-In

Laurence Moroney

This article discusses:

l Expression Encoder plug-in basics
l Building a boilerplate plug-in This article uses the following technologies:

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 12 of 26

l Adding FTP upload functionality

l Building, installing, and debugging WPF, Visual Studio, Expression Encoder

Contents
Expression Encoder and Plug-Ins
Creating a Plug-In
Building the FTP Upload Plug-In
Implementing the Publish Class
Saving Configuration Settings
Compile, Deploy, and Debug

A
recently released tool, Microsoft Expression Encoder helps you encode, enhance, and publish rich media experiences that can be rendered using Microsoft SilverlightTM.
Expression Encoder has an extensible plug-in architecture, giving you the ability to write plug-ins that allow for the publication of Expression Encoder output to different
sources.

In this article, I will step through the building of a plug-in for Expression Encoder using Microsoft Visual Studio 2008 Beta 2. This plug-in will fit within the standard
workflow of an Expression Encoder session, and, at the end of an encoding session, it will upload the resulting output to an FTP server.

Expression Encoder and Plug-Ins

Figure 1 shows Expression Encoder. The left side of the workspace contains the video preview and a control bar that allows you to set the portion of the video to re-encode.
On the right-hand side of the screen, there are three tabs: Settings, for configuring up the encoding session; Metadata, for applying new information to the media; and Output,
for taking post-encode actions.

Figure 1 Expression Encoder

Figure 1 Expression Encoder (Click the image for a larger

view)

When a publishing plug-in is present in <Installation Directory>\Plugins and is loaded correctly, the Output tab will contain a Publish pane that lists the available publishing
options. Each type requires a separate plug-in. Figure 2 shows that the FTP plug-in is available.

Figure 2 Selecting the FTP Plug-In

Figure 2 Selecting the FTP Plug-In

The top part of the Publish pane is fixed for all plug-ins. The Publish To list is populated from the names of the valid plug-ins detected in the plug-ins directory. The Auto
Start Publish checkbox is always available and, if checked, will kick off the publication automatically at the end of an encoding.

The Publish button is grayed out until encoding is complete, at which point it becomes available. By clicking the button, you can then publish all the files associated with the
encoding at once. (This could be a single file if no template is used, or all files needed for the video player if a template is used.) The Settings section lets the user specify the
details necessary for publishing the output. In the case of an FTP output, the server address, user name, and password are necessary. Plug-ins also provide the facility for an
advanced settings area that is not initially visible but can be expanded using the pane expander button.

The Settings pane is free-format, so there's a lot of flexibility available to implement whatever your plug-in needs using XAML and Windows Presentation Foundation
(WPF).

Creating a Plug-In

The first step in constructing a plug-in is to create a new Microsoft .NET Framework 3.0 class library. Then you add the required references that allow you to compile and

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 13 of 26

execute WPF and encoder applications. These are listed in Figure 3.

A plug-in is simply a standard .NET class that derives from the PublishPlugin base class and is decorated with the EncoderPluginAttribute attribute, which adds the metadata
that describes your plug-in. Its constructor takes two strings, the plug-in name and a lengthier description.

Here's the required code:

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Expression.Encoder.Plugins.
Publishing;

namespace TestPlugin
{
[EncoderPlugin("Boiler Plate",
"Boiler Plate App")]
public class Class1 : PublishPlugin
{
}
}

To successfully compile the plug-in, you need to implement the three member functions of the PublishPlugin class. CreateStandardSettingsEditor creates a standard settings
dialog and returns an object that describes the UI for it. CreateAdvancedSettingsEditor creates the advanced settings dialog and returns an object that describes the UI for the
dialog. PerformPublish is called when the Publish button is pressed. It takes two parameters: root and filesToPublish. Root contains the directory of the files to be published.
filesToPublish is an array of paths to the files that Expression Encoder will publish.

Figure 4 shows an example of a boiler plate class that implements these three functions. This class will compile into a DLL. If you copy this DLL to the Plugins directory, it
will thereafter be available as a publish plug-in.

The attribute's constructor arguments define the list entry in the Publish To dropdown list, and the objects returned by the CreateStandardSettingsEditor and
CreateAdvancedSettingsEditor functions define the UI for the Settings panes.

Building the FTP Upload Plug-In

Let's start with the simple boilerplate plug-in (which doesn't actually do anything) and expand it into a functional FTP upload plug-in. The standard and advanced parameters
panes are implemented in WPF XAML, but they require their data to be implemented in a class that inherits from INotifyPropertyChanged in order for the plug-in to detect
changes the user makes to the settings. A single class (in this case PublishData) is sufficient for both parameter panes.

Five properties are implemented for the FTP upload plug-in. UseProxy determines whether system proxy should be used or bypassed. ServerAddress provides the address of
the FTP server to upload to. UserName is the login ID for the server, and UserPassword is, of course, the associated password for the login ID. Finally, ServerDirectory is the
directory on the server that will receive your uploaded files. Figure 5 shows the complete class that implements these properties.

The structure of this class is straightforward. First, the class has to implement INotifyPropertyChanged to receive property notifications from the UI. It then implements a
standard property handler design pattern whereby a private member variable stores the value of the property and is encapsulated by a getter and a setter accessor method.
When the getter is called, the value of the private member variable is returned. When the setter is called, the private member is updated and the PropertyChanged event is
raised. The PropertyChanged event is initialized with an argument containing the name of the property to be changed. The UI uses XAML data binding to hook into this code,
causing user actions to call the getter and setter. An example of the XAML that is used in the dialog for the Server Address property is shown here:

<TextBox Grid.Column="1" Grid.Row="0" Text="{Binding Path=ServerAddress, Mode=TwoWay}" Margin="0,3,0,3" />

The parameter pane's UI controls are implemented using standard WPF XAML user controls. Note that the x:Class for these controls must be set to use the namespace you
have been using thus far in your code; for example, they should be FTPPublishPlugin.StandardParams and FTPPublishPlugin.AdvancedParams for the standard and advanced
panes, respectively.

These are databound to their associated properties within the PublishData class. This is achieved by setting the PublishData class to be the data context for the user control.
When the plug-in instantiates the StandardParams user control, it passes an instance of the PublishData class to the control. The user control then uses this instance to set up
the data context, as you see here:

public StandardParams(PublishData data)

{
this.InitializeComponent();
this.DataContext = data;

Thus, to implement the label and textbox for the Server Address, you simply use the following XAML in the user control:

<Label Grid.Column="0" Grid.Row="0" Margin="0,3,0,3">Server Address

</Label>
<TextBox Grid.Column="1" Grid.Row="0" Text=
"{Binding Path=ServerAddress, Mode=TwoWay}" Margin="0,3,0,3" />

As you can see, the TextBox's text property is bound to the ServerAddress property; the binding mode is two-way, allowing the property to be set and get automatically by
WPF.

Handling a password entry field is a little more difficult, as you cannot databind to it for security reasons. In addition, the automatic styling of labels and textboxes that the
encoder provides is not available to the password box, so you have to set the styling yourself. Thus, in order to implement a password box, you use the XAML shown in
Figure 6 to implement the UI.

This specifies the event handler for the password change to be HandlePW. You use this event handler to determine the typed password and to store it in the PublishData class
that is the current data context. Here's an example of this function:

public void HandlePW(object sender, RoutedEventArgs e)

{
PublishData d = (PublishData) this.DataContext;
d.UserPassword = pass.Password;
}

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 14 of 26

Implementing the Publish Class

Now that the foundations are laid, creating the publish class for an FTP server is straightforward. The first thing you'll need to do is to ensure that an instance of the
PublishData class is created with the plug-in. The natural place to do this is in the constructor for the plug-in:

private StandardParams _standardParams;

private PublishData _publishData;
public Publish() : this(new PublishData())
{
}
public Publish(PublishData publishData)
{
_publishData = publishData;
_standardParams = new StandardParams(_publishData);
}

As you can see, this also initializes the StandardParams user control, so it will display when the plug-in is rendered. It passes the instance of the PublishData class (called
_publishData) to the user control; this will be used to set the data context, as outlined in the previous step.

The next thing you do is to implement the required functions for creating the standard- and advanced-settings editor panes. If you recall the boilerplate example, these
returned strings that were then rendered. Now you want to render the user controls for the respective settings panes:

public override object CreateStandardSettingsEditor()

{
return _standardParams;
}
public override object CreateAdvancedSettingsEditor()
{
return new AdvancedParams(this, _publishData);
}

The final step is to implement the Publish function, which gets called when the user requests publication of the output, either by pressing the Publish button or by selecting the
Auto Publish option. In this case, you are going to publish to FTP, so the code for connecting to the server is implemented here. Figure 7 shows an example.

As you can see, this is fairly straightforward C# code for managing the upload of a set of files to an FTP server. The required data for the FTP server is taken from the
_publishData class, which in turn is set by the controls discussed earlier.

One function of note is OnProgress, which is part of the Plugins API and is not necessary to implement. You simply call this function with a string and a number from 0 to
100. It then renders the string and a bar representing the value as a percentage of progress. In this case, the progress is determined by the number of files uploaded, so if, for
example, there are 10 files, each is considered 10 percent of the progress regardless of the file size.

Also take note of the CancelPublish function. The API provides this function for you to override. It will be fired when the user clicks the Cancel button that is automatically
generated by the UI when you are publishing. In this case, it simply sets the m_Cancelled Boolean to true and cancels publication.

Note that as this code is just a sample application, it has a single exception handler that captures all exceptions and throws them up to the user. In a real application, you
should provide more detailed error information for the various exceptions your application may encounter.

Saving Configuration Settings

In order to keep the user from constantly having to enter credentials for the FTP server being published to, you can override the SaveJobSettings and LoadJobSettings
methods from the PublishPlugin base class. These methods will be called on the plug-in instance by Expression Encoder when the user calls Save Job from the file menu, and
they allow you to serialize and restore your configuration properties. For example, to save your configuration properties, you would write the following:

public override void SaveJobSettings(XmlWriter writer)

{
writer.WriteStartElement("FtpSample");
writer.WriteElementString("UseProxy",
_publishData.UseProxy.ToString());
writer.WriteElementString("ServerAddress",
_publishData.ServerAddress);
writer.WriteElementString("UserName", _publishData.UserName);
writer.WriteElementString("ServerDirectory",
_publishData.ServerDirectory);
writer.WriteEndElement();
}

One thing to note here is that you must nest your configuration properties within a top level XML Element. This is because your properties are serialized into your encoding
job's XML job file, which contains lots of additional data. You can restore your saved properties with the following:

public override void LoadJobSettings(XmlReader reader)

{
reader.ReadStartElement("FtpSample");
_publishData.UseProxy = bool.Parse(xr.ReadElementString("UseProxy"));
_publishData.ServerAddress = xr.ReadElementString("ServerAddress");
_publishData.UserName = xr.ReadElementString("UserName");
_publishData.ServerDirectory = xr.ReadElementString("ServerDirectory");
reader.ReadEndElement();
}

Compile, Deploy, and Debug

Should everything compile happily, you will have three files in your projects bin\debug directory: your plug-in DLL, a .pdb file for your DLL, and EncoderTypes.dll. Copy
only the plug-in DLL file to the Plugins folder. Now, when you launch Expression Encoder, the plug-in should be loaded correctly.

After loading Expression Encoder, return to Visual Studio and select Debug | Attach to Process. The Attach to Process dialog will appear, and from there select
MediaEncoderUI.exe.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 15 of 26

If you want to inspect the publication process, you can set a breakpoint in the Publish function. Then, after completing an encoding job, if you press the Publish button in
Expression Encoder, you will hit your breakpoint and will be able to step through and see what is happening as Expression Encoder publishes your files.

Laurence Moroney is a Senior Technology Evangelist with Microsoft, specializing in Silverlight. He is the author of many books on computing topics, including Silverlight,
AJAX, interoperability, and security. His blog is at blogs.msdn.com/webnext.

2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.
PIAB And WCF

Integrating the Policy Injection Application Block with WCF Services

Hugh Ang and David San Filippo
Code download available at: WCFandPIAB2008_02.exe (196KB)

This article discusses:

l Integrating the Policy Injection Application Block and WCF
l Creating Custom WCF Behaviors This article uses the following technologies:
l Applying behaviors using .NET attributes or configuration WCF, PIAB
l Integration without changing any code

Contents
A Real-World Example
WCF Architecture Overview
Dispatcher
Extending IEndpointBehavior and IContractBehavior
The Configuration Approach
The Attribute Approach
The Sample Code

O
ne of the most important software design principles is the separation of different responsibilities within our applications. In service-oriented design, we separate applications
into services and operations. In real life, however, implementation concerns tend to leak into the services.

This is a problem not only in service-oriented design, but in object-oriented design as well. Enterprise Library 3.0 introduced the Policy Injection Application Block (PIAB) to
solve this problem in object-oriented design. At the time of this writing, however, the latest version of Enterprise Library (3.1) doesn't directly support the integration of PIAB
with Windows Communication Foundation (WCF) services, so using PIAB to separate concerns in service-oriented applications might seem daunting. But, as you'll see, it's
easy enough with the right techniques. Let's get started.

To leverage the PIAB, you must have control over how your objects are instantiated. With service frameworks like WCF that abstract object instantiation away from the
developer, this creates a problem when trying to integrate the PIAB. However, WCF provides a variety of extensibility points through behaviors. Here we will show you how
to leverage custom WCF behaviors in order to apply PIAB at the WCF service point without requiring additional code. With WCF service successfully integrated, PIAB can
truly become a ubiquitous framework for separating service logic from cross-cutting implementation details.

A Real-World Example

To better understand this leakage problem, imagine you were building a simple service to find a custom customer object using a customer ID. You write a simple method in
Visual Studio 2005 to invoke a data access object and return a customer object:

public Customer GetCustomer(string customerID)

{
return CustomerDAO.GetCustomer(customerID);
}

Then you remember the customer ID field cannot be empty. So you add a few lines for validation. You also add a few tracing statements so you can make sure the correct
values are being passed to and returned from the data access object. Finally you add exception handling to wrap any exceptions thrown from the data access object with a
custom exception. Next thing you know, you have 15 lines of code. And it could have been worse. Authorization logic, caching logic, or logic to update performance counters
would have added even more code bloat.

Yet none of these additions has anything to do with retrieving a customer object. Ultimately, what you get for all this work is code that is more difficult to understand,
maintain, and reuse. Unfortunately, examples like this are common in the real world, especially when business logic spans more than one line of code.

Using the PIAB, you would only need a single line of code in your method. The code for validation, exception handling, and so forth would be introduced through the
creation of policies that define the implementation details needed at run time. Policies can be applied to objects through configuration or by decorating classes with attributes.
Each policy has a set of matching rules to determine where it should be applied, and a collection of handlers that determine which behaviors should be introduced when
invoking methods on the object. The PIAB comes with a number of useful handlers for validation, logging, exception handling, managing performance counters,
authorization, and caching. Many of these handlers leverage Enterprise Library's other application blocks.

In order to consume an object that uses these policies, you must create an instance of the class using the PolicyInjectorFactory. Instead of instantiating the class using the
"new" operator, you would pass the type to the Factory's "Create" method. The factory will then determine which policies apply to that type using the matching rules. PIAB
supports matching rules based on types, assemblies, member names, method signatures, namespaces, parameter types, property types, return types, custom attributes, tags
(PIAB gives you a tag attribute that can be set to a value for matching), and your own custom matching rules.

If no matching rules apply, the factory will simply instantiate the type and return it; otherwise it creates a proxy to that object. The proxy intercepts calls to the object and
invokes the handlers in a pipeline, as shown in Figure 1.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 16 of 26

Figure 1 Handlers Pipeline

Figure 1 Handlers Pipeline (Click the image for a larger

view)

The PIAB is exactly what we need to keep the GetCustomer service method to a single line of code. We just need to define a policy to match the "GetCustomer" operation
and include handlers for logging, validation, and exception handling.

Now that you understand PIAB policy, matching rules, and handlers, you need to tackle the problem of integrating this solution into WCF. Let's see how you could still
leverage PIAB if the GetCustomer operation were defined in a WCF service.

WCF Architecture Overview

WCF is the Microsoft platform for building services that support a variety of transports including TCP/IP, HTTP, MSMQ, named pipes, and WS-* protocols. At a high level,
the WCF architecture consists of a service model layer and a channel layer at both the sender and receiver, as shown in Figure 2. The service model layer provides the API for
user code to interface with WCF; the channel layer handles the messaging and transport details.

Figure 2 WCF Architecture

Figure 2 WCF Architecture (Click the image for a larger

view)

When you host a WCF service, you have to define the address, binding, and contractthe ABCsthat define the service's endpoint. These ABCs can be defined in either
code or the configuration file, and they effectively control the channel layer. WCF takes care of building up the channels needed for the binding and protocol, listens for
incoming messages, and sends them to the intended service instances through invocation of its methods.

As discussed previously, to enable PIAB for a type, its instance needs to be created via the PIAB factory. When hosting a WCF service, you only provide the type of the
service class. This implies that the WCF runtime will be responsible for creating an instance of the service when it is needed. So how can we go about altering this behavior so
that a service class instance can be created by the PIAB factory within the WCF runtime? In order to do so, we need to delve deeper into the WCF runtime architecture and
find out how WCF instantiates the service.

Dispatcher

Take a look at the receiving application in Figure 2. The diagram shows a simplified view of the physical WCF dispatcher model, as there actually exist channel dispatchers
and endpoint dispatchers. Ignoring that detail doesn't affect what we're trying to accomplish, and it makes our discussion a lot easier. Note that the dispatcher sits within
WCF's service model layer. The dispatcher plays an essential role for WCF, as it is responsible for deserializing the WCF message, creating the service instance, and
eventually dispatching the call. Upon creating the dispatcher object, the WCF runtime initializes an instance behavior object that contains an instance provider object. The
instance provider is responsible for instantiating the service. This process is shown in Figure 3.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 17 of 26

Figure 3 Service Instantiation

Figure 3 Service Instantiation (Click the image for a larger

view)

Behaviors are the main extensibility point for WCF. By creating our own behaviors, we can alter the way WCF creates the instance of the target service, thereby putting the
PIAB mechanism in place. Figure 4 shows how the PIAB is introduced when we create our own custom behavior and custom instance provider.

Figure 4 Introducing the PIAB

Figure 4 Introducing the PIAB (Click the image for a larger

view)

Now, let's create a custom behavior and instance provider and plug them into the WCF runtimewe will demonstrate this with both configuration data and Microsoft .NET
Framework attributes. You will see that no code is necessary to make WCF instantiate services using the PIAB. Note that our custom behavior and instance provider are
generically designed so that they are not tightly coupled with service types. This means you can download the code and use it for any services to which you would like to
apply cross-cutting logic via the PIAB.

Extending IEndpointBehavior and IContractBehavior

Before we create our custom behavior, we must understand how WCF behaviors are discovered and applied, particularly in the context of instantiating services. When the
WCF runtime is instructed to host a service, the service model layer will go through the following process:

1. Find behaviors in configuration.

2. Find behaviors defined as attributes on the service class or operation.
3. Build channel listeners for handling messaging and transport.
4. Apply dispatch behaviors for both the configuration and attributes described in Steps 1 and 2.

For our purposes, we need to define configuration or attributes that can be applied to our service code so that our custom behavior can be found by the WCF runtime. In Step
4 of the process, the WCF runtime will apply the dispatch behavior as defined in custom behaviors. This is our chance to substitute the default instance provider with our own
instance provider.

Regarding behaviors, WCF exposes the following interfaces: IServiceBehavior, IEndpointBehavior, IContractBehavior, and IOperationBehavior. IServiceBehavior deals with
behaviors at the host level, and IOperationBehavior is limited to the operation's scope, so they are not suitable for creating a service instance. Both IEndpointBehavior and
IContractBehavior require the implementation of the ApplyDispatchBehavior method, and it is from within this method where we can make the switch and substitute our own
custom instance provider.

So if both IEndpointBehavior and IContractBehavior offer the extensibility point for us to create a service instance via the PIAB factory, which one do we use? It turns out
that we will use both. IEndpointBehavior can be used to extend the dispatch behavior in a configuration file, and IContractBehavior can be used to extend the dispatch
behavior as a .NET attribute.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 18 of 26

The Configuration Approach

The WCF configuration simply piggybacks on the .NET Framework configuration settings. In the app.config or web.config file of the .NET application that hosts the WCF
runtime, there is a <system.serviceModel> section, which you will see later.

While seemingly complex, it is really well aligned with the WCF architecture, where service and client contain endpoints that consist of address, binding, and contract
definitions.

Elements such as <behaviors>, <extensions>, and so on are simply details that are referenced by <service> and <endpoint> elements. In the configuration, behavior
customization can be performed on a <service> or an <endpoint> element. The <behaviors> element contains <serviceBehaviors> and <endpointBehaviors>. Because the
service level is not the appropriate place for us to customize behavior to instantiate a service type, we will provide a custom endpoint behavior.

As shown in Figure 5, our custom behavior will implement the IEndpointBehavior interface. Additionally, in order for it to be applied through configuration, it must derive
from System.ServiceModel.Configuration.BehaviorExtensionElement and override the BehaviorType property and the CreateBehavior method.

ApplyDispatchBehavior is the key method in IEndpointBehavior, as it allows for our custom endpoint behavior to substitute the WCF default instance provider with our
custom instance providerPolicyInjectionInstanceProvider. As shown in Figure 6, our provider implements the System.ServiceModel.Dispatcher.IInstanceProvider interface.
The GetInstance method will then be called by the WCF runtime in order to get the instance of the service type. Our implementation of GetInstance is straightforward; it asks
for the Enterprise Library's PIAB to create a policy injection proxy based on the service type and contract type (interface) of the received message. Because the default PIAB
interception mechanism uses .NET remoting's transparent proxy, a type derived from MarshalByRefObject is required if no specific contract type (interface) is specified.
Typically you don't have to worry about this requirement when using PIAB with WCF services since WCF contracts are .NET interfaces.

Now with these two custom types for IEndpointBehavior and IInstanceProvider ready, we can plug them into the configuration code, as shown in Figure 7. Note that in the
<BehaviorExtensions> element, we add the type definition for PolicyInjectionBehavior and, we apply it to the endpoint for our service. Along with the PIAB configuration,
we are now able to inject policies into the CustomerService service without writing any additional lines of code.

The Attribute Approach

In .NET, another common method of declarative programming (in addition to the use of a configuration file) is to use attributes, which get compiled into the assembly as
metadata and are visible to the CLR at run time. In Figure 8, we define a class, PolicyInjectionBehaviorAttribute, that implements
System.ServiceModel.Description.IContractBehavior and System.ServiceModel.Description.IContractBehaviorAttribute interfaces. It will need to derive from the Attribute
base class of .NET Framework in order to be applied as an attribute.

The implementation for IContractBehaviorAttribute is very simplewe're returning null in the TargetContract property, as we're targeting any contract that has the
PolicyInjectionBehaviorAttribute attached. The implementation of ApplyDispatchBehavior is similar to that of PolicyInjectionEndpointBehavior. We substitute the default
instance provider with our own PolicyInjectionInstanceProvider. This is the same instance provider that is used for the endpoint behavior for the configuration approach. The
code below shows how to inject policies through this attribute. Notice that no other effort is required from the developer besides applying this attribute:

[PolicyInjectionBehavior]
public class DecoratedCustomerService : ICustomerService {
public Customer GetCustomer(string customerID) {
return CustomerDAO.GetCustomer(customerID);
}
}

The Sample Code

To demonstrate the use of the configuration-driven and attribute-based implementations of our custom WCF Behaviors, we created a sample that illustrates both. In the
sample, we defined the service contract as an interface named ICustomerService. We then created two concrete implementations of that interface: CustomerService and
DecoratedCustomerService. The only difference between these two classes is that DecoratedCustomerService has been decorated with the PolicyInjectionBehavior attribute.

Our sample code includes a console application that hosts both CustomerService and DecoratedCustomerService. We modified the configuration of the CustomerService class
to introduce our PolicyInjectionEndPointBehavior. We also included a client application that calls each service once to perform GetCustomer functions. Here we show you
callCustomerService, but callDecoratedCustomerService looks the same:

private static void callCustomerService() {

CustomerService.CustomerServiceClient client =
new CustomerService.CustomerServiceClient();
Console.WriteLine("Retreiving Customer 1");
CustomerService.Customer cust = client.GetCustomer("1");
Console.WriteLine("Retreived Customer, Name: [" + cust.Name + "]");
}

For both the configuration and attribute approaches, we set up the PIAB configuration (see Figure 9) to include a logging handler for calls made to the services. You can add
any other handlers, such as exception handling and caching, if you like. The logging handler in our sample writes audit trails to a flat file called audit.log. After the calls are
made, audit.log is created in the same directory as the executing code unless the configuration is modified to create it elsewhere.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 19 of 26

Figure 9 App.config in Visual Studio

Figure 9 App.config in Visual Studio (Click the image for a

larger view)

As you can see, the PIAB is a good way to separate cross-cutting logic from domain-specific logic, and it can be used by WCF services. To give it a try, download the sample
code from msdn.microsoft.com/msdnmag/code08.aspx. To apply the PIAB to your WCF Services, just add a reference to the PIBehaviors assembly and then either apply the
PolicyInjectionEndpointBehavior via configuration or apply the PolicyInjectionBehaviorAttribute to your services directly.

Hugh Ang is a Senior Solution Architect at Avanade and is working as VP of Application Development for a client, overseeing its enterprise architecture as well as managing
the application development department. You can read his blog at tigerang.blogspot.com.

David San Filippo is a Solutions Developer at Avanade and holds MCPD Enterprise Application Developer, MCSD.Net, and MCDBA credentials. You can read his blog at
www.mtelligent.com.

2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.
WF How-To

Building State Machines with Windows Workflow Foundation

Keith Pijanowski

This article discusses:

l Sequential workflows vs. state machine workflows
l When to use state machine workflows This article uses the following technologies:
l Building and designing state machine workflows Windows Workflow Foundation, Visual Studio
l Raising events to running workflows
l Passing data to running workflows
Code download available at: WFStateMachines2008_02.exe (177KB)

Contents
State Machine Theory
State Machines and Windows WF
Communicating with State Machine Workflows
Events Required for Data Exchange and State Transitions
Defining an External Data Exchange Interface
Adding EventDriven Activities
Referencing the External Data Exchange Interface
Implementing an External Data Exchange Service
Add an External Data Exchange Service to the Workflow Runtime
Using the External Data Exchange
Useful State Machine APIs
Tying Up Loose Ends

T
he Microsoft .NET Framework 3.0 introduced new capabilities for visually and declaratively building workflows that can be hosted by managed code. Out of the box,
the .NET Framework 3.0 allows developers to build both sequential workflows and state machine workflows. Much has been written about sequential workflows because they
closely resemble conventional programming techniques.

State machine workflows, on the other hand, represent a different way of thinking about program logic. When designed and implemented correctly, they are just as valuable as
sequential workflows. Furthermore, this style of workflow is a good starting point for building human workflows. Consequently, state machine workflows should be a part of
every architect's and developer's skill set.

State Machine Theory

Additional Resources
Business logic that is implemented as a state machine workflow follows a very different path than business

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 20 of 26

logic implemented as a sequential workflow. Rather than flowing from activity to activity like a sequential The code download for this article contains all the
workflow, state machines transition from state to state. Think of a state as a well-known step, stage, or even a workflows, code figures, and code snippets shown in this
status within a business process. States are useful because they indicate how much of a workflow has article. It is designed as a sandbox or test arena for state
completed and how much must still occur. This lends itself very well to workflows that are long running. machine workflows; consequently it will only run one
instance of a workflow at a time.
A good indication that a business process is suited for a state machine workflow is that the requirements
describe data or logic as moving in discreet steps (or states) from some starting point to some ending point. This is a Visual Studio 2005 solution that contains the
Take a look below at the description of a supply fulfillment business process that could be implemented as a following projects:
state machine. StateMachineLibrary contains the two workflows presented
here. These workflows are the SimpleStateMachine and the
1. Employees fulfilling customer orders may need additional supplies to fulfill an order. Requests for SupplyFulfillment workflows.
additional supplies should be submitted to a central system for processing. ConsoleHost is a console application used to host the
2. When a request is submitted, it will be assigned to a specific user's work queue for review. simple state machine workflow presented at the beginning
3. Once the request has been assigned, it will be either approved or rejected based on criteria external to the of this article.
business process. WinFormHost is a Windows Forms application used to host
4. A request that is rejected will be either reassigned or canceled based on criteria external to the business the SupplyFulfillment workflow. It is in this project that the
process. Reassigned orders will be processed the same as assigned orders. Canceled orders will be ExternalDataExchange is used to raise events to an instance
considered completed. of the SupplyFulfillment workflow.
5. If the employee request is approved, the resource that is required will be ordered. ExternalDataExchange is a class library that contains the
6. Once the order is received, the business process surrounding that order is considered complete. IEventService interface, the implementation of this
interfaceEventService, and the SupplyFulfillmentArgs
Key steps of this business process that can be considered states are shown in Figure 1. class. It is this service that allows external events (and data)
to be sent to the current state of a state machine workflow.
Figure 1 Logic Design of the Supply
Fulfillment Workflow If you would like to know more, the following resources
will be useful to you:

l Using SqlTrackingService
l Using SqlWorkflowPersistenceService
l A Tour of Windows Workflow Activities
l Simplify Development with the Declarative Model of
Windows Workflow Foundation
l Windows Workflow Foundation
l Windows Workflow Foundation, Part 2

Figure 1 Logic Design of the Supply

Fulfillment Workflow

There are a few key aspects that are worth noting before building this workflow using Windows Workflow Foundation (Windows WF). First, the workflow has exactly one
initial state and exactly one completed state: submitted and completed, respectively. Every instance of a state machine workflow begins in the state that has been designated as
the initial state and ends when the workflow reaches the state designated as the completed state.

Second, state transitions are predefined. In other words, part of the definition of each state is a list of allowable next states. Figure 2 lists the states and the allowed transitions.

Third, backtracking is allowed. Notice that our requirements specify that rejected requests can be reassigned. A reassigned state could have been placed into the logical
design, but this would not have been an efficient design. The requirements specify that reassigned requests are to be processed in the same manner as newly assigned requests,
so there is no need to create a separate branch off of the rejected state for handling reassigned requests. In other words, transitioning a workflow back to a previous state
allows for better reuse within the workflow.

Fourth, state machine workflows are suited for long-running processes. If a state machine workflow flows from its initial state to its completed state in a matter of
milliseconds and never pauses for external input, then the workflow could be more efficiently implemented using a sequential approach.

State Machines and Windows WF

Before implementing the logical design discussed earlier, let's explore a simple Windows WF state machine to get a feel for all the tools WF provides. I'll build a simple state
machine workflow as an introduction to the design-time experience of Visual Studio .NET and the tools in the .NET Framework 3.0. Note that all the workflows and code
snippets shown here come from a working Visual Studio solution that is packaged as a code download for this article. For more information, see the "Additional Resources"
sidebar.

When you install the Visual Studio 2005 extensions for .NET Framework 3.0 (Windows WF) from go.microsoft.com/fwlink/?LinkId=105219, several new project templates
are added to the Visual Studio .NET 2005 environment (see Figure 3). Note that if you are using Visual Studio 2008, all of these project templates have been included as a
part of the core product. Once you select one of the state machine project templates, a Visual Studio .NET project will be set up with the correct references and a workflow
file that can be used as a starting point for the development of a state machine workflow.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 21 of 26

Figure 3 Workflow Foundation Project Templates

Figure 3 Workflow Foundation Project Templates (Click the

image for a larger view)

The designer for a state machine workflow is much more restrictive than the designer for a sequential workflow. Most state machines will only require the State activity to be
dragged onto the designer. (It is also possible to add an EventDriven activity to the designer for handling events that are not associated with a specific state.) The State activity
represents a step, stage, or status within a state machine workflow. This activity is used to represent the initial state, the completed state, and all the potential states that may
occur in between.

The first step in building a state machine is to place all the needed states on the design surface by dragging the state activity from the workflow toolbox and assigning each
state a meaningful name. Workflow states are named using the properties dialog in the same way that Windows controls are named in a Windows Forms project. Figure 4 is
an example of a simple state machine workflow containing three states: Submitted, Ordered, and Completed.

Figure 4 Simple State Machine Workflow

Figure 4 Simple State Machine Workflow (Click the image

for a larger view)

Workflow Foundation needs to know the initial state and the completed state for the purposes of proper run-time management. Therefore, the next step is to specify them via
the property dialog of the Workflow itself. To view this dialog make sure that the property dialog is visible within Visual Studio .NET and then click on the design surface of
your workflow, or just right-click the workflow designer and select properties. The lower right-hand corner of Figure 4 shows the property dialog for a state machine
workflow. The InitialStateName and CompletedStateName properties are used to specify the initial state and the completed state respectively. Once these properties are
specified, the designer will modify the graphical depiction of the specified states. Look closely at Figure 4 and you'll see the green light icon on the initial state and the red
light icon on the completed state.

The State activity is a composite activity, meaning that it can contain other activities. Once again there are restrictions to consider. A state activity may contain only one
StateInitialization activity, only one StateFinalization activity, one or more EventDriven activities, and one or more other State activities for nested states. These are the only
four out-of-the-box activities that can be placed inside a state activity. Nested states are beyond the scope of this article and the use of the EventDriven activity will be
described in the next section. Our simple state machine example will only utilize the StateInitialization activity and the StateFinalization activity.

Experimenting with the StateInitialization and StateFinalization activities is a good way to get a feel for how to work with the state machine designer. Playing around with
these two activities will also provide a deeper understanding of how to control the flow of a state machine workflow.

All of the states in the simple example in Figure 4 have been equipped with a StateInitialization activity and a StateFinalization activitywith the exception of the completed
state. It turns out that once a state is specified as completed, no activities can be added to it. This is because the act of transitioning to the completed state signifies the end of a
workflow's lifecycle.

When program control is passed to a state activity and it begins execution, the state activity will check to see whether a StateInitialization activity is present. If one is present,
it will be executed. Similarly, when a state activity is about to pass control to another state, it will check to see if a StateFinalization activity is present. If so, it will be
executed. It is important to note that both the StateInitialization activity and the StateFinalization activity are also composite activities. Unlike the state activity, there are very
few restrictions on what can be placed inside these activities. In other words, all the activities that can be used in a sequential workflow can be placed inside these activities.
Visual Studio will provide a different design view for placing logic into these composite activities. Figure 5 shows the design view that is displayed after double-clicking on
the StateInitialization activity of the Submitted state shown in Figure 4. Here the Submitted state has been equipped with a StateInitialization activity named
SubmittedInitialization. The purpose of this design view is to allow additional activities to be placed inside the StateInitialization activity. In the example in Figure 5, a Code
activity and a SetState activity have been placed within the SubmittedInitialization activity. The following code has been placed into the execute event of the code activity:

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 22 of 26

Figure 5 Design View for a StateInitialization

Activity

Figure 5 Design View for a

StateInitialization Activity

private void codeSubmittedInitialization_ExecuteCode(object sender,

EventArgs e)
{
Console.WriteLine("The Submitted state has been initialized at " +
DateTime.Now.ToString() + ".");
}

The SetState activity (setStateOrdered in Figure 5) is used to transition control to another state in the StateMachine workflow. No code is necessary to utilize this activity.
Figure 6 shows the properties dialog for setStateToOrdered. The TargetStateName property must be set to a valid state in the StateMachine workflow. Immediately after this
activity executes, the current state's StateFinalization activity will execute and control will pass to the Ordered state. No additional activities will execute within the
SubmittedInitialization activity. Therefore, care must be taken to insure that it is the last activity. The StateFinalization activity for our Submitted State has been configured
with a single code activity with the following code in its execute event:

Figure 6 Properties for a SetState Activity

Figure 6 Properties for a SetState Activity (Click the image

for a larger view)

private void
codeSubmittedFinalization_ ExecuteCode(
object sender, EventArgs e)
{
Console.WriteLine("The Submitted " +
"state has been finalized at " +
DateTime.Now.ToString() + ".");
}

It is important to note that a SetState activity cannot be placed within a StateFinalization activity. This wouldn't make sense anyway since the only way a StateFinalization
activity can execute is if a SetState has already executed. A SetState activity can only reside within a StateInitialization or an EventDriven activity.

To return the designer to the original design view, which shows the entire workflow, just click the workflow name that appears in the upper left corner of the designer shown
in Figure 5. This area of the designer is used for navigation. Clicking on "SimpleStateMachine" returns the design view to the original view that shows all the states in the
workflow. If your state machine is very complicated and contains many levels of nested states, this navigational area of the designer will show you exactly where you are in
the workflow.

At this point, the state machine designer will detect any state transitions specified via the setState activities and draw the appropriate transition arrows on the design surface.
The TargetStateName property for SetState activities can be modified in two ways. The first, which was just described, is to use the property dialog. The TargetStateName
property can also be set directly from the workflow's design surface by dragging and dropping the arrow end of a transition arrow to the desired State activity.

The ordered state in our simple state machine workflow (Figure 5) was set up in a fashion similar to the submitted state. In other words, a Code activity was used in both the
StateInitialization and the StateFinalization activities to write a message to the console. In addition, a SetState activity was used in the StateInitialization activity to transition
control to the completed state in the state machine workflow.

We are now ready to use the workflow. The code to instantiate the workflow runtime, create an instance of this workflow, and start the workflow from a console application is
shown here:

static void Main(string[] args)

{
// Create the workflow runtime.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 23 of 26

using (WorkflowRuntime workflowRuntime = new WorkflowRuntime())

{
WorkflowInstance instance =
workflowRuntime.CreateWorkflow(typeof(SimpleStateMachine));
instance.Start();

Console.ReadLine();
}
}

After this code is executed, the following output will appear in the console output window:

The Submitted state has been initialized at 8/6/2007 7:40:36 PM.

The Submitted state has been finalized at 8/6/2007 7:40:36 PM.
The Ordered state has been initialized at 8/6/2007 7:40:36 PM.
The Ordered state has been finalized at 8/6/2007 7:40:36 PM.

Notice that the workflow described in this section implemented three of the needed states (Submitted, Ordered, and Completed) for the Supply Fulfillment workflow I
described at the beginning of the article. I will now add the other states (Assigned, Approved, and Rejected) to this workflow to illustrate backtracking as well as states that
can transition to more than one possible state. I will also add events.

Communicating with State Machine Workflows

Now that you've seen the basic moving parts of a WF state machine, I'd like to discuss some real-world considerations. State machines in the real world are often long-
running, spending most of their lifetime paused within a state, waiting for input that is external to the workflow itself. An executing state can be paused to wait for external
information using the EventDriven activity. As I stated previously, the EventDriven activity is one of the few activities that can be dropped into a State activity (the others are
StateInitialization and StateFinalization). External information is passed to the EventDriven activity by way of events. Specifically, events are raised by an external data
service and are received by the EventDriven activity. The EventDriven activity can be a little tricky to set up, but the following steps should help to get you started.

1. Determine the events required for data exchange and state transitions.
2. Define an External Data Exchange interface.
3. Add EventDriven activities to the state machine workflow.
4. Reference the external data exchange interface.
5. Implement the External Data Exchange interface.
6. Add the External Data Exchange to the workflow runtime.
7. Use the External Data Exchange to exchange data and transition states.

Figure 7 shows exactly how the pieces mentioned in the previous steps fit together.

Figure 7 Configuring and Utilizing a State Machine Workflow

Figure 7 Configuring and Utilizing a State Machine

Workflow (Click the image for a larger view)

Events Required for Data Exchange and State Transitions

The first step is to determine which events are needed by each state in the state machine workflow. State machine events typically cause the workflow to change states.
Therefore, the best way to determine the needed events is to review requirements and look for key words that indicate state transitions. Recall that to build the logical design
of Figure 1, we needed to review the requirements to determine the necessary states and state transitions. That information can also be used to determine the events that are
needed. Figure 8 expands upon the data in Figure 2, which listed the state transitions for each state within the Supply Fulfillment workflow. Figure 8 shows each state in the
workflow, the events that each state will listen for, and the state that will be transitioned to for each event listed. If data is needed by the workflow instance in order to move to
a new state, then Workflow Foundation provides capabilities that allow information to be passed as event parameters.

Notice that the Assigned state is listening for two events. The Assigned state will listen for both of these events at the same time. The event received determines the state
transition. This is how external entities can determine the path that a state machine workflow will follow. (The Rejected state will also listen for two events.)

Defining an External Data Exchange Interface

The next step is to define these events within a standard .NET interface. Figure 9 shows the interface needed for the Supply Fulfillment workflow and Figure 10 shows the
custom EventArgs class that is used to pass data to an instance of a workflow whenever one of these events is raised.

There are a few things to note about this interface. The first is the use of the ExternalDataExchange attribute. A class that implements an interface marked with the
ExternalDataExchange attribute can be added to the workflow runtime engine just like any other service. (Persistence and tracking are examples of other services that can be
added to the workflow runtime.)

Also notice that SupplyFulfillmentArgs, which is used as a parameter in the events declared in the interface, must inherit from ExternalDataEventArgs. Furthermore,
ExternalDataEventArgs requires the workflow instance ID to be passed to its constructor so that the event will get raised to the correct workflow instance. Finally,
SupplyFulfillmentArgs must be marked as serializable in case the workflow runtime needs to serialize it to persistent storage.

Adding EventDriven Activities

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 24 of 26

Once the interface for the External Data Exchange is set up, the next step is to add EventDriven activities to the state machine. Similar to the StateInitialization and the
StateFinalization activities, the EventDriven activity is a composite activity. Its purpose, therefore, is to contain other activities. Once the EventDriven activity is added to a
State activity, we will be able to drill into it in the same way we drilled into the StateInitialization and the StateFinalization activities for the purposes of defining additional
logic. Unlike the StateInitialization and the StateFinalization activities, we have to follow a few rules when placing activities inside the EventDriven activity. It turns out that
the EventDriven activity inherits from the Sequence activity and adds the restriction that the first child activity must be an activity that implements the public interface
IEventActivity. All subsequent activities can be of any type. In the example presented here, the HandleExternalEvent activity is used. This activity is capable of receiving an
event that is external to the workflow runtime. Two other activities that also implement the IEventActivity interface are the Delay activity and the WebServiceInput activity.

In the SupplyFulfillment workflow, there are seven logical events that will require an EventDriven activity. Figure 8 contains the information needed for placing the
EventDriven activities into the correct state. It is also important to note that logic placed within these activities will ultimately result in the state changes. Figure 11 shows the
designer view of the SupplyFulfillment state machine once all the EventDriven activities have been added. Notice that the transition arrows originate from the EventDriven
activities and end on the State activities. Figure 12 shows the logic contained within the OnAssigned EventDriven activity.

Figure 11 EventDriven Activities in a State Machine Workflow

Figure 11 EventDriven Activities in a State Machine

Workflow (Click the image for a larger view)
Figure 12 Logic within an EventDriven
Activity

Figure 12 Logic within an EventDriven

Activity

Technically, the StateInitialization and the StateFinalization activities which are shown in Figure 11 are no longer needed. They are included here and equipped with console
messages for demonstration purposes only.

It is worth mentioning that the visual depiction of the Supply Fulfillment workflow in Figure 11 is just as easy to read as the logical diagram in Figure 1 that was created with
Microsoft Visio. Development artifacts that developers, architects, and end users can easily understand and verify are a core tenet of business process management.

Referencing the External Data Exchange Interface

Once the EventDriven activities are in place and the child activities for each EventDriven activity are set up in a manner similar to Figure 11, the next step is to configure the
HandleExternalEvent activities. Remember that when setting up an EventDriven activity, one of the rules of construction is that the first child activity must be an activity like
the HandleExternalEvent activity. This can be seen in the logic in Figure 12 where an EventDriven activity named OnAssigned has as its first child activity a
HandleExternalEvent activity named handleAssigned, followed by a SetState activity named SetStateAssigned.

The HandleExternalEvent activity is what will cause a StateMachine workflow to block while waiting to handle an event that is raised by an external data exchange service.
The specific event that a HandleExternalEvent activity will wait for is specified in the properties dialog (shown in Figure 13). The first property to set is the InterfaceType
property, which must be set to an interface that is marked with the ExternalDataExchange Attribute. In the case of the Supply Fullfilment workflow, this interface is the
IEventService interface shown in Figure 9. Once the InterfaceType property is set, the EventName property can be specified. This properties dialog will reflect over the
interface specified via the InterfaceType property and provide a dropdown box for the EventName property. The dialog shown in Figure 13 is for the Supply Fulfillment

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 25 of 26

workflow's OnAssigned event.

Figure 13 handleAssigned Properties

Figure 13 handleAssigned Properties (Click the image for a

larger view)

Before leaving this dialog, there is one more consideration. Remember that all the events in the ExternalDataExchange.IEventService interface require a
SupplyFulfillmentArgs argument. This argument can be used to pass data into an instance of a workflow when an external event is raised and the workflow receives the event.
There are two ways to make this data type accessible to other activities within the workflow. One way is to use the "e" property shown in Figure 13. This property allows you
to specify a property within the workflow that can be used to hold the argument passed by the event. Once this event parameter is set into a workflow property, it can be
utilized by other activities in the workflow. Obviously the property specified will need to be of the same type as the event parameter. In the case of the SupplyFulfillment
workflow, this would be the SupplyFulfillmentArgs type. Alternatively, the Invoked property can be used to specify an event handler within the workflow. This event handler
will be called after the external event is received and it will be passed a parameter of type ExternalDataEventArgs, which can be cast into the correct type
(SupplyFulfillmentArgs).

It is important to note at this point that the HandleExternalEvent activity deals only with the ExternalDataExchange interface and not with an actual implementation of the
interface.

Implementing an External Data Exchange Service

In order to successfully implement an external data exchange service, you need to implement the interface referenced by the HandleExternalEvent activities in the workflow.
Specifically, the IEventService interface must be implemented. The following code creates the EventService class and implements the interface:

[Serializable]
public class EventService : IEventService
{
public event EventHandler<SupplyFulfillmentArgs> Assigned;
public event EventHandler<SupplyFulfillmentArgs> Approved;
public event EventHandler<SupplyFulfillmentArgs> Rejected;
public event EventHandler<SupplyFulfillmentArgs> Reassigned;
public event EventHandler<SupplyFulfillmentArgs> Canceled;
public event EventHandler<SupplyFulfillmentArgs> Ordered;
public event EventHandler<SupplyFulfillmentArgs> OrderReceived;
...
}

Since events can only be raised from within the class in which they are defined, we need to add helper functions to facilitate raising events. This class will need one helper
function per event. The following code shows the helper function for the Assigned event:

public void RaiseAssignedEvent(System.Guid instanceId, string

assignedTo)
{
// Check to see if event is defined
if (this.Assigned != null)
{
// Create the EventArgs for this event
LocalService.SupplyFulfillmentArgs args = new
LocalService.SupplyFulfillmentArgs(instanceId);
args.AssignedTo = assignedTo;

// Raise the event

this.Assigned(this, args);
}
}

Note that when the event is raised there are two parameters that are passed to all registered delegates. The first parameter is the sender. The code above passes the current
instance of the EventService class, which must be marked as serializable for an instance of it to be passed as the sender. If it is not, then you will receive an error stating that
the event could not be delivered. Here is an example of this error message:

Event "Assigned" on interface type

"ExternalDataExchange.IEventService"
for instance id "0f9e8545-39bf-48b6-b890-
c12dec11cb9f" cannot be delivered.

The second parameter is the SupplyFulfillmentArgs parameter. When this class is instantiated, it requires the workflow instance identifierthe Guid that was generated when
the workflow was first instantiated. It is this Guid that will be used by the workflow runtime to locate the workflow instance that should receive the event. For this to occur,
however, an instance of the EventService class must be added to the workflow runtime as an external service.

Add an External Data Exchange Service to the Workflow Runtime

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016
Roll Your Own: Create a Language Compiler for the .NET Framework Page 26 of 26

Since the workflow runtime manages all execution and interaction of instantiated workflows, the class that was just created previously must be added to the workflow runtime
as an external data exchange service. The following code shows how to add an instance of the EventService class to the workflow runtime as an ExternalDataExchange
service class:

// Create a new instance of the local service and host it in an

//ExternalDataExchangeService.
ExternalDataExchangeService externalDataSvc = new
ExternalDataExchangeService();
wr.AddService(externalDataSvc);
eventService = new EventService();
externalDataSvc.AddService(eventService);

The workflow runtime is very particular about how this is done. The external data service must first be created and added to the workflow runtime before the EventService
class can be added to the ExternalDataExchangeService class. If you try to use the ExternalDataExchangeService object before it is added to the workflow runtime, you will
receive an error that states "WorkflowRuntime container does not contain ExternalDataExchangeService."

Once this is complete, events raised from the instance of the EventService class will find their way to the correct instance of a workflow and to the correct
HandleExternalEvent activity within the workflow.

Using the External Data Exchange

Everything is now set up and the External Data Exchange can be used to exchange data and transition the states of the workflow. Using the External Data Exchange is as easy
as calling into the helper functions that were set up within the EventService class shown earlier. The code snippet below shows the use of the helper function that will change
the workflow state from Submitted to Assigned. Notice that the workflow instance ID and an "assignTo" parameter are passed into the function. Both of these parameters will
be packaged into an instance of the SupplyFulfillmentArgs class so they can be passed to the workflow via the Assigned event. As I stated, the workflowInstanceID parameter
is used by the workflow runtime so it can raise the Assigned event to the correct workflow instance. The assignTo parameter ("Joe") will be delivered directly to the workflow
to be used during workflow execution:

eventService.RaiseAssignedEvent(workflowInstanceId, "Joe");

It is important to note that at any given point, a running state machine workflow is only listening for the events within the current State activity. If any other events are raised,
the calling code will receive an exception. For example, once the SupplyFullfilment workflow is instantiated, it is in its initial state (the Submitted state), which means the
workflow is waiting for the Assigned event to be raised. If any of the other helper functions are called that raise other events (Approved, Rejected, Reassigned, Canceled,
Ordered, and OrderReceived), you will receive an error stating that the event could not be delivered. Below is an example of an error message that will occur if the Ordered
event is called when the workflow is in any state other than Approved:

Event "Ordered" on interface type "ExternalDataExchange.IEventService" for instance id "59657ed0-1532-42c5-8abe-6523bf121fff" cannot be delivered.

As you can see, this very generic error message occurs whenever anything goes wrong with the delivery of an event and there is no information in this message that helps to
determine exactly what went wrong.

Obviously, this raises a few questions as to how to write workflow code that is not fragile. Specifically, how can your code know the current state of a state machine instance
so you can programmatically ensure that the correct event is sent to the workflow? It turns out that there are a handful of useful APIs that can be used by the code consuming a
state machine workflow.

Useful State Machine APIs

WF comes with a utility class named StateMachineWorkflowInstance specifically designed for managing instances of state machine workflows. Figure 14 provides a brief
description of the constructor, properties, and methods of the StateMachineWorkflowInstance class.

If you do use this class, however, heed these warnings. First, state machines that are event-driven (like the SupplyFulfillment workflow) are inherently asynchronous.
Therefore, if the public properties shown in Figure 14 are queried immediately after raising an event to a state machine instance, they will not accurately reflect the state of the
workflow. Event-driven workflows will be idle once they have finished processing the current event and are waiting for the next event. Therefore, the OnWorkflowIdled event
of the workflow runtime is a good place to query an instance of a state machine using these properties.

Second, the StateHistory property requires a tracking service to be added to the workflow runtime. If this property is used without a tracking service, you will receive the
following error:

'StateHistory' property requires the 'System.Workflow.Runtime.Tracking.SqlTrackingService' service.

Finally, be careful when exposing the functionality of the SetState method to end users. This method can cause a transition to any state in the workflow and is best used by
administrators or managers dealing with non-standard scenarios.

Tying Up Loose Ends

While I covered state machine theory, the design-time experience, and the basic activities needed to build a usable state machine, there are still a few important issues to
consider. First, if a state machine is to run in a server environment, it must optimize system resources while executing and must be able to survive system restarts.
Additionally, it may be desirable to track and save historic data so that business processes can be optimized based on tracking reports. For example, if the Supply Fulfillment
workflow in this article were used to run a small business, it would be desirable to see how long it takes for each workflow to move from the Ordered state to the Completed
state. If a specific supplier constantly violated service level agreements, the business process of acquiring new supplies could be optimized by simply omitting the offending
supplier.

Another topic that deserves further investigation is human workflows. State machines are a good starting point for building human workflows. If a state machine is to be used
to manage business processes with human input, care must be taken when building the user interface. The user should have a clear view of the desired path through the state
machine and should only be presented with options that are relevant for the current state.

Keith Pijanowski is a Platform Strategy Advisor for the Microsoft Developer and Platform Evangelism team, helping customers apply new ideas and new technologies to
business problems. Keith is also a frequent speaker at Microsoft events in the New Jersey and New York area. You can reach him at www.KeithPij.com.

2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.

file:///C:/Users/puneetsr/AppData/Local/Temp/~hhE488.htm 12/22/2016