Unit Testing

In C# with MSTest, NUnit and xUnit

 Create a unit test project 1
• In Visual Studio 202x add or
create a new "ProjNameTest"
project
• Filter on C#, Windows and
Test
• Or via cmd line in .NET X:
”dotnet new mstest, xunit or
nunit” in your src folder
• Enter dotnet build ProjName
or SolutionName to build your
source files
• Enter ”dotnet new -h” for all
options in the cmd console
• PS C:\>dotnet new list
 Create a unit test project 2
<ItemGroup>
• At least the following NuGet <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
packages should be referenced in <PackageReference Include="xunit" Version="2.7.0" />
the .csproj file if a mstest, nunit or <PackageReference Include="xunit.runner.visualstudio" Version="2.5.7" />
</ItemGroup>
xunit .NET X test project is created
(more packages are included) <ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
• NuGet packages may be updated <PackageReference Include="MSTest.TestAdapter" Version="3.3.1" />
via the NuGet package manager <PackageReference Include="MSTest.TestFramework" Version="3.3.1" />
</ItemGroup>
• NuGet storage: [Drive]\Users\
[UserName]\.nuget\packages <ItemGroup>
• Prune nuget packages with <PackageReference Include="nunit" Version="4.1.0" />
<PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
</ItemGroup>
https://github.com/chrisraygill/NuGetCleaner
If you are not able to review/edit the .csproj file via right click project and select Edit <project name>
Perform the following action to view or edit the .csproj file in visual studio.
- Right click on your project in solution explorer and select Unload Project.
- Right click on the project (tagged as unavailable in solution explorer) and click "Edit yourproj.csproj".
This will open up your CSPROJ file for editing.
- After making any changes you want, save and close the file. Then right click and select Reload Project.
 Create a unit test project 3
• Add a Project reference... from the unit test projects to the tested project source
using System; // Program class cont.
namespace MathOps;
public static class MathHelper
class Program {
{ public static int Add(int a, int b) => a + b;
static void Main(string[] args) }
{ }
Console.WriteLine(MathHelper.Add(5, 10));
Console.WriteLine("\nPress any key to exit."); // Note the minor differences in
Console.ReadKey(); // class/method attribute naming
}
MathOps and MathOps***Test // -------------------------------------- > cont.
// and Assert.That/(ARE)Equal
using MathOps;
using NUnit.Framework; using MathOps; using MathOps;
using Xunit; using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace MathOpsNuTests;
namespace MathOpsXuTests; namespace MathOpsMsTests;
[TestFixture]
public class UnitTest1 public class UnitTest1 [TestClass]
{ { public class UnitTest1
[Test] [Fact] {
public void Test_Add() public void Test_Add() [TestMethod]
{ { public void Test_Add()
var actual = MathHelper.Add(1, 1); var actual = MathHelper.Add(1, 1); {
var expected = 2; var expected = 2; var actual = MathHelper.Add(1, 1);
Assert.That(expected, Assert.Equal(expected, actual); var expected = 2;
Is.EqualTo(actual)); } Assert.AreEqual(expected, actual);
} } }
} }
 .NET architectural components 1
• A .NET app is developed for and runs on one or more implementations of .NET
• There are four primary .NET implementations that Microsoft actively develops and
maintains

.NET X (.NET Core 3.x, 5.x and higher) is a cross-platform implementation of .NET and designed to
handle server and cloud workloads at scale. It runs on Windows, macOS and Linux. It implements
the .NET Standard, so code that targets the .NET Standard can run on .NET X/Core. ASP.NET X/Core
runs on .NET X/Core

.NET Framework (4.x) is the original .NET implementation that has existed since 2002

Mono is a .NET implementation that is mainly used when a small runtime is required. It is the runtime
that powers Xamarin (.NET MAUI) applications on Android, Mac, iOS, tvOS and watchOS and is focused
primarily on a small footprint. Mono also powers games built using the Unity engine

UWP (Universal Windows Platform) is an implementation of .NET that is used for building modern, touch-
enabled Windows applications and software for the Internet of Things (IoT)
• There is an API specification common to all implementations of .NET that's called
the .NET Standard

The .NET Standard is a set of API:s that are implemented by the Base Class Library of a .NET
implementation. .NET Standard is a target framework which is common for all implementations

https://docs.microsoft.com/en-us/dotnet/standard/components
 .NET architectural components 2
/ /
.NET .NET MAUI

.NET Framework vs .NET Core vs .NET vs .NET Standard vs C#: https://www.youtube.com/watch?v=4olO9UjRiww

.NET >= 5.0 => .NET Core vNext

PS C:\>dotnet --info

= General Availability (GA)

https://dotnet.microsoft.com/en-us/
 Target frameworks and .NET X
In the VS project file:
https://docs.microsoft.com/en-us/dotnet/standard/frameworks
ProjName.csproj

TFM:s <Project
Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>

<TargetFramework>net9.0</TargetF
ramework>

MAUI <ImplicitUsings>enable</ImplicitU
sings>
<Nullable>enable</Nullable>
</PropertyGroup>

and higher <ItemGroup>

<PackageReference
Include="editorconfig"
Version="0.12.2" />
</ItemGroup>

</Project>
Xamarin vNext: https://github.com/dotnet/maui
 Big Performance Improvements
in .NET 5/6/7/8/9/...
• Performance Testing

BenchmarkTesting project in lectures/code_csharp folder

https://github.com/dotnet/benchmarkdotnet

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042

Intel Core i7-6700HQ CPU 2.60GHz (Skylake), 1 CPU, 8 logical and 4 physical cores

| Type | Runtime | Toolchain | Ratio |

|-------------- |-------------- |------------- |------:|
| DoubleSorting | .NET 4.8 | net48 | 1.00 |
| DoubleSorting | .NET Core 3.1 | netcoreapp31 | 0.82 |
| DoubleSorting | .NET Core 5.0 | netcoreapp50 | 0.43 |
| | | | |
BenchmarkTesting
| Int32Sorting | .NET 4.8 | net48 | 1.00 |
| Int32Sorting | .NET Core 3.1 | netcoreapp31 | 0.76 |
| Int32Sorting | .NET Core 5.0 | netcoreapp50 | 0.44 |
| | | | |
| StringSorting | .NET 4.8 | net48 | 1.00 |
| StringSorting | .NET Core 3.1 | netcoreapp31 | 0.83 |
| StringSorting | .NET Core 5.0 | netcoreapp50 | 0.54 |

https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/
Performance Improvements in .NET 8/9/... and AOT
• Publishing your app as Native AOT produces an app that's self-contained and that has
been ahead-of-time (AOT) compiled to native code
• Native AOT apps have faster startup time and smaller memory footprints. These apps
can run on machines that don't have the .NET runtime installed
• https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/
Add the following to the PropertyGroup setting in the .csproj file
<PropertyGroup>
<PublishAot>true</PublishAot> BenchmarkTesting_net8
</PropertyGroup>
dotnet publish -r win-x64 -c Release
dotnet publish -r linux-arm64 -c Release

| Type | Runtime | Ratio |

|-------------- |--------- |------:|
| DoubleSorting | .NET 7.0 | 1.00 |
| DoubleSorting | .NET 8.0 | 0.80 |
| | | |
| Int32Sorting | .NET 7.0 | 1.00 |
| Int32Sorting | .NET 8.0 | 0.95 |
| | | |
| StringSorting | .NET 7.0 | 1.00 |
| StringSorting | .NET 8.0 | 0.79 |
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/native-aot
 .NET X Modernizing
• Windows Desktop Apps and .NET 5 (good overview) matching-game-xxx


https://www.codemag.com/Article/2010122/Windows-Desktop-Apps-and-.NET-5
• Modernizing Desktop Apps on Windows with .NET 7 (book)

https://docs.microsoft.com/en-us/dotnet/architecture/modernize-desktop/
• Overview of porting from .NET Framework to .NET

try-convert (dotnet tool install / update -g try-convert)

upgrade-assistant (dotnet tool install / update -g upgrade-assistant)

.NET API Portability Analyzer

.NET API Analyzer

https://docs.microsoft.com/en-us/dotnet/core/porting/
• Porting Projects to .NET 5 (links in YouTube comments)

https://www.youtube.com/watch?v=bvmd2F11jpA
• More info and resources at:
OneDrive > gmi2j3 > docs.and.code > Porting_Projects_to_.NET5+/
 VS – Extensions > Manage Extensions...

https://
marketplace.vis
ualstudio.com/

https://visualstudio.microsoft.com/vs/features/extend/
 Unit testing in C#
• Test methods recommended naming convention

(Test)_MethodToTest_ScenarioWeTest_ExpectedBehaviour

In the test method // In C# with MSTest, Nunit, xUnit
[TestClass], [TestFixture],
the pattern we use is public
{
class ReservationsTests

”tripple A” [TestMethod], [Test], [Fact]

public void CanBeCancelledBy_AdminCancelling_ReturnsTrue()

Arrange {
// The arrangement below is called tripple A
// Arrange - here we initialize our objects

Act var reservation = new Reservation();

Assert // Act - here we act on the objects
var result = reservation.CanBeCancelledBy(
new User { IsAdmin = true });

// Assert - here we verify the result

Assert.IsTrue(result); // MSTest
Assert.That(result, Is.True); // NUnit
Assert.True(result); // xUnit
}
}
 Assert.AreEqual()
https://docs.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.testtools.unittesting.assert.areequal

+17 overloads
 Run the test from VS and cmd
• Run (Ctrl+R, A) and/or debug tests via the
main menu or the Test Explorer (right-click)
• Test Settings > Processor Architecture for ...
may need some review (x86/x64)
• If the test is not discoverable you may
have to rebuild or reload the solution
• By default the tests are grouped by output
result, so you can quickly find failed tests
• Run the tests from the command line

Using .NET X, you can run tests from
the console using the following command

dotnet test

dotnet test docs: https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test
 Visual Studio CodeLens
• CodeLens which is extremely useful may not be enabled – it can however be
activated and configured via the Tools > Options menu
• Above every important line of code like methods, classes or properties, Visual
Studio inserts a line with helpful links (code must be version controlled – git etc.)

Latest Tests and their Status

References used and Code Maps

Last author
• Read more about CodeLens here:

https://docs.microsoft.com/en-us/visualstudio/ide/
find-code-changes-and-other-history-with-codelens
 Exploring Asserts
• Assertions are central to unit testing regardless of test
framework
• An assertion method is a method that takes parameters and
ensure a condition is met
• If an assertion fails, an exception is thrown, the method call
does not return and an error is reported

The tested method should throw an exception with a formatted message
about the error in question
[TestMethod]

If a test contains multiple public void TestMultipleAsserts()
assertions, any assert that {
follow the one that failed Assert.AreEqual(10, 5 + 5); // Success
Assert.AreEqual(10, 1 + 2); // 10 ≠ 3 => Error
will not be executed! Assert.AreEqual(20, 1 + 2); // Not executed
}
 Standard C# Asserts
• Trace.Assert and Debug.Assert checks for a specific condition

If the condition is false, they display a message box that shows the call stack

If the condition is true, a failure message is not sent and the message box is not displayed

.NET X may not show a message box until .NET X WinForms is referenced?
• By default, the Debug.Assert method works only in debug builds
• Use the Trace.Assert method if you want to do assertions in both debug and release builds
/// <summary> TryCatchFinallyYield
/// https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.trace.assert
/// https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.debug.assert
/// </summary>
/// <param name="type"></param>
/// <param name="baseType"></param>
public static void MyMethod(Type type, Type baseType)
{
// Create an index for an array.
int index;
// Perform some action that sets the index.
index = -40;
// Test that the index value is valid.
Debug.Assert(index > -1);
// Perform some processing.
Trace.Assert(type != null, "Type parameter is null");
}
 MSTest - Assertion on a single value 1
• The following Assert and StringAssert methods validate that a value meet a specific condition
• Assert.AreEqual(object expected, object actual): Tests whether the specified values are equal. Different numeric types are
treated as unequal even if the logical values are equal. 42L is not equal to 42.
• Assert.AreNotEqual(object expected, object actual): Tests whether the specified values are unequal. Same as AreEqual for
numeric values.
• Assert.AreSame(object expected, object actual): Tests whether the specified objects both refer to the same object
• Assert.AreNotSame(object expected, object actual): Tests whether the specified objects refer to different objects
• Assert.IsTrue(bool condition): Tests whether the specified condition is true
• Assert.IsFalse(bool condition): Tests whether the specified condition is false
• Assert.IsNull(object value): Tests whether the specified object is null
• Assert.IsNotNull(object value): Tests whether the specified object is non-null
• Assert.IsInstanceOfType(object value, Type type): Tests whether the specified object is an instance of the expected type
• Assert.IsNotInstanceOfType(object value, Type type): Tests whether the specified object is not an instance of the wrong
type
• StringAssert.Contains(string value, string substring): Tests whether the specified string contains the specified substring
• StringAssert.StartsWith(string value, string substring): Tests whether the specified string begins with the specified substring
• StringAssert.Matches(string value, Regex regex): Tests whether the specified string matches a regular expression
• StringAssert.DoesNotMatch(string value, Regex regex): Tests whether the specified string does not match a regular
expression
 MSTest - Assertion on a single value 2
• The usage is pretty clear, but the difference for AreEqual and AreSame is more subtile

[TestMethod] MathOpsMsTests
[Timeout(TestTimeout.Infinite)] // Test timeout attribute in Milliseconds
public void TestSingleValAsserts()
{
var expected = "a";
var actual = "a";

Assert.AreEqual(expected, actual); // success

Assert.AreSame(expected, actual); // fail

Assert.AreNotEqual(expected, actual); // fail

Assert.AreNotSame(expected, actual); // success

Assert.AreEqual(42, 42); // success

Assert.AreEqual(42, 42L); // fail

StringAssert.Contains("Bla bla bla...", "bla"); // success

StringAssert.StartsWith("Bla bla bla...", "Bla"); // success
}
 MSTest - Assertion on collections 1
• The following CollectionAssert methods validate that ”items” meet a specific condition
• CollectionAssert.AreEqual(ICollection expected, ICollection actual): Tests whether the specified collections have the same
elements in the same order and quantity.
• CollectionAssert.AreNotEqual(ICollection expected, ICollection actual): Tests whether the specified collections does not
have the same elements or the elements are in a different order and quantity.
• CollectionAssert.AreEquivalent(ICollection expected, ICollection actual): Tests whether two collections contain the same
elements.
• CollectionAssert.AreNotEquivalent(ICollection expected, ICollection actual): Tests whether two collections contain different
elements.
• CollectionAssert.AllItemsAreInstancesOfType(ICollection collection, Type expectedType): Tests whether all elements in the
specified collection are instances of the expected type
• CollectionAssert.AllItemsAreNotNull(ICollection collection): Tests whether all items in the specified collection are non-null
• CollectionAssert.AllItemsAreUnique(ICollection collection): Tests whether all items in the specified collection are unique
• CollectionAssert.Contains(ICollection collection, object element): Tests whether the specified collection contains the
specified element
• CollectionAssert.DoesNotContain(ICollection collection, object element): Tests whether the specified collection does not
contain the specified element
• CollectionAssert.IsSubsetOf(ICollection subset, ICollection superset): Tests whether one collection is a subset of another
collection
• CollectionAssert.IsNotSubsetOf(ICollection subset, ICollection superset): Tests whether one collection is not a subset of
another collection
 MSTest - Assertion on collections 2
• The usage is pretty clear, but the difference for AreEqual and AreEquivalent is more
subtile
• The ICollection interface is the base interface for classes in the System.Collections
namespace.
• https://docs.microsoft.com/en-us/dotnet/api/system.collections.icollection
[TestMethod] MathOpsMsTests
public void TestCollectionAsserts()
{
CollectionAssert.AreEqual(new[] { 1, 2 }, new[] { 1, 2 }); // success
CollectionAssert.AreEqual(new[] { 1, 2 }, new[] { 1, 2, 3 }); // fail
CollectionAssert.AreEqual(new[] { 1, 2 }, new[] { 2, 1 }); // fail

CollectionAssert.AreEquivalent(new[] { 1, 2 }, new[] { 1, 2 }); // success

CollectionAssert.AreEquivalent(new[] { 1, 2 }, new[] { 1, 2, 3 }); // fail
CollectionAssert.AreEquivalent(new[] { 1, 2 }, new[] { 2, 1 }); // success

CollectionAssert.IsSubsetOf(new[] { 1, 2 }, new[] { 1, 2, 3 }); // success

CollectionAssert.IsNotSubsetOf(new[] { 4, 5 }, new[] { 1, 2, 3 }); // success
}
 When does finally run?
• Try, catch, finally, throw

https://stackoverflow.com/questions/345091/will-code-in-a-finally-statement-
fire-if-i-return-a-value-in-a-try-block/345270 TryCatchFinallyYield
class Program {
public static void Main(string[] args) The output result is:
{
Console.WriteLine("before");
before
Console.WriteLine(Test()); try
Console.WriteLine("after");
} finally
private static string Test()
{
return
try { after
// throw new Exception("error in test()");
Console.WriteLine("try"); or if Exception is thrown
return "return";
}
before
catch (Exception e) { Exception: System.Exception:
Console.WriteLine("e" + e.ToString()); Error in Test() ...
return "catch";
} finally
finally {
Console.WriteLine("finally");
catch
} after
}
}
 MSTest - Test if an exception is thrown
• Assert.ThrowsException<T>(Action action): Tests whether
the code specified by Delegate throws exact given exception
of type T (and not of derived type)
• Assert.ThrowsExceptionAsync<T>(Func<Task> action):
Same as ThrowsException but with async code
• To clarify, both Asserts throws an AssertFailedException
– If tested code does not throw an exception of type T,
or tested code throws an exception of an other type than T
• When debugging tested code – do not break when exception is thrown so test can run or
press continue (may need many continue press)
• https://docs.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.testtools.unittesting.assert.throwsexception
[TestMethod] MathOpsMsTests
public void TestThrowsAsserts1()
{
Assert.ThrowsException<ArgumentNullException>(() => new Regex(null!)); // success
Assert.ThrowsException<ArgumentException>(() => new Regex(null!)); // fail
Assert.ThrowsException<DivideByZeroException>(() => MathHelper.DivideBy(0)); // success
Assert.ThrowsException<DivideByZeroException>(() => MathHelper.DivideBy(1)); // fail
}
 MSTest - Other asserts
• The following assert methods are special

Assert.Fail(string message)

Assert.Inconclusive(string message)
• Fail() immediately change the status of the test to "error" – Always throws an
AssertFailedException

This can be useful to write your own assert tests in test methods
• Inconclusive() indicates that the test could not be completed. In this case, the test is
neither a success nor a failure – Always throws an AssertInconclusiveException
• https://docs.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.testtools.unittesting.assert
[TestMethod] MathOpsMsTests
public void TestOtherAsserts()
{
if (DateTime.Now.Hour != 12)
Assert.Inconclusive("Must be executed at noon");
if (DateTime.Now.Hour != 18)
Assert.Fail("Must be executed at the evening");
}
 MSTest - ExpectedException attribute
• Use the ExpectedException attribute to assert that the correct exception has been thrown. The
attribute causes the test to fail unless an ArgumentOutOfRangeException is thrown
• If the method under test throw a more generic ApplicationException when the debit amount is less
than zero, the test code behaves “correctly” – which means that the test fails
• https://docs.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.testtools.unittesting >
ExpectedExceptionAttribute
[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
BankAccountTests
public void Debit_WhenAmountIsLessThanZero_ShouldThrowArgumentOutOfRange()
{
double beginningBalance = 11.99;
double debitAmount = -100.00;
BankAccount account = new BankAccount("Mr. Bryan Walton", beginningBalance);
account.Debit(debitAmount); // Assert is handled by the ExpectedException attribute on the test method
}

public void Debit(double amount) // BankAccount.cs – production code

{
if (amount < 0) {
// ArgumentOutOfRangeException(string paramName, object actualValue, string message);
throw new ArgumentOutOfRangeException("amount", amount, "Debit amount is less than zero");
}
if (amount > m_balance) {
throw new ArgumentOutOfRangeException("amount", amount, "Debit amount exceeds balance"); // cont. on next slide
 MSTest – Correct ArgumentOutOfRangeException
• Refactor, retest, rewrite, and reanalyze - same Debit() method as before
1) To make sure the correct ArgumentOutOfRange error is thrown we can use try - catch and the
StringAssert.Contains() method which provides the ability to compare two strings
2) If there is a bug in the code and ArgumentOutOfRangeException is not thrown the test passes
To resolve this add a return statement to exit catch when test is correct (we got exception)
3) and an Assert.Fail() if test does not throw an ArgumentOutOfRangeException
// cont. from previous slide
BankAccountTests
}
m_balance -= amount;
}
[TestMethod]
public void Debit_WhenAmountIsGreaterThanBalance_ShouldThrowArgumentOutOfRange()
{
double beginningBalance = 11.99;
double debitAmount = 20.0;
BankAccount account = new BankAccount("Mr. Bryan Walton", beginningBalance);
try {
account.Debit(debitAmount);
}
catch (ArgumentOutOfRangeException e) {
StringAssert.Contains(e.Message, "Debit amount exceeds balance"); // 1)
return; // 2)
}
Assert.Fail("The expected ArgumentOutOfRangeException was not thrown!"); // 3)
}
 NUnit 4 Constraint Model
https://github.com/nunit/docs/wiki/Constraint-Model
• The constraint-based NUnit 4 framework model (the old classic syntax is
not supported) uses a single method of the Assert class for all assertions
• The logic necessary to carry out each assertion is embedded in the
constraint object passed as the second parameter to that method
https://github.com/nunit/nunit-csharp-samples/blob/master/syntax/AssertSyntaxTests.cs
[Test] NUnitFrameworkTest
public void Test_Constraints() {
string phrase = "Hello World!";
// Here's a very simple assert using the constraint model
Assert.That(phrase, Is.EqualTo("Hello World!"));
Assert.That(phrase, Does.Contain("World"));
Assert.That(phrase, Does.Not.Contain("goodbye")); // Only available using new syntax
Assert.That(2 + 2, Is.EqualTo(4));
int[] array = new int[] { 1, 2, 3 };
Assert.That(array, Has.Exactly(1).EqualTo(3));
Assert.That(array, Has.Exactly(2).GreaterThan(1));
Assert.That(array, Has.Exactly(3).LessThan(100));
Assert.AreEqual(5.0d, 4.99d, 0.05d); // old Classic syntax
Assert.AreEqual(5.0f, 4.99f, 0.05f);
Assert.That(4.99d, Is.EqualTo(5.0d).Within(0.05d)); // new Constraint syntax
Assert.That(4.0d, Is.Not.EqualTo(5.0d).Within(0.5d));
Assume.That(true);
Assert.Pass("This will be executed because of the assumption.");
Assume.That(false);
Assert.Fail("This will not be executed because of the assumption.");
}
Comparing xUnit.net to other frameworks 1
= [decorators]
Comparing xUnit.net to other frameworks 2

object nada = null;

// old Classic syntax

Assert.IsNull(nada);

// new Constraint Syntax

Assert.That(nada, Is.Null);

xUnit is written
by the original
inventor of
NUnit

Comparison
from:
https://
xunit.net/
docs/
comparisons
Comparing xUnit.net to other frameworks 3

This comparison table

may have been updated!

ThrowsException<T>
 [Theory]

Data driven tests [InlineData(3)]

[InlineData(5)]
[InlineData(6)]
public void MyFirstTheory(int value)
xUnit

• Parameterized tests with as much {

Assert.True(MathHelper.IsOdd(value));
indata rows that you want }

[Theory]
public static class MathHelper [InlineData(1, 1, 2)]
{ [InlineData(12, 30, 42)]
public static int Add(int a, int b) => a + b; [InlineData(14, 1, 15)]
public static bool IsOdd(int value) public void Test_Add_DataDriven(int a, int b, int expected)
{ {
return value % 2 == 1; Assert.Equal(expected, MathHelper.Add(a, b));
} }
}

[DataTestMethod] [Theory]
[DataRow(3)] MSTest [TestCase(3)]
[DataRow(5)]
[DataRow(6)]
[TestCase(5)]
[TestCase(6)]
NUnit
public void MyFirstTheory(int value) public void MyFirstTheory(int value)
{ {
Assert.IsTrue(MathHelper.IsOdd(value)); Assert.That(MathHelper.IsOdd(value), Is.True);
} }

[DataTestMethod] [Theory]
[DataRow(1, 1, 2)] MathOps and MathOps***Test [TestCase(1, 1, 2)]
[DataRow(12, 30, 42)] [TestCase(12, 30, 42)]
[DataRow(14, 1, 15)] [TestCase(14, 1, 15)]
public void Test_Add_DataDriven(int a, int b, int expected) public void Test_Add_DataDriven(int a, int b, int expected)
{ {
Assert.AreEqual(expected, MathHelper.Add(a, b)); Assert.That(expected, Is.EqualTo(MathHelper.Add(a, b)));
} }
 MSTest/xUnit – DynamicData
• If your data cannot be set into an attribute parameters (non constant values or complex
objects), you can use the [DynamicData] attribute
• This attribute allows to get the values of the parameters from a method or a property
• The method or the property must return an IEnumerable<object[]>
Each row corresponds to the values of a test
• https://www.infoworld.com/article/3122592/my-two-cents-on-the-yield-keyword-in-c.html
[DataTestMethod]
[DataTestMethod] [DynamicData(nameof(Data), DynamicDataSourceType.Property)]
[DynamicData(nameof(GetData), DynamicDataSourceType.Method)] public void Test_Add_DynamicData_Property(int a, int b, int expected)
public void Test_Add_DynamicData_Method(int a, int b, int expected) {
{ var actual = MathHelper.Add(a, b);
var actual = MathHelper.Add(a, b); Assert.AreEqual(expected, actual);
Assert.AreEqual(expected, actual); }
}
yield return public static IEnumerable<object[]> Data
DynamicDataMsTests
public static IEnumerable<object[]> GetData() {
{ get DynamiDataXUnitTests
yield return new object[] { 1, 1, 2 }; {
yield return new object[] { 12, 30, 42 }; yield return new object[] { 1, 1, 2 };
yield return new object[] { 14, 1, 15 }; yield return new object[] { 12, 30, 42 };
} yield return new object[] { 14, 1, 15 };
}
}
MSTest – lifecycle attributes and test log 1
• Initialize/Cleanup by Assembly,
Class and Test
• Some tests may have pre-
conditions and may also need
some cleanup
• For instance, if you need to set a
global configuration, or to delete
some files after a test ran. This
may be more useful for
integration tests than for unit
tests

Also an execution log may be
needed to find more complicated
errors that are hard to debug

Open test log – Ctrl+L

Output can be outputted via
Standard Output or via
TestContext Messages

LifecycleAttributes
 MSTest – lifecycle [ClassInitialize]
public static void ClassInitialize(TestContext testContext)
{

attributes and log 2

_testContext = testContext;
Console.WriteLine("ClassInitialize1");
_testContext.WriteLine("ClassInitialize1");
}
Initialize/Cleanup by Assembly, Class and Test [ClassCleanup]
public static void ClassCleanup()
{
[TestClass]
Console.WriteLine("ClassCleanup1");
public class TestAssemblyInitialize _testContext.WriteLine("ClassCleanup1");
{ }
private static TestContext _testContext; [TestInitialize]
public TestContext TestContext { get; set; } public void TestInitialize()
{
[AssemblyInitialize] Console.WriteLine("TestInitialize1");
public static void AssemblyInitialize(TestContext testContext) _testContext.WriteLine("TestInitialize1");
{ }
_testContext = testContext; [TestCleanup]
Console.WriteLine("AssemblyInitialize"); public void TestCleanup()
_testContext.WriteLine("AssemblyInitialize"); {
Console.WriteLine("TestCleanup1");
}
_testContext.WriteLine("TestCleanup1");
}
[AssemblyCleanup] [TestMethod]
public static void AssemblyCleanup() public void Test1() LifecycleAttributes
{ {
Console.WriteLine("AssemblyCleanup"); Console.WriteLine("Test1");
_testContext.WriteLine("AssemblyCleanup"); _testContext.WriteLine("Test1");
} }
} [TestMethod]
public void Test2()
[TestClass] {
public class TestClass1 Console.WriteLine("Test2");
{ _testContext.WriteLine("Test2");
private static TestContext _testContext; }
public TestContext TestContext { get; set; } }
 MSTest – Create Custom Asserts
• In addition to Assert, StringAssert and CollectionAssert you sometimes want asserts that are suited
for specific tasks. In this case you can create your own assert methods
• Assert validates that a value fills a specific criteria. If the criteria is not satisfied, an exception of
type AssertFailedException is thrown. But any exception can be thrown
• By using the That property we can return a standard instance of one of the built-in ***Assert types.
This allows us to create custom asserts with extension methods that also is more readable
internal static class MyCustomAssert private static string GetMessage(string expected, string actual)
{ {
// used by MyAssertTest1 ... // compares the strings and give difference message
public static void StringEquals(string expected, string actual) }
{ [TestClass]
if (expected == actual) public class MyCustomAssertTest
return; {
[TestMethod]
throw new AssertFailedException(GetMessage(expected, actual)); public void MyAssertTest1()
} {
// extension method used by MyAssertTest2 MyCustomAssert.StringEquals("abc def", "abc eef");
public static void StringEquals(this Assert assert, }
string expected, string actual) [TestMethod]
{ public void MyAssertTest2()
if (expected == actual) MyCustomAssert {
return; // Calls the extended Assert.StringEquals method
Assert.That.StringEquals("abc def", "abc def");
throw new AssertFailedException(GetMessage(expected, actual)); }
} }
 MSTest – Create Custom Exceptions
• In addition to throw various built-in Exceptions and test against you sometimes
want to throw your own Exceptions that are suited for specific tasks
In this case you can extend to your own custom Exception class
• Assert.ThrowsException<T>(Action action): Tests whether the code specified
by Delegate (Action or TestDelegate) throws exact given exception of type T
[Serializable]
public class MyCustomException : Exception [TestClass]
{ public class MyCustomAssertTest
public static void MyCustomException() { } {
[TestMethod]
public static void MyCustomException(string message) : base(message)
public void MyAssertTest3()
{ }
{
public static void MyCustomException(string message, Exception inner) // success
: base(message, inner) { } Assert.ThrowsException<MyCustomException>(() =>
} MyCustomAssert.StringEquals(null!));
internal static class MyCustomAssert
MyCustomAssert
{ // fail
// used by MyAssertTest3 Assert.ThrowsException<MyCustomException>(() =>
public static void StringEquals(string anyString) MyCustomAssert.StringEquals("abc"));
{ }
if (anyString == null) }
throw new MyCustomException("How to throw a custom exception");
}
 MSTest – Custom test execution 1
• By default, the test runner/adapter will execute the methods decorated by [TestMethod]
• Looking into: https://github.com/microsoft/testfx/blob/main/src/TestFramework/TestFramework/
Attributes/TestMethod/ >TestMethodAttribute.cs we see that the class TestMethodAttribute contains
the logic of the test execution
• The method is virtual so we can override this class to customize the way the method is called. For
instance, if we need to execute the method in an STA (Single Thread Apartment model) thread
• The STAThreadAttribute (https://stackoverflow.com/questions/1361033/what-does-stathread-do)
is essentially a requirement
for the Windows message /// <summary> GitHub microsoft / testfx / ...
pump to communicate with /// The test method attribute.
/// </summary>
COM components [AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class TestMethodAttribute : Attribute
• Although core Windows {
/// <summary>
Forms does not use COM, /// Executes a test method.
many components of the /// </summary>
/// <param name="testMethod">The test method to execute.</param>
OS such as system /// <returns>An array of TestResult objects that represent the outcome(s) of the test.</returns>
/// <remarks>Extensions can override this method to customize running a TestMethod.</remarks>
dialogs still use this public virtual TestResult[] Execute(ITestMethod testMethod)
technology {
return new TestResult[] { testMethod.Invoke(null) };
}
}
 MSTest – Custom test execution 2
• We extend/inherit to our own attribute class and override the Execute method to use a STA thread
• We change [TestMethod] to [STATestMethod] for the test since STATestMethod must begin with
the same name as the class. We can do it with TestClassAttribute as well, see example code
namespace MyCustomTestExecuteMethod
{
public class STATestMethodAttribute : TestMethodAttribute
{ MyCustomExecuteTest
public override TestResult[] Execute(ITestMethod testMethod)
{
if (Thread.CurrentThread.GetApartmentState() == ApartmentState.STA)
return Invoke(testMethod);

TestResult[] result = null; [TestClass]

var thread = new Thread(() => result = Invoke(testMethod)); public class TestClass1
{
thread.SetApartmentState(ApartmentState.STA);
[STATestMethod]
thread.Start();
public void Test_STA()
thread.Join();
{
return result;
Assert.AreEqual(ApartmentState.STA,
} Thread.CurrentThread.GetApartmentState());
}
private TestResult[] Invoke(ITestMethod testMethod) }
{ }
return new[] { testMethod.Invoke(null) };
}
}
 MSTest – Execute tests in parallel 1
• By default the MSTest runner/adapter executes
the tests of an assembly sequentially
• If your tests are well isolated you can run them
in parallel to reduce the execution time
• To opt-in for parallel execution, we have to set the [Parallelize] attribute in the assembly
(your code.cs file)
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

• Workers property: The number of threads to run the tests. Set it to 0 to use the number of cores on
your computer
• Scope property: Determine if the runner must parallelize tests on method or class level. MethodLevel
will run all tests in parallel. ClassLevel will run all test classes in parallel, but tests in a class are run
sequentially. You should use ClassLevel if the tests within classes have mutual dependencies
• If you have multiple test assemblies that you want to <?xml version="1.0" encoding="utf-8"?>
<RunSettings>
parallelize in your solution you can create a file named <MSTest>
<Parallelize>
<runsettings>.runsettings <Workers>8</Workers>
<Scope>ClassLevel</Scope>
To set the run settings file, select: Test > Configure Run </Parallelize>
</MSTest>
Settings > Select Solution Wide runsetting File </RunSettings>
 MSTest – Execute tests in parallel 2
• If you want to get the maximum performance, you should use
Workers = 0 and Scope = ExecutionScope.MethodLevel
• A method can contains multiple tests if you use data driven tests.
Each row of a data test is executed sequentially
• To opt-out parallelize for a specific test or class use [DoNotParallelize]
• Test Manager may not report total time correct! It should be 15 + 5 = 20 sec.
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]
namespace MyparallelExecutionTest [TestClass]
{ public class TestClass2
[TestClass] {
public class TestClass1 [TestMethod]
{ [DoNotParallelize]
[TestMethod] // will add extra 5 sec. if ExecutionScope.MethodLevel
public void Test1() => Thread.Sleep(5000); public void Test1() => Thread.Sleep(5000);
[TestMethod]
public void Test2() => Thread.Sleep(5000);
[TestMethod]
public void Test2() => Thread.Sleep(5000);
[DataTestMethod]
[DataRow(5000)] [TestMethod]
[DataRow(5000)] public void Test3() => Thread.Sleep(5000);
[DataRow(5000)] }
public void Test3(int i) => Thread.Sleep(i); }
} MyparallelExecutionTest
 Unit test coverage
• Test > Analyze Code Coverage
Red = Not covered, Blue = Covered

• May need to double

click on a method to
get colored results

https://docs.microsoft.com/en-us/visualstudio/test/using-code-coverage-to-determine-how-much-code-is-being-tested
 Code Coverage Results window
• Coverage results and code coloring are not automatically updated after you
modify your code or when you run tests
• Code coverage is counted in blocks. A block is a piece of code with exactly
one entry and exit point
• Display results in terms of lines by choosing Add/Remove Columns when right
clicking the table
• [ExcludeFromCodeCoverage] results decorator
• Export and import buttons
 Recommended viewing and reading
• Blog post about MSTest v2

https://www.meziantou.net/2018/01/22/mstest-v2-setup-a-test-project-and-run-tests
• Unit Testing Tutorial for C# Developers

https://www.youtube.com/watch?v=HYrXogLj7vg
• Intro to Unit Testing in C# using XUnit

https://www.youtube.com/watch?v=ub3P8c87cwk
• Rövarspråket in CSharp

OneDrive > gmij23 > lectures > code_rovarsprak
• NUnit documentation

https://github.com/nunit/docs
• MSTest documentation

https://github.com/Microsoft/testfx-docs
• xUnit documentation

https://xunit.net/#documentation