Framework Profiles in .NET

There are a lot of different .NET runtimes. Most developers are aware of the desktop framework, Silverlight, and Windows Phone. WinRT/Metro is also coming up shortly. You can download targeting packs to target different frameworks.

Every once in a while (usually while working with NuGet), I find myself needing a refresher on the frameworks and profiles. It's a pain to look all this up again, so I'm collecting it here for future reference.

FrameworkName and Version

A target framework is indicated by a FrameworkName, which has three components: a required framework Identifier, a required framework Version, and an optional framework Profile.

Both Identifier and Profile are always case-insensitive. The FrameworkName constructor allows some flexibility when it parses strings (and NuGet allows even more flexibility), but the canonical structure is as such: Identifier ",Version=v" Version [ ",Profile=" Profile ].

Note that the Version applies to the Identifier; there is no version on a Profile.

Getting the FrameworkName

If you have NuGet installed, you can type the following into the Package Manager Console window to view the target framework for any project:

$p = Get-Project "MyProjectName"
$p.Properties.Item("TargetFrameworkMoniker").Value

Known Framework Identifiers and Profiles

.NETFramework

The .NETFramework identifier is used for the regular desktop framework. For example, ".NETFramework,Version=v3.5" refers to .NET 3.5. You can also target specific updates, e.g., ".NETFramework,Version=v4.0.2" refers to .NET 4.0 Platform Update 2.

If there is no profile specified, the framework refers to the full profile.

Client specifies the client profile; e.g., ".NETFramework,Version=v4.0,Profile=Client" refers to the .NET 4.0 Client Profile. Note that the client profile is not supported in .NET 4.5.

Historical note: The CompactFramework profile specifies the .NET Compact Framework. This probably should have been its own identifier, but doesn't really matter anymore since Visual Studio 2008 was the last version to support CF directly.

Silverlight

The Silverlight identifier is used for the Silverlight framework. For example, "Silverlight,Version=v4.0" refers to Silverlight 4.

If there is no profile specified, the framework refers to the desktop Silverlight framework.

WindowsPhone specifies the first Windows Phone profile; e.g., "Silverlight,Version=v3.0,Profile=WindowsPhone" refers to Windows Phone 7.0. I believe this profile is only applicable to Silverlight version 3.0.

WindowsPhone71 specifies the newer Windows Phone profile; e.g., "Silverlight,Version=v4.0,Profile=WindowsPhone71" refers to Windows Phone 7.5 (Mango). That's not a typo: "7.5" came from marketing; the internal version numbers are all "7.1". However, some (all?) Microsoft tools will treat WindowsPhone75 just like WindowsPhone71, so you may be able to get away with that. I believe this profile is only applicable to Silverlight version 4.0.

Windows Phone probably should have had its own identifier, so that it could be versioned naturally instead of creating profile names like "WindowsPhone71" that are tied to a specific version of an identifier. Oh, well, too late now.

.NETCore

The .NETCore identifier is used for the new .NET framework for Metro style applications. For example, ".NETCore,Version=v4.5" refers to the upcoming Metro/WinRT framework. Note that the first version of this framework will be 4.5. Also, remember that Metro is different than the desktop .NET 4.5 ".NETFramework,Version=v4.5", which will probably be released at the same time.

At this time, details are still a little sketchy, but if I had to guess: you can use .NETCore for Metro development, including applications that will run on Windows RT (Windows on ARM). .NET 4.5 can be used to develop Windows 8 (Intel) desktop apps.

.NETPortable

The .NETPortable identifier is used for portable libraries. Each portable library may run on a number of different platforms, indicated by a profile named ProfileN. For example, ".NETPortable,Version=v4.0,Profile=Profile1" refers to a portable library that can run on .NET 4.0, Silverlight 4, Windows Phone 7, Metro, or XBox 360.

I don't know what meaning the Version has for the .NETPortable identifier.

The .NETPortable identifier requires a profile. The profiles are listed below (as of the Portable Libraries Beta 2 installation):

Version	Profile	Display Name	Frameworks
4.0	Profile1	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4, Silverlight 4, Windows Phone 7, Xbox 360)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone* (Windows Phone) Xbox,Version=v4.0,Profile=* (Xbox 360)
4.0	Profile2	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4, Silverlight 4, Windows Phone 7)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone* (Windows Phone)
4.0	Profile3	.NET Portable Subset (.NET Framework 4, Silverlight 4)	.NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight)
4.0	Profile4	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4.5, Silverlight 4, Windows Phone 7)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.5,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone* (Windows Phone)
4.0	Profile5	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0,Profile=* (.NET Framework)
4.0	Profile6	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4 Platform Update 3)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0.3,Profile=* (.NET Framework)
4.0	Profile14	.NET Portable Subset (.NET Framework 4, Silverlight 5)	.NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v5.0 (Silverlight)
4.0	Profile18	.NET Portable Subset (.NET Framework 4 Platform Update 3, Silverlight 4)	.NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight)
4.0	Profile19	.NET Portable Subset (.NET Framework 4 Platform Update 3, Silverlight 5)	.NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v5.0 (Silverlight)
4.0	Profile23	.NET Portable Subset (.NET Framework 4.5, Silverlight 4)	.NETFramework,Version=v4.5,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight)
4.0	Profile24	.NET Portable Subset (.NET Framework 4.5, Silverlight 5)	.NETFramework,Version=v4.5,Profile=* (.NET Framework) Silverlight,Version=v5.0 (Silverlight)
4.0	Profile36	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4, Silverlight 4)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight)
4.0	Profile37	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4, Silverlight 5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v5.0 (Silverlight)
4.0	Profile41	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4 Platform Update 3, Silverlight 4)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight)
4.0	Profile42	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4 Platform Update 3, Silverlight 5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v5.0 (Silverlight)
4.0	Profile46	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4.5, Silverlight 4)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.5,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight)
4.0	Profile47	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4.5, Silverlight 5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.5,Profile=* (.NET Framework) Silverlight,Version=v5.0 (Silverlight)
4.0	Profile88	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4, Silverlight 4, Windows Phone 7.5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone7* (Windows Phone)
4.0	Profile95	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4 Platform Update 3, Silverlight 4, Windows Phone 7)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone* (Windows Phone)
4.0	Profile96	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4 Platform Update 3, Silverlight 4, Windows Phone 7.5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone7* (Windows Phone)
4.0	Profile104	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4.5, Silverlight 4, Windows Phone 7.5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.5,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone7* (Windows Phone)
4.0	Profile131	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4 Platform Update 3, Silverlight 4, Windows Phone 7, Xbox 360)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.0.3,Profile=* (.NET Framework) Silverlight,Version=v4.0 (Silverlight) Silverlight,Version=v4.0,Profile=WindowsPhone* (Windows Phone) Xbox,Version=v4.0,Profile=* (Xbox 360)
4.5	Profile7	.NET Portable Subset (.NET for Metro style apps, .NET Framework 4.5)	.NETCore,Version=v4.5,Profile=* (.NET for Metro style apps) .NETFramework,Version=v4.5,Profile=* (.NET Framework)

Xbox and .NETMicroFramework

The Xbox identifier is used for XBox 360 projects. For example, "Xbox,Version=v4.0" refers to the XBox 360 platform. The .NETMicroFramework identifier targets (surprise) the .NET Micro Framework.

And that's all I know about those.

Thinking about Async Tasks

In this post, I'm going to clarify how Tasks are used by async/await. This is a little bit different than Tasks as used by the Task Parallel Library, and it's also a little bit different than awaitables as used by async/await.

Tasks Are a Future

A future is some operation that will complete at some future time.

An async Task is not a thread - not even a tiny one. This is one of the most common conceptual hurdles to working with async. Task != Thread.

Similarly, an async Task is not a delegate.

Some tasks do contain a delegate, and they represent some work to be done on a thread. However, as we saw in the Creating Tasks post, tasks created with TaskCompletionSource<T> have no code or delegate at all!

Tasks Complete Once

A task will complete exactly one time. It can complete successfully or with error (cancellation is treated as a special kind of error).

Because tasks complete only once, they're not ideal for representing streams of data or event subscriptions. We'll deal with stream/subscription scenarios in a later post.

Tasks Support Continuations

A continuation is some code that is attached to a task and executed when that task completes. Tasks have direct support for continuations via the ContinueWith method.

However, you usually do not need to call that method. The await keyword will use task continuations to schedule the remainder of the async method as necessary.

Differences between Async Tasks and TPL Tasks

The Task class was introduced with the Task Parallel Library. The TPL usage of Task is a bit more general than the Async usage of Task. Also, the TPL was designed with fork/join parallelism in mind, and those portions of the Task class API are not used with async tasks.

Under the TPL, the creation of a task and the scheduling of that task may be separate. It is possible to create a Task object and not start it until later. Under Async, every task is already in progress; its operation is started when the Task object is created. Because of this, you may have to call Task.Start on a Task returned from TPL code if you want to await it.

TPL has a concept of parent and child tasks. Async tasks do not use this mechanism. There is a logical hierarchy among async tasks, but they do not use the parent/child relationship provided by the TPL.

Each TPL task may have multiple errors. Even if a task only has one exception, it is wrapped in an AggregateException. Async tasks are only expected to have one error, so the await operator will automatically unwrap the single exception if necessary.

Differences between Async Tasks and Awaitables

An awaitable is a very generic form of background operation. Awaitables support testing for completion (IsCompleted), scheduling continuations (OnCompleted), and retrieving the results of the operation (GetResult).

The await operator uses a well-defined pattern, so it's possible to have some very strange awaitables that do work correctly.

For example, the awaitable returned by the Task.Yield method never returns true for IsCompleted, and its OnCompleted will immediately schedule the completions. So, on the one hand it never completes, but on the other hand it is already completed.

WinRT awaitables are also not quite like Task objects. The most important difference is that WinRT operations do not start their operation immediately. Normally, the WinRT awaitable will start the operation for you when it is used in an await expression. However, this won't work as well if you want to have multiple operations running simultaneously. In this case, you can convert any WinRT awaitable into an async Task by calling the StartAsTask extension method.

Functional Concepts

In conclusion, I'd like to point out that we're witnessing more functional concepts gradually enter our imperative language: both future and continuation are concepts borrowed from functional languages.

This helps explain why async code will gently nudge you into a functional programming style. And I'll say it again: this is natural, and should be embraced.

Reporting Progress from Async Tasks

Today, we'll look at how async methods satisfy a common requirement of background operations: reporting progress.

Progress Reporter Abstraction

When asynchronous methods report progress, they use an abstraction of the "progress reporter" concept: IProgress<in T>. This interface has a single method: void Report(T value). You can't get much simpler than that!

An asynchronous method that wants to report progress just takes an IProgress<T> parameter, with some appropriate type for T. There are two important things to keep in mind:

1. The parameter can be null. This means that no progress reports are needed.
2. IProgress<T>.Report is thread-safe, but asynchronous. In other words, you're "posting" the progress reports to the progress reporter. The progress reporter probably hasn't responded to the progress update by the time your method continues.

That second rule can trip people up - it means you can't modify the progress object after it's passed to Report. It is an error to keep a single "current progress" object, update it, and repeatedly pass it to Report.

To avoid this problem, you should create a new progress object each time you call Report. This is easy if your progress type is a value type (the compiler makes a copy of it for you). Alternatively, you could make your progress type immutable and make your own copies.

This is one small step towards a functional mindset. Async/await will gently nudge you away from OOP and towards functional programming. This is natural and should be embraced.

Progress Reporter Implementation

Now let's look at the "receiving" side of progress reports. The caller of the asynchronous method passes in the progress reporter, so it has complete control of how progress reports are handled.

There is one built-in progress reporter: Progress<T>. You can either pass an Action<T> into the constructor or handle the ProgressChanged event.

One important aspect of this class is that it invokes ProgressChanged (and the Action<T>) in the context in which it was constructed. So it's natural to write UI updates:

public async void StartProcessingButton_Click(object sender, EventArgs e)
{
  // The Progress<T> constructor captures our UI context,
  //  so the lambda will be run on the UI thread.
  var progress = new Progress<int>(percent =>
  {
    textBox1.Text = percent + "%";
  });

  // DoProcessing does most of its work on the thread pool.
  await DoProcessingOnThreadPoolAsync(progress);
  textBox1.Text = "Done!";
}

The context keeps the updates nicely synchronized.

However, this doesn't work as well if there's no context to capture. In this case, Progress<T> uses the thread pool context, and you'll have to deal with these problems:

• Multiple simultaneous updates. Since the event is raised on a thread pool thread, fast updates can cause the same event handlers to run on different thread pool threads at the same time.
• Updates after completion. If a method issues an update just before it completes, the event may be raised on a thread pool thread after the task has been completed!

You do need to be aware of these problems when using Progress<T> without a UI context. We'll cover more advanced progress composition in a later post, and consider solutions to these problems.

Progress Report Exceptions

Progress<T> raises its event within a captured context. However, this event is not wrapped in a Task or anything like that; it is just executed directly. This means that any exceptions from that event's handlers will propagate directly to the context.

In other words, exceptions from Progress<T>.ProgressChanged are treated just like exceptions from other event handlers.

In other words, don't throw exceptions from Progress<T>.ProgressChanged. :)

More Progress Reporter Implementations!

The callback-based Progress<T> is great for general use, but there's no reason you couldn't write your own IProgress<T> that works better with your own code base. Here are some implementations from the AsyncEx library:

• PropertyProgress<T> has a property called Progress and implements INotifyPropertyChanged, so progress updates can update data bindings. This implementation also captures its context just like Progress<T>, which is expected for data binding updates.
• ObserverProgress<T> forwards progress updates to an IObserver<T>, where they can be composed using Rx.
• ProducerProgress<T> and DataflowProgress<T> both place progress updates into containers (an IProducerConsumerCollection<T> or an ITargetBlock<TInput>, respectively).

Defining "Progress"

We've covered a lot about progress reporting without actually saying much about the progress update itself (other than it must be passed by value - so either a value type, or an immutable reference type works best).

The information in this section is not Gospel. It's just a tip from my own (limited) experience dealing with progress updates from async methods. YMMV.

It's natural to think of a progress report as cumulative - the canonical example being "percent complete." However, I recommend a different approach: have incremental progress reports for all reusable code and only convert it to cumulative just before it is displayed to the user.

So an FTP file downloader would report the number of bytes transferred after each write to disk, not the entire number of bytes transferred so far:

public async Task DownloadFileAsync(string fileName, IProgress<int> progress)
{
  using (var fileStream = ...) // Open local file for writing
  using (var ftpStream = ...) // Open FTP stream
  {
    while (true)
    {
      var bytesRead = await ftpStream.ReadAsync(...);
      if (bytesRead == 0)
        return;
      await fileStream.WriteAsync(...);
      if (progress != null)
        progress.Report(bytesRead);
    }
  }
}

Perhaps it's just me, but I find it easier to compose incremental updates like this rather than cumulative ones.

Creating Tasks

Microsoft will give us lots of awaitables in .NET 4.5, but there are some situations where we want to create our own awaitable. Task<T> and Task are the easiest awaitable types to work with in .NET, so today we'll look at different ways awaitable Task objects can be constructed.

All Task objects fall into one of two types: "code" and "event". Code-based tasks have a delegate that they are trying to run. Event-based tasks have no code; they're just waiting for some event to take place.

Tasks as Events

Tasks without code can represent any kind of event. The most common examples are I/O completion events, but event-based tasks can actually wrap any kind of event.

To create an event-based task, use the TaskCompletionSource<TResult> class:

public static Task<int> MyIntegerEventAsync()
{
  TaskCompletionSource<int> tcs = new TaskCompletionSource<int>();

  // Register for the "event".
  //   For example, if this is an I/O operation, start the I/O and register for its completion.

  // When the event fires, it should call:
  //   tcs.TrySetResult(...); // For a successful event.
  // or
  //   tcs.TrySetException(...); // For some error.
  // or
  //   tcs.TrySetCanceled(); // If the event was canceled.

  // ThreadCompletionSource is thread-safe, so you can call these methods from whatever thread you want.

  // Return the Task<int>, which will complete when the event triggers.
  return tcs.Task;
}

Remember that Task<T> and Task are awaitable, so you can await the result of MyIntegerEventAsync even though it's not an async method.

You probably won't have to use TaskCompletionSource<TResult> directly; you can use TaskFactory.FromAsync to wrap any operation that uses IAsyncResult (and most existing asynchronous I/O methods do use IAsyncResult). Internally, FromAsync does use TaskCompletionSource<TResult>.

Nito.AsyncEx includes an AsyncFactory type which works like TaskFactory.FromAsync but is slightly easier to use and supports more arguments. It also includes a (non-generic) TaskCompletionSource, which is easier to use when creating (non-generic) Tasks.

Tasks as Code

Remember (from our async intro post) that the async keyword does not run the method on a background thread. However, you can use Task.Run to run code on a background thread:

public static void MyThreadPoolMethod()
{
  // Do work (assuming we're running on the thread pool).
}

public async Task DoStuffAsync()
{
  var cpuResult = await Task.Run(MyThreadPoolMethod);

  // Use cpuResult...
}

Task.Run will take a delegate and run it on the thread pool for you. It wraps the delegate into a Task<T> or Task, and the Task wrapper takes care of all the error handling and other stuff correctly.

What about other threads? What if you don't want to run your code on the thread pool?

If you have a particular context in which you want to run your code, you can use a TaskFactory initialized with a TaskScheduler that targets your context. You can then call TaskFactory.StartNew to run your code in that context. In fact, Task.Run is just a shorthand for Task.Factory.StartNew.

It's possible to capture the current synchronization context into a TaskScheduler by calling TaskScheduler.FromCurrentSynchronizationContext, and then use it later (e.g., from a background thread) to run code within that context. Normally there are better ways to do this, but it is an option.

Before async/await, this was a good way to send progress reports from a background task to the UI. With async/await, there is now a better way.

Writing your own TaskScheduler is possible, but frustrating due to the lack of MSDN documentation. Fortunately, it's almost never necessary.

If you need a specific thread (e.g., an STA thread) that supports TaskScheduler, you can use the AsyncContextThread type in the Nito.AsyncEx library.

There are some pretty cool tricks we can pull off using the built-in TaskScheduler/TaskFactory types. We'll look at more advanced TaskScheduler situations in later posts.

Tasks as Async Methods

This is a special case of code-based Tasks - and it's easy to overlook!

The flat-out easiest way to create Task objects is to write an asynchronous method:

public async Task<int> DivideAsync(int numerator, int denominator)
{
  await Task.Delay(100);
  return numerator / denominator;
}

We do not create a Task<int> in our code, but the compiler rewrites our code so that a Task<int> is created and returned. When the method completes, the task completes.

That's the easiest way to create Task objects! However, it only works if you're building on existing awaitables; if you aren't in this situation, then you should use TaskFactory.StartNew or TaskCompletionSource<TResult>.

Async Unit Tests, Part 2: The Right Way

Last time, we looked at incorrect approaches to async unit testing. We also identified the underlying problem: that unit tests do not have an appropriate async context.

At this point, the solution should be pretty obvious: give the unit tests an async context!

It really is that easy! Why, all you have to do is write your own SynchronizationContext implementation. Keep in mind that thread-safety is paramount, because the methods under test may interact with the thread pool or other async contexts. Note that async void methods interact with SynchronizationContext in a different way than other async methods. Oh, and also remember that exceptions need special handling in some cases so their original call stack is preserved appropriately, and if you're on VS2010 you'll need to hack this in because there's no support for it on .NET 4.0.

Just kidding! Ha, ha! The good folks on the Async team have done all the hard work for you. :)

Right Way #1: The Official Approach

If you have the Async CTP installed, then check out the "My Documents\Microsoft Visual Studio Async CTP\Samples\(C# Testing) Unit Testing\AsyncTestUtilities" folder. You'll find not just one, but three async-compatible contexts, ready for you to use!

You should use GeneralThreadAffineContext unless you absolutely need another one. To use it, just copy AsyncTestUtilities.cs, CaptureAndRestorer.cs, and GeneralThreadAffineContext.cs into your test project.

Then, take each unit test and re-write it so that it has a context:

[TestMethod]
public void FourDividedByTwoIsTwo()
{
  GeneralThreadAffineContext.Run(async () =>
  {
    int result = await MyClass.Divide(4, 2);
    Assert.AreEqual(2, result);
  });
}

[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public void DenominatorIsZeroThrowsDivideByZero()
{
  GeneralThreadAffineContext.Run(async () =>
  {
    await MyClass.Divide(4, 0);
  });
}

Our unit test methods are not async. Each one sets up an async context and passes the actual test into it as an async lambda expression. So, the actual test code can still be written with all the benefits of async/await, and the async context takes care of making sure it runs as expected:

Just as importantly, the async context ensures that tests that should fail, will fail:

[TestMethod]
public void FourDividedByTwoIsThirteen_ShouldFail()
{
  GeneralThreadAffineContext.Run(async () =>
  {
    int result = await MyClass.Divide(4, 2);
    Assert.AreEqual(13, result);
  });
}

And everyone lived happily ever after!

Well, sort of. This solution does work, but it's a bit cumbersome. Copying code files into each test project? Modifying every unit test to set up its own async context? Really?

Right Way #2: Now with Less Effort!

Boy, if only there was some way to have the MSTest framework apply the async context for us, then we could just write async unit test methods and not worry about it!

Oh yeah - there is. Visual Studio allows you to define a custom "test type." It really is that easy! Why, all you have to do is... ah, forget it. A custom "async unit test" type is already available:

• Install the AsyncUnitTests-MSTest NuGet package into your test project.
• Add a using Nito.AsyncEx.UnitTests; line.
• Change your [TestClass] attribute to [AsyncTestClass].

Sweet.

Now you can write async unit tests the obvious way:

[TestMethod]
public async void FourDividedByTwoIsTwoAsync()
{
  int result = await MyClass.Divide(4, 2);
  Assert.AreEqual(2, result);
}

[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public async void DenominatorIsZeroThrowsDivideByZeroAsync()
{
  await MyClass.Divide(4, 0);
}

And it works:

And test failures written the obvious way actually fail:

[TestMethod]
public async void FourDividedByTwoIsThirteenAsync_ShouldFail()
{
  int result = await MyClass.Divide(4, 2);
  Assert.AreEqual(13, result);
}

Sniff... It's... so... beautiful...

But not quite perfect. You still have to add a NuGet package and remember to change [TestClass] to [AsyncTestClass].

Tip: You can download an Async Unit Test item type which uses [AsyncTestClass] instead of [TestClass]. This makes writing new async tests just a little bit easier, but not entirely foolproof.

Future Directions

xUnit.NET has recently released first-class support for asynchronous unit tests: in version 1.9 (2012-01-02) and newer, for any test method returning Task/Task<T>, the test framework will wait until the task completes before declaring success/failure. However, as of now, it does not support async void unit tests; this is planned for a future release.

I've been in contact with some people inside of Microsoft regarding this issue, and they said they're aware of it and are considering various options. They wouldn't give me any details, of course, but they did suggest that I would be "pleasantly surprised" when Visual Studio vNext comes out.

So, that's where we are today. Hopefully Microsoft will ship built-in async unit test support in Visual Studio vNext, and I'll be able to look back at this blog post and laugh at how fraught with peril async unit testing used to be.

Async Unit Tests, Part 1: The Wrong Way

"Code without tests does not exist."

(Overheard at CodeMash)

The core meaning of this quote is that code without unit tests is not as useful as code with unit tests. The speaker even goes so far as to say he won't use code without tests.

I don't take a position quite this extreme, but I definitely agree with the underlying sentiment: that code with unit tests is far more useful. Unit tests prove correct functionality (at least for a limited set of cases). Unit tests also provide a sort of documentation.

If you don't write unit tests - or if you or your manager think writing tests just delays software development - then I refer you to the best computer book ever written, Code Complete. In that book, Steve McConnell presents some very interesting hard facts about testing.

I hope we can all agree that unit testing is a fundamental skill in Modern Programming. And this brings me to a sad chapter in async/await support: the "obvious" way to do unit tests is wrong.

Let's start with a simple asynchronous method. So simple, in fact, that it will just pretend to work for a while and then do a single integer division:

public static class MyClass
{
  public static async Task<int> Divide(int numerator, int denominator)
  {
    // Work for a while...
    await Task.Delay(10); // (Use TaskEx.Delay on VS2010)

    // Return the result
    return numerator / denominator;
  }
}

Boy, it doesn't seem that there can be much wrong with that code! But as we'll see, there's a lot that can be wrong with the unit tests...

When developers write unit tests for async code, they usually take one of two mistaken approaches (with the second one being what I call "obvious"). We'll look at each of these approaches in this post and examine why they're wrong, and look at solutions next time.

Wrong Way #1: Using Task.Wait and Task.Result

This mistake is most common for people new to async: they decide to wait for the task to complete and then check its result. Well, that seems logical enough, and some unit tests written this way actually work:

[TestMethod]
public void FourDividedByTwoIsTwo()
{
  Task<int> task = MyClass.Divide(4, 2);
  task.Wait();
  Assert.AreEqual(2, task.Result);
}

But one of the problems with this approach is unit tests that check error handling:

[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public void DenominatorIsZeroThrowsDivideByZero()
{
  Task<int> task = MyClass.Divide(4, 0);
  task.Wait();
}

This unit test is failing, even though the async method under test is throwing a DivideByZeroException. The Test Results Details explains why:

The Task class is wrapping our exception into an AggregateException. This is why the Task.Wait and Task.Result members should not be used with new async code (see the end of last week's async intro post).

Well, we could await the task, which would unwrap the exception for us. This would require our test method to be async. Congratulations, you can move on to the next section.

Wrong Way #2: Using Async Test Methods

This mistake is more common for people who have used async in some real-world code. They've observed how async "grows" through the code base, and so it's natural to extend async to the test methods. This is what I consider the "obvious" solution:

[TestMethod]
public async void FourDividedByTwoIsTwoAsync()
{
  int result = await MyClass.Divide(4, 2);
  Assert.AreEqual(2, result);
}

Yay! It works!

...

Wait...

[TestMethod]
public async void FourDividedByTwoIsThirteenAsync()
{
  int result = await MyClass.Divide(4, 2);
  Assert.AreEqual(13, result);
}

Um, that test should certainly not be passing! What is going on here???

We've encountered a situation very similar to async in Console programs: there is no async context provided for unit tests, so they're just using the thread pool context. This means that when we await our method under test, then our async test method returns to its caller (the unit test framework), and the remainder of the async test method - including the Assert - is scheduled to run on the thread pool. When the unit test framework sees the test method return (without an exception), then it marks the method as "Passed". Eventually, the Assert will fail on the thread pool.

There is now a race condition. There's no race condition in the test itself; it will always pass (incorrectly). The race condition is when the assertion fires. If the assertion fires after the unit test framework finishes the test run, then you'll see a successful test run (like the last screenshot). But if the assertion fires before the unit test framework finishes the test run, then you'll see something like this:

Clicking on the link shows that the Assertion is indeed failing on the thread pool, some time after the test is considered completed and "Passed":

Next Time: The Right Way

During CodeMash, I gave a lightning talk about async unit testing. You could almost hear the teeth grinding at this point, when the TDD/BDD fans discovered that async unit tests were essentially broken. But do not give up hope!

Tomorrow we'll look at the right way to do async unit testing. In fact, we'll look at two right ways: the "official" way, and a brand-new way that's even better!

Async Console Programs

Once you start using asynchronous code, it kind of "grows" through your codebase. It's easier for asynchronous code to work with other asynchronous code, so it's natural to start making everything asynchronous.

If you're writing a console program, you may end up wanting an asynchronous main method, like this:

class Program
{
  static async void Main(string[] args)
  {
    ...
  }
}

Unfortunately, that doesn't work (and in fact, the Visual Studio 11 compiler will reject an async Main method). Remember from our intro post that an async method will return to its caller before it is complete. This works perfectly in UI applications (the method just returns to the UI event loop) and ASP.NET applications (the method returns off the thread but keeps the request alive). It doesn't work out so well for Console programs: Main returns to the OS - so your program exits.

You can work around this by providing your own async-compatible context. AsyncContext is a general-purpose context that can be used to enable an asynchronous MainAsync:

class Program
{
  static void Main(string[] args)
  {
    AsyncContext.Run(() => MainAsync(args));
  }

  static async void MainAsync(string[] args)
  {
    ...
  }
}

Update (2012-02-16): AsyncContext will propagate exceptions, and the RunTask method can handle tasks with return values. So a more realistic console example would be:

class Program
{
  static int Main(string[] args)
  {
    try
    {
      return AsyncContext.RunTask(() => MainAsync(args)).Result;
    }
    catch (Exception ex)
    {
      Console.Error.WriteLine(ex);
      return -1;
    }
  }

  static async Task<int> MainAsync(string[] args)
  {
    ...
  }
}

Note: normally, you do not want to call Task.Result in async/await code; you want to await the task instead. The example above is an exception to this rule: RunTask will propagate the task exceptions, so any task returned from RunTask completed successfully, so it's safe to call Result.

That's all for today; next week we'll start looking at asynchronous unit tests, which suffer from a similar problem.