Blog

Developing applications that need asynchronous execution is usually much more complicated then ones do not need it. Unfortunately, automated testing of such applications became even harder. Some time ago I experienced difficulties with testing asynchronous Google App Engine service that uses Task Queue. This time it’s WPF desktop application that performs most of its job in background PTL (Parallel Task Library) tasks. Disregarding not-important details I can describe it in the following way:

1. NET Framework 4.0 desktop application
2. UI made with WPF (it’s optional, may be created by controller at some point, binded to model and then disposed)
3. Model detached from the UI. Parts of Model are DependencyObject and ObservableCollection when it’s needed.
4. Main Application Controller starts in the main thread and runs all other stuff as PTL Tasks.
5. Main Controller creates, maintains and synchronizes Model with server-side and file system - all is asynchronous and may be performing at the same time.

How was I going to test it?

1. NUnit
2. Each test creates an instance of Main Controller, mocking required part of services
3. Then calling Main Controller methods (or it’s sub-controllers) to perform some part of business logic.Most of methods of Main Controller is asynchronous and exits
4. immediately returning Task (or Task[]) being executed
5. Test initiates one or few of such tasks and then it is supposed to wait until all of them  are done and verify result.

Everything was fine in a theory, but the problem is that if you want your model to be accesible in many threads which are different from the main one you need to write code like this in C#:

   1:  public static readonly DependencyProperty MyPropProperty = DependencyProperty.Register("MyProperty", typeof(bool), typeof(MyModel));

   2:   

   3:  public bool MyProp

   4:  {

   5:             get

   6:             {

   7:                         (bool) (Dispatcher.CheckAccess()

   8:                                  ? GetValue(MyPropProperty)

   9:                                  : Dispatcher.Invoke((Func<DependencyProperty, object>) GetValue, MyPropProperty));

  10:             }

  11:             set

  12:             {

  13:                     if (Dispatcher.CheckAccess())

  14:                             SetValue(MyPropProperty, value);

  15:                     else

  16:                             Dispatcher.Invoke((Action<DependencyProperty, object>) SetValue, MyPropProperty, value);                   

  17:             }

  18:   }

The code is pretty strait-forward and it works good in application, but when I tried to run it under NUnit test Dispatcher.Invoke just hangs because there was no synchronization context set.

I hate any kind of IF-WE-ARE-UNDER-TEST condition so I wondered is there simple solution to be able to run controller that uses such a model. And naturally the solution should provide ability to wait while Tasks are executing to verify result of them.

At this point the only acceptable idea I got is to create DispatcherFrame before each test in the testing thread (thread the Main Controller and Model is creating in) and provide a method that pushes the frame to the testing thread dispatcher and then waits while certain tasks are done. The test method should create and initialize controller, run it’s methods getting the running Tasks as a result (actually all tasks will be paused as soon as their touch Dispatcher and they will be waiting for dispatcher frame for infinite period of time), wait while they are performing with the help of the method described above and check the result. The code looks like this in C#:

   1:  private DispatcherFrame _dispatcherFrame;

   2:   

   3:  [SetUp]

   4:  public void BeforeTest()

   5:  {

   6:   

   7:      _dispatcherFrame = new DispatcherFrame();

   8:  }

   9:   

  10:  private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(5);

  11:   

  12:  protected void WaitFor(params Task[] tasks)

  13:  {

  14:      bool timedOut = false;

  15:      _dispatcherFrame.Continue = true;

  16:   

  17:      new Task(() =>

  18:          {

  19:              timedOut = !Task.WaitAll(tasks, WaitTimeout);

  20:                 _dispatcherFrame.Continue = false;

  21:          }).Start();

  22:   

  23:          Dispatcher.PushFrame(_dispatcherFrame);

  24:   

  25:      if (timedOut)

  26:          throw new InvalidOperationException("Waiting timeout");

  27:  }

  28:   

  29:  [Test]

  30:  public void MyTest()

  31:  {

  32:   

  33:      var mocks = /* configure mocks */

  34:   

  35:          using (controller = new MainController(mocks))

  36:          {

  37:              Task[] firstBunchOfTasks =

  38:                             new Task[]

  39:                         {

  40:                             controller.StartFoo(),

  41:                             controller.StartDoo()

  42:                         };

  43:   

  44:                 WaitFor(firstBunchOfTasks);

  45:   

  46:                     // assert something at this point

  47:   

  48:                 Task someOtherJob = controller.StartJob();

  49:   

  50:                 Task fooAgain = controller.StartFoo();

  51:   

  52:                     WaitFor(someOtherJob, fooAgain);

  53:   

  54:                 // verify final result

  55:             }

  56:  }

Such solution works fine for me but if you know more elegant approach or think it can be done in a completely different way, please share you thoughts with me.

Just tried free brainbench test on C# 4.0. I dislike some questions they asked but I like result I reached - 100% correct answers:)

Here is the transcript

It is generally agreed that automated testing is extremely important for software development process. It is so, despite the actual product’s complexity. I personally think that the primary reason of automated tests usage is possibility  to make changes to your code and quickly check whether the very basic business workflow still works. And because of that I dislike unit testing which I think makes sense not so much often and only to verify some of isolated components with complicated logic. Putting effort into unit-testing 50-60-70-80-90% coverage and even more of application code is in vain  - it takes too much time and it causes a problem it’s supposed to solve - when you have so much percentage of unit-tested code you are not able to make changes to your software quickly, 5 minutes refactoring causes 1 hour of fixing tests.

When I talk about automated tests I usually mean integration tests - the ones which could be implemented either as unit tests or using any of UI clicking tool (the last one is usually harder to maintain).

While working at Emails to RSS Forwarding service I faced the problem that app engine local testing environment does not support Task Queue but the service itself uses Task Queue very extensively for processing, archiving, RSS generation and maintenance tasks. Well, I exaggerated a bit, actually Google App Engine does support local testing using Task Queue - but in a very limited way at this point - the only thing it allows to schedule asynchronous task during testing and to verify that the task is added to particular queue and nothing more. If business workflows of your application depend on asynchronous jobs as in my case, possibility to run them under jUnit is necessary for successful integration testing. To resolve that problem in a way GAE considers it as legal I implemented the LocalTaskQueueCallback to run the right servlet with the right parameters. The code of it can be downloaded from here TaskQueueRunner.java To use this task queue runner in your application put the TaskQueueRunner class as a parameter to LocalTaskQueueTestConfig.setCallbackClass method before you do LocalServiceTestHelper.setUp().

If you choose to disable automatically asynchronous tasks running to be able to manually check that everything is scheduled correctly and then run the async jobs TaskQueueRunner provides two overloads of processTasks static method.

We usually do not realize the amount of information we get every day, every hour, every minute. Me personally didn't really take it into consideration until recently when I returned back from one-week vacation :) Yes during it I didn't have any access to the Internet :) And to be honest, I do not have such vacations often. The other fact I learned from the HUGE amount of emails, rss updates, social network messages and etc was that such information capacity is much higher than it was 2 years ago and enormously higher than 5 year ago.

Beside this, among other questions I asked myself at that time was the following: why is so much information (like just notifications,  google groups posts, social networks notifications, software updates, repositories commits, new twitter followers and etc...) delivered via emails?

At least me personally want it to be delivered via RSS because of the following reasons:
1. RSS distracts me less from most important activities
2. I would like to use email for communication only
3. I would like communication messages be more distracting than notifications about new repository commits or new friend on facebook, or new discussion posted on google groups.

Actually I would like to have ability to assign different levels of distraction for different information sources, and even more I want ability to configure conditions for each information source to setlevel of distraction, and even more I would like to have ability to switch between different rule sets based on what kind of activity I am busy right now. But that's just a dream, or maybe I'll do it someday, at least I would let you know if I made any progress on it. Do you think it makes sense of having such tool/service? If so what do you personally want it to do? As far as I do not clearly understand how such system would look like yet and would it be pretty complicated to integrate it into existing information flows, I decided to implement pretty stupid service to just cleanup my inbox from all kinds of "spam" I mentioned above and maybe at some point I will extend it to cover all of my expectations.

The service does very simple job: receives emails forwarded to the address (es) it provides for you and aggregates these email messages into generated RSS feed(s). So I have configured forwarding rules on my gmail account to send it to the service and remove from my inbox. Can’t say it has  solved all my problems with  lots of information but now my inbox is clean and I am able to review all the updates from time to time with Google Reader; so it makes me a bit happier than before:)

Here is the service's address http://emails2rss.appspot.com so you can try it. If you find it helpful or think it sucks or you do have another idea how to deal with notifications and updates let me know.

To sum up my post - it's very helpful to go on vacation and very serviceable by the way :)