Beginning to test

For the first two releases, I had only manual tests. This was clearly inadequate - I want to make some major changes to the app, and it's going to be pretty annoying to have to go and walk through all the app functionality after each change. I test software for a living, and I strongly believe in testing early and often - I am in theory a fan of test driven development, but I totally fell over when it came to actually building a new project from scratch. Time to fix that.

I began looking into all the test frameworks available for Windows Phone apps. There are several, and I didn't really know which way was going to be 'best' or 'easiest'. I already had the Silverlight Toolkit installed, and was working in a theoretically MVVM framework, so I looked for approaches designed to work with that. I found (among many others) these guides:

• http://stackoverflow.com/questions/5994672/state-of-unit-testing-for-windows-phone
• http://www.diaryofaninja.com/blog/2012/01/23/8-musthave-tools-for-windows-phone-7-development 
• 
  http://codingsolutions.blogspot.com/2010/03/windows-phone-7-tdd-kata-using-mvvm-and.html
• http://smartypantscoding.com/a-cheat-sheet-for-unit-testing-silverlight-apps-on-windows-phone-7

I then walked through the setup outlined in the TDD Kata, as far as the beginning of Test #1. Around this stage, I spent maybe two hours trying to get my classes to implement the interface as described, but getting blocked by the way they all inherited from phone specific classes.. So I moved away from getting 'unit tests' per the usual definition here, and wrote some more general tests - mostly only the simple ones that didn't require me building any fake objects. A few specific points that tripped me up
- A test failure (Assert.Fail or even Assert.Inconclusive) is treated as an exception during debugging, and there appears to be no way to turn this off. If, like me, you just wrote a bunch of tests that fail and you want to debug a specific one, you'll just have to Continue a lot.
- to use the (Tag[""]) functionality, you need "using Microsoft.Silverlight.Testing" and not just "using Microsoft.Silverlight.UnitTesting".

The next week, I realised that I wanted to get my app update published, and I didn't want to publish all the new test code I was writing. To get the update published ASAP, I created a new WindowsPhone Test Project, pulled out all the test code (handily saved into the Test folder for easy extraction) and shipped the existing app with a link to my survey added.

When I launched the new test project, I found that it crashed immediately when it started running the tests - and I couldn't run it in debug mode to figure out why. While looking for tips on how I should have set up this project, I found that the creator of this project, Jake Ginnivan, had done the 'building testable phone objects' work for me in the Windows Phone Essentials toolkit. Eventually I discovered that the problem was the exceptions thrown by failed tests (noted earlier) - so the Test Project was essentially useless to me. It's possible I simply missed the option to run it in debug mode, but my solution was to create a regular Windows Phone App project, and put the tests in there where I could safely run under debug mode. While investigating this I found another tutorial - Catel - Part 8 of n: WP7 Mango and unit testing the camera.

So, the current status of the project is
- need to reimplement my core MVVM framework with testable objects
- need to write actual unit tests around each base class
- I have about half the broader tests passing.Currently I'm working on one that fails because it doesn't wait for the asynchronous object population to complete. I think that's worth a whole new post.

While poking around in all these investigations, I also installed these useful looking dev tools, which I haven't yet explored:

• PortableClass Library test project
• IsolatedStorage Explorer
• Metro grid helper