Sunday, February 12, 2012

Implementing asynchronous testing

So when I had got my test project set up so it at least ran, I started creating a whole set of tests, most of which just do Assert.Inconclusive("Not yet implemented"). Now I'm going through and implementing them one by one.

Class - ScheduleViewModel contains a private instance of the class Schedule. When I call ScheduleViewModel.Load(), it calls Schedule.Load(), which fires an event on completion and then the method ScheduleViewModel.LoadingComplete() handles that event. Unfortunately,  without just making everything public (or internal, but that's not much cleaner) I can't make my test aware of this event, so that it can wait to assess the results of LoadingComplete()

So what I think I'm doing here, is changing the method ScheduleViewModel.LoadingComplete to catch the event from schedule, and throw another one that will be caught by a new ScheduleViewModel.LoadReallyComplete method. This new event will be visible to the test code, so we can wait on it.

 test method:
            _viewModel = new ScheduleViewModel(pivotView, pivotParam);
             // make sure we wait for the ScheduleLoadingComplete to happen            
            _viewModel.VM_ScheduleLoadingComplete += 
                new EventHandler<ScheduleLoadingEventArgs>(ScheduleCallback);
            // wait for our method to trigger on the event and set this to true
            EnqueueConditional(() => _callbackDone);
            foreach (var slice in _viewModel.EventSlices)
        //event handler helper - triggered by the ScheduleLoading event in the ScheduleViewModel, sets _callbackDone to true
        public void ScheduleCallback(object sender, ScheduleLoadingEventArgs e)
            _callbackDone = true;

class under test:
  public event EventHandler<ScheduleLoadingEventArgs> VM_ScheduleLoadingComplete;
        // this method will be called when the Schedule instance finishes loading. It will then throw its own event
        // that will trigger the LoadPopulatedSchedule() method  
        public void Schedule_ScheduleLoadingComplete(object sender, ScheduleLoadingEventArgs e)
            // throw an event for finished loading
            if (VM_ScheduleLoadingComplete != null)
                VM_ScheduleLoadingComplete(this, e);
        public void LoadPopulatedSchedule(object sender, ScheduleLoadingEventArgs e)
//does a lot of stuff            // Fire Event on UI Thread
            view.Dispatcher.BeginInvoke(() =>

Now that I'd come this far, however, I realised that I actually want to fire my event for the test right at the end of LoadPopulatedSchedule, where I'm currently firing the UI event. So I could get rid of my 'trigger on one event, raise another' method, and add the new event to the end of LoadPopulatedSchedule, right after the UI Dispatcher invoke.(It seems like I should be able to make the test trigger on the UI event, but for now this seems close enough)

Of course, once I had everything triggering correctly on events, it turned out that I actually hadn't initialised the data structure correctly in the first place, and then that I was looking at the wrong set of results, and then that I wasn't correctly clearing my test data so that every subsequent test failed. And once I'd solved all them, I found that the UI method being dispatched was calling a method that referred to variables instantiated in the xaml, so were failing with null references when run under the test harness.

Reading material from this session:
basic stack overflow question

Saturday, February 11, 2012

Beginning to test

For the first two releases, I had only manual tests. This was clearly inadequate - I want to make some major changes to the app, and it's going to be pretty annoying to have to go and walk through all the app functionality after each change. I test software for a living, and I strongly believe in testing early and often - I am in theory a fan of test driven development, but I totally fell over when it came to actually building a new project from scratch. Time to fix that.

I began looking into all the test frameworks available for Windows Phone apps. There are several, and I didn't really know which way was going to be 'best' or 'easiest'. I already had the Silverlight Toolkit installed, and was working in a theoretically MVVM framework, so I looked for approaches designed to work with that. I found (among many others) these guides:
I then walked through the setup outlined in the TDD Kata, as far as the beginning of Test #1. Around this stage, I spent maybe two hours trying to get my classes to implement the interface as described, but getting blocked by the way they all inherited from phone specific classes.. So I moved away from getting 'unit tests' per the usual definition here, and wrote some more general tests - mostly only the simple ones that didn't require me building any fake objects. A few specific points that tripped me up
- A test failure (Assert.Fail or even Assert.Inconclusive) is treated as an exception during debugging, and there appears to be no way to turn this off. If, like me, you just wrote a bunch of tests that fail and you want to debug a specific one, you'll just have to Continue a lot.
- to use the (Tag[""]) functionality, you need "using Microsoft.Silverlight.Testing" and not just "using Microsoft.Silverlight.UnitTesting".

The next week, I realised that I wanted to get my app update published, and I didn't want to publish all the new test code I was writing. To get the update published ASAP, I created a new WindowsPhone Test Project, pulled out all the test code (handily saved into the Test folder for easy extraction) and shipped the existing app with a link to my survey added.

When I launched the new test project, I found that it crashed immediately when it started running the tests - and I couldn't run it in debug mode to figure out why. While looking for tips on how I should have set up this project, I found that the creator of this project, Jake Ginnivan, had done the 'building testable phone objects' work for me in the Windows Phone Essentials toolkit. Eventually I discovered that the problem was the exceptions thrown by failed tests (noted earlier) - so the Test Project was essentially useless to me. It's possible I simply missed the option to run it in debug mode, but my solution was to create a regular Windows Phone App project, and put the tests in there where I could safely run under debug mode. While investigating this I found another tutorial - Catel - Part 8 of n: WP7 Mango and unit testing the camera.

So, the current status of the project is
- need to reimplement my core MVVM framework with testable objects
- need to write actual unit tests around each base class
- I have about half the broader tests passing.Currently I'm working on one that fails because it doesn't wait for the asynchronous object population to complete. I think that's worth a whole new post.

While poking around in all these investigations, I also installed these useful looking dev tools, which I haven't yet explored:

PAX Digital Assistant: up to now

My plan for this blog is to be mostly about my app development work - building, testing, designing, using toolkits and frameworks and so on. I have one app under development and ideas for more. This post gives the background of my current app. 

In the beginning - conception, ambition, scaling back, release
Late 2010, when the SDK for Windows Phone had been released and I was all excited about building an app, I realised that I should build an app for PAX. I began, but didn't get anything done in time for PAX of that year. Then I bought tickets to PAX East 2011, and decided that I would have an app out for that. I looked into MVVM programming, and got pretty much my first taste of actually using an MVC pattern (big props to Laurent Bugnion's toolkit). I was practicing all my (underdeveloped) project management skills, creating user stories, sketching out the interface, etc. Shockingly, I didn't give much thought to testing it (pretty egregious oversight, given that I'm an SDET).

Unsurprisingly, my first ideas were ridiculously ambitious - it should interface with twitter, allowing you to follow all the pax-related accounts, and to tweet, and it would automatically append a #pax tag to any tweets you sent, and you should be able to view the map and zoom in from a suburb level right to a room level, and make notes on the map and share them with your friends in realtime. I handwrote the events into a custom, simple-as-possible xml format, then saved this schedule into the app itself (easy at the time, but meant that a full update was required for any schedule changes). As the dates got closer, I started cutting features. I actually got the app through certification about a week before PAX began, and then had to create an update with the full schedule that made it to the app store the day of PAX East 2011. I was unbelievably proud, and seeing the (mostly positive) reviews and getting a few emails from users was an amazing feeling.

version 2 - poor design bites back
I planned to improve it for PAX Prime, August, but in the end didn't really make any changes - my clunky overall design meant that simply updating all the content to be relevant for Prime was an effort. At PAX Prime I had some new users, but still a few bugs.  In particular, some users complained it was crashing frequently, which I couldn't repro. Eventually one of the users figured out that if he switched his region format from Canada to the USA, it stopped crashing - I thought I'd covered all my bases by randomly testing in region formats like french, two cyrillic languages and an asian language, but no. Immediately after PAX I was still motivated enough to do some work, and I fixed this bug as well as implementing a search feature among all the events.

doing it all right this time, for real
Coming up now we have PAX East 2012. I am in the process of redesigning the app under a test driven framework, so that I can add some major new features like exporting your scheduled events to your calendar, sharing them with friends on an associated website, and updating the event list from a website rather than releasing a new version of the app each time the schedule changes.  I created a Trello board to track what I'm doing - I have mostly been pretty consistent at keeping all my notes and sketches on my ipad (I use NoteTaker HD), but a dedicated task tracker seems like it'll make life easier. I decided that it would be useful to get more feedback from users (related: I'm taking a User Centred Design Certificate) so I wrote a survey, piloted it on three friends who used the app, and then published (well, submitted for certification, so far) an update that adds a link to this survey in the about page and a notice the first time the user opens it asking them to go take the survey. I also have some feedback in email already from users who contacted me through my about page.

Next few posts will probably cover
 - how I'm building the test framework for my app, and problems I've run into where my design isn't test friendly
- installing and beginning to work with the Skydrive API
- setting up the website for the schedule to update from

Wednesday, February 1, 2012

Why this blog?

So there are a lot of reasons I've been planning to start blogging. A short summary:
1. for my records, like a journal. But why not just keep a journal?
2. the knowledge that others could read this will encourage me to organise my thoughts - similar to the teddy-bear solution, simply arranging my ideas for the consumption of others should make them clearer to me.
3. I think I can share information that will be useful to others, mostly around windows phone and my app development process.