Technology

If unit testing is the cake, functional testing is just the icing

I’ve heard a lot of different view points on software testing over the years. Especially when it comes to unit testing. Some people swear by it and treat code coverage metrics like it’s their greatest achievement in life (maybe it is). Others think it’s an unnecessary burden. I’ve heard arguments like “how can you refactor code if you have to update all of your tests each time” and “how can you refactor code if you don’t have sufficient unit test coverage” or “you’re flying by the seat of your pants without good coverage” vs “your tests are just a burden, you’re slowing me down!”.

I’d like to give a view from the trenches as it were. Albeit a view my trench working on a REST web service.

Some background

Before I begin I’d like to clarify what I mean by “unit test” and “functional test”.

  1. Unit testing – This is breaking down the application code into individual units that can be tested in isolation at a code level.
  2. Functional testing – This is more of a higher level, black-box test (normally from a user point of view) where the test doesn’t care about how the application functions. In this context functional are REST API calls simulating user behaviours and flows.

Over the past 18 months we’ve tried to balance our approach to testing. To do this we have taken the view that the end user is the most important piece of the puzzle. So we should focus our testing efforts on user scenarios. This means more functional tests, and less focus on unit testing.

The problem

For quite a while this approach worked well. We have suits of functional tests that make REST API calls to our web service and check the responses coming back. The tests walk through specific scenarios (creating user accounts, configuring the state of user accounts, testing scenarios with these accounts etc) and assert that the system behaves as expected. If something breaks (like you change the location of a field, or something that was an integer in the response is now a string, then the test fails, or a field is removed from the response.

Fast forward 18 months. We now have several hundred of these tests. They run every time code in the “dev” branch is updated (for us that means every time a feature branch is merged in and the code is pushed to our test environment). “That’s good right?” Well, here are some down-sides we’ve seen

  1. Tests are slow! – These kind of functional tests aren’t quick to execute, they are testing real systems with complex logic behind them.
  2. They are dependent on things I don’t care about – Should I really care if our user account team has a server down in our test environment? Should it block my work? What if the storage platform is down? In this instance if my test can’t create a user account then it fails and I have to investigate why.
  3. They test my colleagues work – Remember when I said these tests run every time code is merged in from feature branches. That means that not only are these codes testing my work, but they are testing other peoples work too. If I’m testing that value A = B and someone else is working on a feature where A = B+C then we have a race to see who gets there first with both code changes and test changes (granted this could be avoided by managing the work but it’s not always so simple in the real world).
  4. They are slow! – I’ve said it twice because it’s important. When testing changes I want to be confident that my changes work as expected, but I also want to be sure that I haven’t broken anything. Functional tests often run late in the development process (they need to run on a test environment after the code is built & deployed) which means you get feedback late too. If you did break something, finding out at this stage normally means you’ve broken your service / app for someone else in the testing environment.

The solution

I honestly think that both unit testing and functional testing have their place in web app development. It’s critical to test your application from the inside (via unit tests) and the outside (via functional tests). After all, your users are the most important thing so you have to test that they get the behaviour they expect.

However, a lot of people ignore unit testing, or at least treat it like second class citizen. Maybe it’s because they associate it with TDD or other approaches they dislike. Maybe it’s a lack of knowledge of how to write testable code (dependency injection, inversion of control, TDD all help with this).

Personally, I think that unit testing should be the primary focus of your testing efforts. Why?

  1. Unit tests are fast (well, if written properly) – Yes, it’s possible to write slow unit tests, but, if you’re breaking your code down into small individual units and writing testable code in the first place, then the tests should be quick to execute.
  2. Feedback is isolated- Unit tests run against my local changes. So if a test breaks I know it’s almost certainly because I’ve broken something and I need to fix it. No need to go on a wild goose chase only to find out its someone else’s change.
  3. You can refacor with confidence – I can look at code coverage reports and other metrics and assess if the code I’m going to refactor has test coverage and then make changes with a higher degree of confidence. If something breaks I get feedback straight away.
  4. They enforce good patterns and practices – Just like it’s possible to write a slow test, it’s also possible to write a monolithic test to test monolithic code. Unit testing makes you think about how you’re going to test your code (like how can I test what happens if dependency X throws an exception). Unit testing nudges you towards practices like dependency injection, inversion of control (etc) all of which makes code easier to maintain and understand. Everyone wins.

Having a solid set of unit tests means you catch issues early and can refactor with confidence, leaving the functional tests there as a safety net rather than first line of defence. Like I said in the title, unit tests should really be the cake, and functional tests are just the icing.

Progress on the Kingsgate Media Player

Just a quick update on the progress of the Kingsgate media player. Lots of things have been completed, the main feature being that video playback is now supported! Here’s a video showing the app in action on my Amazon FireTV.

There have been a lot of other internal changes too, including:

  1. Implemeted dependency injection using Dagger 2
  2. Added automated builds using Travis-CI
  3. Added new HttpClient, HttpRequest and HttpResponse object using DI

Lots more to come!

Solving a problem, with technology

I’ve been attending a fantastic church, Kingsgate, for over a year now. The level of technology that the church uses is excellent, but there is one problem.

Podcasts.

For ages podcasts have been a pain to use. Some platforms make it easy, like iOS who have a dedicated podcast app where you can search for Kingsgate and you see the podcast. It’s not a bad experience but it’s not great either. For one, you only get the audio feed not the video feed. The logo is also missing and it just looks pretty crappy.

My main issue is that I want to watch the video of the sermons from Church not just listen to the audio. I have a fairly nice TV and I can stream a tonne of great programs from Netflix and Amazon Prime, and BBC iPlayer. So why is it so hard to watch the sermons from church on my TV?

I know there are RSS apps for a lot of different platforms (even the Amazon Fire TV) but I’m not really the target audience. What about my parents, how can they watch the podcast of the sermons on their TV:

  1. Go to the app store of choice
  2. Search for RSS (no “dar-ess”, the letters “R”,”S”,”S”)
  3. Now install one (if you’re lucky there is only one and you don’t have to make a choice)
  4. Now go to settings, and remove any default subscriptions (CNN, BBC etc)
  5. Click add subscription
  6. Now type “http://”
  7. Give up

There must be a better way…

I have an Amazon Fire TV which is essentially and Android box with a better interface. So I’ve decided to create an app that lets you easily access the sermon podcasts on the Amazon Fire TV.

12661840_921537694627093_5182743669736874803_n.jpg

Right now this is totally unofficial, and in no way linked to Kingsgate. The app is currently fetching a list of available sermons. The next steps are:

  1. Full-screen media playback
  2. Keeping track of played / un-played sermons
  3. Porting to iOS (iPad, Apple TV) and Windows 10 (desktop, tablet, windows phone and Xbox One)

All of the code for this is on GitHub so if you want to help out then feel free to fork the repo and send a pull request!

“Could not load file or assembly” in Azure Web Role

Every now and again you hit what seems like a simple problem only to dig deeper and find it’s not as straight forward as first thought…Today was one such time. A simple upgrade of an Azure application to the latest version of the Azure SDK (2.8 at the time of writing this). Everything is updated, compiled, tested, and running locally. Then deployed to Azure and… bang!

image

The full error reads:

Could not load file or assembly ‘Microsoft.WindowsAzure.Diagnostics, Version=2.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’

Surely this should have affected the app when running locally too, and even if it didn’t it’s simple to fix. Just add a binding redirect (in this case in my web.config file).

<dependentAssembly>
<assemblyIdentity name="Microsoft.WindowsAzure.Diagnostics" publicKeyToken="31bf3856ad364e35" culture="neutral"/>
<bindingRedirect oldVersion="0.0.0.0-2.8.0.0" newVersion="2.8.0.0"/>
</dependentAssembly>

Simple right… Not quite.

Looking deeper at the stack trace you’ll see:

Microsoft.WindowsAzure.ServiceRuntime.Implementation.Loader.RoleRuntimeBridge.<InitializeRole>b__0()

When you’re using IIS (like we are) the startup process for your app is handled by WAIISHost.exe not but w3wp.exe (aka. the worker process). This means that the web.conf file isn’t applicable and the binding redirect is useless here.

There are various blog posts and stack overflow questions/answers about this. A lot of them mention adding an App.config file or a WaIISHost.exe.config file. Neither of these worked with the full IIS hosting model and Azure SDK 2.8.

Solved: I had to add a file <WebProjectName>.dll.config and set it to copy always. Once that was packaged and deployed the app started as normal and the binding redirect worked fine. For reference here is the contents of my <WebProjectName>.dll.config

<?xml version="1.0" encoding="utf-8"?>
<!--
For more information on how to configure your ASP.NET application, please visit
http://go.microsoft.com/fwlink/?LinkId=169433
-->
<configuration>

<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

<dependentAssembly>
<assemblyIdentity name="Microsoft.WindowsAzure.Diagnostics" publicKeyToken="31bf3856ad364e35" culture="neutral"/>
<bindingRedirect oldVersion="0.0.0.0-2.8.0.0" newVersion="2.8.0.0"/>
</dependentAssembly>

</assemblyBinding>
</runtime>

</configuration>

Hopefully this helps anyone else who finds themselves in a similar position with all of the binding rediects in place but still not working when deploying to Azure.

An end to the importance of domain names?

Don’t worry about the title, I’m not proclaiming and end to domain names altogether, but I’m noticing a new trend in the way websites are advertised in the mainstream media. 

There have been a number of adverts on TV that publicise a search term rather than the URL of a site.  The most recent advert that I’ve seen is for the Madeleine Peyroux album Bare Bones, which asks viewers to search “who is she” for more details. There’s also the advert for the More 4 pub quiz where they say “search online for more4 pub quiz” at the end of the advert (the video is below).

There’s been a few other adverts too, including government adverts for services, and health care. 

I think this is an interesting trend, and shows that companies are starting to realise that having the perfect domain isn’t the most important factor in having a strong web presence. After all just because you have a great domain for your product or service doesn’t mean people are going to know about it. People use search engines every day to find things online, and that’s certainly nothing new, so companies are starting to realise how important search engine optimisation really is. 

The trend towards advertising a search term rather than a domain adds a new twist. In traditional SEO you would identify what search terms people use, and then adjust the content of your website accordingly, in effect you take your website to your potential visitor. But larger companies are now advertising the search term, and creating the popularity and driving traffic towards an already optimised site, for what would otherwise be a mediocre search term.

There’s also the factor of the human memory. Take for example the Madeleine Peyroux website, if I had seen the advert and they advertised, say, madeleinepeyroux.com then I wouldn’t have a clue how to spell it, and would probably  struggle to locate the site. But advertising with the term “who is she” is nice and simple and easy to remember, so chances are I’ll actually find the site I’m looking for.

This isn’t the first time where people have taken an ‘unpopular’ search term and created hype around it to drive traffic. A few years ago somone did just that with the term “miserable failure“, which gained a lot of popularity. That was mainly for a joke, but we’re starting to see commercial companies taking the same approach.

In principle it works, you take an unpopular search term, and create the popularity, but it has it’s down sides. For a start you have to be a fairly large company to have access to the kind of media coverage that’s required to create the popularity in the first place, although it won’t be long before we start to see some kind of guerrilla marketing campaign that achieves the same result. There’s also the issue of SEO itself, you always run the risk that someone else with have a better search engine rank than you. Which undoes all of your hard work in generating the popularity in the first place and could quite easily direct traffic to a competitors website (if it’s better optimised than yours), meaning you’ve spent hard earned money and someone else is reaping the rewards.

This could be the achelies heel in this kind of marketing, but it will be interesting to see if advertising in the mainstream press starts to move away from domain names in favour of search terms.

Synergy – The ultimate desktop sharing tool

Synergy logoI’ve been doing some internal testing at work this week using Safari, and I’ve moved our Mac Mini onto my desk rather than using VNC. I’ve been using some awesome software called Synergy which lets you share your keyboard and mouse over the network. So I currently have my main work PC (running Windows) which has dual-monitors, next to a third monitor which is hooked up to the Mac Mini. With synergy I can move my mouse cursor between all three monitors and work on either machine without having to switch keyboards, or use a KVM.

Obviously I can’t drag/drop files or move windows between machines, but it’s made life a lot easier for performing Mac testing. It even supports cut/copy/paste from one machine to the other. Here’s a picture of my desktop setup at work (click to see the Flickr version that has notes)

Microsoft vs Adobe (the new ‘Expression’ suite)

Microsoft Expression Web LogoIt looks like Microsoft have officially settled on the name for their new suite of creative web applications, called ‘Expression‘, and they seem to have Adobe in their sights.

One of the products (Expression Web) went on sale on Amazon.com on 4th December, but at the moment it’s only available to pre-order for $269.10, or if you’re lucky enough to have Frontpage, then you can upgrade for $89.96 (the best $89.96 you’ll ever spend if you ask me!).

The suite contains four products:

Expression Web is already shipping, and according to the website “is a professional design tool to create modern, standards-based sites which deliver superior quality on the Webis a professional design tool to create modern, standards-based sites which deliver superior quality on the Web“. I’m not entirely sure if this is a direct replacement of Frontpage but it claims to have “A sophisticated CSS rendering engine” which would be a welcome step towards a standards based design application, it will be interesting to see how IE6 handles the standards based code.

Microsoft Expression Web Interface

Expression Blend, according to the website, “is the professional design tool to create engaging web-connected experiences“. Sound familiar? Although the site talks a lot about designing user interfaces, so it could be a prototyping application for designing web-interfaces before they’re implemented, rather than developing rich media applications, similar to how designers use Adobe Fireworks to layout a website.

Microsoft Expression Blend Interface

Expression Design is Microsoft’s new vector illustration tool. It’s a brave move considering the dominance of products like Adobe Illustrator (in the web market at least).

Microsoft Expression Design Interface

Expression Media is a digital asset management tool and an enhancement to their existing iView MediaPro package (which they acquired earlier this year). It promises to offer ‘effortless organization’, workflow integration, image and video editing, and presentation capabilities, from within a single application. It seems like a direct competitor to Adobe Bridge. According to the website (and the screenshots) this is/will be Mac compatible.

Microsoft Expression Media Interface

My initial thoughts on the Expression Suite are obvious, why compete against the likes of Adobe? After some thought I realised there are a lot of people developing ASP.NET applications using Visual Studio, and the Expression Suite will fill the obvious design void for those users.

LG Chocolate (KG800) Review

LG Chocolate (KG800) BlackI’ve been using the LG Chocolate (a.k.a the KG800) for a few months now, and I’ve decided to write a little review of what it’s like to use the phone on a day to day basis.

KG describe the phone as Unique, Articulate, Exquisite”, but I can think of few other words I’d add to that list.

I’ll be honest, I bought this phone after reading reviews online, and hadn’t picked it up or browsed through it, prior to getting it, but online, it sounded excellent, great design, great UI, stylish etc…

I have to admit that the phone looks great, and usually gets an “oooooooo” response when people first see it,  but the more I’ve used it more annoyed I’ve become.

Here’s a few issues that I’ve come across:

  1. Long startup time
    On average it takes 1:29sec for the phone to be in a usable state after switching it on (i.e. from entering the PIN number to being able to browse the phone book, access stored messages etc). This might sound like I’m being impatient but if you’ve had the phone off during a meeting and need to get someone’s number then it’s a long wait.
  2. Alarms
    I don’t actually own an alarm clock, because I’ve never needed to! My mobile phone is usually on my bedside table at night, and the built-in alarm is usually enough to get me up in the morning, but not with the KG800. The main problem is that the alarm obeys the active profile, so if the phone’s in silent mode then you don’t get an audible alarm at all, if it’s in vibrate mode then the alarm will just trigger a vibrate. My old phone (and most Nokia phones) will always have an audible alarm, regardless of the active profile. So I’ve had to keep the standard profile on, and manually disable message alters so I don’t get woken up by late night messages!Another problem is that I’ve woken up a few times without the alarm going off at all. When I’ve checked the phone it’s prompted me to enter my PIN because it restarted during the night for no apparent reason. This seems to happen randomly, and I’ve not been able to identify what’s causing it.
  3. Deleting text messages
    I tend to use SMS messages to communicate with friends & family rather than calling (it’s easier, and cheaper), but deleting old messages can be a bit of a pain. Mainly because of the way the phone handles lists of items. When you delete a text message you’re given a confirmation message (which is fine) and then the first message (i.e. the latest message you’ve received) is selected, forcing you to scroll back through the list to delete other messages. If you’re deleting a few messages to save space (but you don’t want to use the ‘delete all’ option) then it gets annoying pretty quickly. The same thing happens when deleting contacts from your phone book.
  4. Touch ‘sensitive’ interface
    In LG’s words the

    “Unique heat sensitive touch pad glows red when active. Sensitive to a warm touch you can be sure that the touch pad won’t activate by items in your pocket or bag.”

    In practice that isn’t true, I’ve had a few calls & text messages from friends saying that I’ve called them, when the phone’s been in my pocket and hasn’t been touched.

    In general, I’ve managed to get used to the lack of feedback (something that’s always put me off touch-sensitive interfaces) but the interface is very sensitive and I’ve often hit the ‘return’ button only for the menu to scroll right (which is the button directly above return).

  5. Contact photos
    If you want to assign a photo to a contact in your address book then you have to take it at 72×72, there’s no automatic resizing like the K750i!

To be honest, if your mobile phone is more than just a fashion accessory, and gets used on a daily basis then I’d stay away from the KG800 (especially the white version). Which is a shame, because it looks great!

 Scroll to top