If unit testing is the cake, functional testing is just the icing

I’ve heard a lot of different view points on software testing over the years. Especially when it comes to unit testing. Some people swear by it and treat code coverage metrics like it’s their greatest achievement in life (maybe it is). Others think it’s an unnecessary burden. I’ve heard arguments like “how can you refactor code if you have to update all of your tests each time” and “how can you refactor code if you don’t have sufficient unit test coverage” or “you’re flying by the seat of your pants without good coverage” vs “your tests are just a burden, you’re slowing me down!”.

I’d like to give a view from the trenches as it were. Albeit a view my trench working on a REST web service.

Some background

Before I begin I’d like to clarify what I mean by “unit test” and “functional test”.

  1. Unit testing – This is breaking down the application code into individual units that can be tested in isolation at a code level.
  2. Functional testing – This is more of a higher level, black-box test (normally from a user point of view) where the test doesn’t care about how the application functions. In this context functional are REST API calls simulating user behaviours and flows.

Over the past 18 months we’ve tried to balance our approach to testing. To do this we have taken the view that the end user is the most important piece of the puzzle. So we should focus our testing efforts on user scenarios. This means more functional tests, and less focus on unit testing.

The problem

For quite a while this approach worked well. We have suits of functional tests that make REST API calls to our web service and check the responses coming back. The tests walk through specific scenarios (creating user accounts, configuring the state of user accounts, testing scenarios with these accounts etc) and assert that the system behaves as expected. If something breaks (like you change the location of a field, or something that was an integer in the response is now a string, then the test fails, or a field is removed from the response.

Fast forward 18 months. We now have several hundred of these tests. They run every time code in the “dev” branch is updated (for us that means every time a feature branch is merged in and the code is pushed to our test environment). “That’s good right?” Well, here are some down-sides we’ve seen

  1. Tests are slow! – These kind of functional tests aren’t quick to execute, they are testing real systems with complex logic behind them.
  2. They are dependent on things I don’t care about – Should I really care if our user account team has a server down in our test environment? Should it block my work? What if the storage platform is down? In this instance if my test can’t create a user account then it fails and I have to investigate why.
  3. They test my colleagues work – Remember when I said these tests run every time code is merged in from feature branches. That means that not only are these codes testing my work, but they are testing other peoples work too. If I’m testing that value A = B and someone else is working on a feature where A = B+C then we have a race to see who gets there first with both code changes and test changes (granted this could be avoided by managing the work but it’s not always so simple in the real world).
  4. They are slow! – I’ve said it twice because it’s important. When testing changes I want to be confident that my changes work as expected, but I also want to be sure that I haven’t broken anything. Functional tests often run late in the development process (they need to run on a test environment after the code is built & deployed) which means you get feedback late too. If you did break something, finding out at this stage normally means you’ve broken your service / app for someone else in the testing environment.

The solution

I honestly think that both unit testing and functional testing have their place in web app development. It’s critical to test your application from the inside (via unit tests) and the outside (via functional tests). After all, your users are the most important thing so you have to test that they get the behaviour they expect.

However, a lot of people ignore unit testing, or at least treat it like second class citizen. Maybe it’s because they associate it with TDD or other approaches they dislike. Maybe it’s a lack of knowledge of how to write testable code (dependency injection, inversion of control, TDD all help with this).

Personally, I think that unit testing should be the primary focus of your testing efforts. Why?

  1. Unit tests are fast (well, if written properly) – Yes, it’s possible to write slow unit tests, but, if you’re breaking your code down into small individual units and writing testable code in the first place, then the tests should be quick to execute.
  2. Feedback is isolated- Unit tests run against my local changes. So if a test breaks I know it’s almost certainly because I’ve broken something and I need to fix it. No need to go on a wild goose chase only to find out its someone else’s change.
  3. You can refacor with confidence – I can look at code coverage reports and other metrics and assess if the code I’m going to refactor has test coverage and then make changes with a higher degree of confidence. If something breaks I get feedback straight away.
  4. They enforce good patterns and practices – Just like it’s possible to write a slow test, it’s also possible to write a monolithic test to test monolithic code. Unit testing makes you think about how you’re going to test your code (like how can I test what happens if dependency X throws an exception). Unit testing nudges you towards practices like dependency injection, inversion of control (etc) all of which makes code easier to maintain and understand. Everyone wins.

Having a solid set of unit tests means you catch issues early and can refactor with confidence, leaving the functional tests there as a safety net rather than first line of defence. Like I said in the title, unit tests should really be the cake, and functional tests are just the icing.

Progress on the Kingsgate Media Player

Just a quick update on the progress of the Kingsgate media player. Lots of things have been completed, the main feature being that video playback is now supported! Here’s a video showing the app in action on my Amazon FireTV.

There have been a lot of other internal changes too, including:

  1. Implemeted dependency injection using Dagger 2
  2. Added automated builds using Travis-CI
  3. Added new HttpClient, HttpRequest and HttpResponse object using DI

Lots more to come!

Solving a problem, with technology

I’ve been attending a fantastic church, Kingsgate, for over a year now. The level of technology that the church uses is excellent, but there is one problem.

Podcasts.

For ages podcasts have been a pain to use. Some platforms make it easy, like iOS who have a dedicated podcast app where you can search for Kingsgate and you see the podcast. It’s not a bad experience but it’s not great either. For one, you only get the audio feed not the video feed. The logo is also missing and it just looks pretty crappy.

My main issue is that I want to watch the video of the sermons from Church not just listen to the audio. I have a fairly nice TV and I can stream a tonne of great programs from Netflix and Amazon Prime, and BBC iPlayer. So why is it so hard to watch the sermons from church on my TV?

I know there are RSS apps for a lot of different platforms (even the Amazon Fire TV) but I’m not really the target audience. What about my parents, how can they watch the podcast of the sermons on their TV:

  1. Go to the app store of choice
  2. Search for RSS (no “dar-ess”, the letters “R”,”S”,”S”)
  3. Now install one (if you’re lucky there is only one and you don’t have to make a choice)
  4. Now go to settings, and remove any default subscriptions (CNN, BBC etc)
  5. Click add subscription
  6. Now type “http://”
  7. Give up

There must be a better way…

I have an Amazon Fire TV which is essentially and Android box with a better interface. So I’ve decided to create an app that lets you easily access the sermon podcasts on the Amazon Fire TV.

12661840_921537694627093_5182743669736874803_n.jpg

Right now this is totally unofficial, and in no way linked to Kingsgate. The app is currently fetching a list of available sermons. The next steps are:

  1. Full-screen media playback
  2. Keeping track of played / un-played sermons
  3. Porting to iOS (iPad, Apple TV) and Windows 10 (desktop, tablet, windows phone and Xbox One)

All of the code for this is on GitHub so if you want to help out then feel free to fork the repo and send a pull request!

“Could not load file or assembly” in Azure Web Role

Every now and again you hit what seems like a simple problem only to dig deeper and find it’s not as straight forward as first thought…Today was one such time. A simple upgrade of an Azure application to the latest version of the Azure SDK (2.8 at the time of writing this). Everything is updated, compiled, tested, and running locally. Then deployed to Azure and… bang!

image

The full error reads:

Could not load file or assembly ‘Microsoft.WindowsAzure.Diagnostics, Version=2.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’

Surely this should have affected the app when running locally too, and even if it didn’t it’s simple to fix. Just add a binding redirect (in this case in my web.config file).

<dependentAssembly>
<assemblyIdentity name="Microsoft.WindowsAzure.Diagnostics" publicKeyToken="31bf3856ad364e35" culture="neutral"/>
<bindingRedirect oldVersion="0.0.0.0-2.8.0.0" newVersion="2.8.0.0"/>
</dependentAssembly>

Simple right… Not quite.

Looking deeper at the stack trace you’ll see:

Microsoft.WindowsAzure.ServiceRuntime.Implementation.Loader.RoleRuntimeBridge.<InitializeRole>b__0()

When you’re using IIS (like we are) the startup process for your app is handled by WAIISHost.exe not but w3wp.exe (aka. the worker process). This means that the web.conf file isn’t applicable and the binding redirect is useless here.

There are various blog posts and stack overflow questions/answers about this. A lot of them mention adding an App.config file or a WaIISHost.exe.config file. Neither of these worked with the full IIS hosting model and Azure SDK 2.8.

Solved: I had to add a file <WebProjectName>.dll.config and set it to copy always. Once that was packaged and deployed the app started as normal and the binding redirect worked fine. For reference here is the contents of my <WebProjectName>.dll.config

<?xml version="1.0" encoding="utf-8"?>
<!--
For more information on how to configure your ASP.NET application, please visit
http://go.microsoft.com/fwlink/?LinkId=169433
-->
<configuration>

<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

<dependentAssembly>
<assemblyIdentity name="Microsoft.WindowsAzure.Diagnostics" publicKeyToken="31bf3856ad364e35" culture="neutral"/>
<bindingRedirect oldVersion="0.0.0.0-2.8.0.0" newVersion="2.8.0.0"/>
</dependentAssembly>

</assemblyBinding>
</runtime>

</configuration>

Hopefully this helps anyone else who finds themselves in a similar position with all of the binding rediects in place but still not working when deploying to Azure.

Time for a spring clean

It’s been far too long since I last looked at this blog but I decided it’s about time for a quick facelift and a bit of TLC. If you’re reading this via a feed reader then head over to the blog and take a look at the new theme.

The theme is PixelPower and makes nice use of both HTML5 and CSS3 to produce a responsive design that works well on mobile devices (so I’ve been able to do away with WPTouch) and still have an accessible site on mobile devices.

Giving the blog a facelift was the easy part. Now I need to actually get some time to write some posts!

Stunnel & Apache (Invalid method in request x80gx01x03)

Here’s a really quick post about an issue I’ve encountered recently when using stunnel to connect through to Apache via HTTPS.  I set up the connections and then tried to view the end-point using ‘links’ (https://localhost) and received an SSL error. The apache logs listed:

Invalid method in request x80gx01x03

The stunnel config that I was using looked something like this

[web]
accept=443
connect=someserver.com:1234 # Apache SSL listing on a non-standard port
cert=/path/to/my/cert.pem

It turned out to be a really simple fix. Because I was connecting to stunnel using SSL it was being encrypted by my browser then encrypted by stunnel. At the other end it was being decrypted by stunnel and then left with my original browser encrypted data which Apache couldn’t do anything with and couldn’t understand the request.

The fix was to simply change the config to:

[web]
accept=80
connect=someserver.com:1234 # Apache SSL listing on a non-standard port
cert=/path/to/my/cert.pem

Then test the connection via ‘links http://localhost’ and let stunnel handle the encryption and certificate negotiation on its own.

An end to the importance of domain names?

Don’t worry about the title, I’m not proclaiming and end to domain names altogether, but I’m noticing a new trend in the way websites are advertised in the mainstream media. 

There have been a number of adverts on TV that publicise a search term rather than the URL of a site.  The most recent advert that I’ve seen is for the Madeleine Peyroux album Bare Bones, which asks viewers to search “who is she” for more details. There’s also the advert for the More 4 pub quiz where they say “search online for more4 pub quiz” at the end of the advert (the video is below).

There’s been a few other adverts too, including government adverts for services, and health care. 

I think this is an interesting trend, and shows that companies are starting to realise that having the perfect domain isn’t the most important factor in having a strong web presence. After all just because you have a great domain for your product or service doesn’t mean people are going to know about it. People use search engines every day to find things online, and that’s certainly nothing new, so companies are starting to realise how important search engine optimisation really is. 

The trend towards advertising a search term rather than a domain adds a new twist. In traditional SEO you would identify what search terms people use, and then adjust the content of your website accordingly, in effect you take your website to your potential visitor. But larger companies are now advertising the search term, and creating the popularity and driving traffic towards an already optimised site, for what would otherwise be a mediocre search term.

There’s also the factor of the human memory. Take for example the Madeleine Peyroux website, if I had seen the advert and they advertised, say, madeleinepeyroux.com then I wouldn’t have a clue how to spell it, and would probably  struggle to locate the site. But advertising with the term “who is she” is nice and simple and easy to remember, so chances are I’ll actually find the site I’m looking for.

This isn’t the first time where people have taken an ‘unpopular’ search term and created hype around it to drive traffic. A few years ago somone did just that with the term “miserable failure“, which gained a lot of popularity. That was mainly for a joke, but we’re starting to see commercial companies taking the same approach.

In principle it works, you take an unpopular search term, and create the popularity, but it has it’s down sides. For a start you have to be a fairly large company to have access to the kind of media coverage that’s required to create the popularity in the first place, although it won’t be long before we start to see some kind of guerrilla marketing campaign that achieves the same result. There’s also the issue of SEO itself, you always run the risk that someone else with have a better search engine rank than you. Which undoes all of your hard work in generating the popularity in the first place and could quite easily direct traffic to a competitors website (if it’s better optimised than yours), meaning you’ve spent hard earned money and someone else is reaping the rewards.

This could be the achelies heel in this kind of marketing, but it will be interesting to see if advertising in the mainstream press starts to move away from domain names in favour of search terms.

Twitter

twitter_logo_headerI’ve decided that it’s time to get back on to Twitter. I used to use it about a year ago, and had the username ‘jb’ but decided to give it up as I wasn’t using my account. I’ve just added my twitter posts to the sidebar of the site, but if you use twitter then feel free to follow me, and drop me a message every now and again…

Snow snow snow…

View from the bathroomI’m writing this post for two exciting reasons… One it’s snowing outside! Something that hardly ever happens in the UK (especially where I live) and secondly I’m writing this post from our new house, looking out at the snow-covered garden!

Here’s a pic of the view into our new garden.

We moved in a few weeks ago and we’re finally getting sorted out. We’ve moved from a two bed ground-floor flat into a two bed house, so it’s nice to actually have enough space! No doubt I’ll post more pictures and details as we get settled in and start to decorate and make the place our own.

If you want our new address then you know how to contact me to ask…

The Red Ring of Death

The single phrase that strikes fear into the heart of any Xbox 360 owner, “The Red Ring of Death”, and it struck on Friday evening! There I was, sat playing Call of Duty 4 when without warning the screen turned to solid green. I’ve not had any problems with my Xbox or games crashing in the past so I quickly rebooted it using the front power switch (the console was completly unresponsive). When I switched it back on I was greeted with the following image

dsc00123

Thankfully Microsoft have an extended warranty policy for these problems (generally refered to as a “general hardware failure”), so I’m able to send it off for repair, however the repairs are completed in Germany so I’m hoping that I’ll be able to get the console back in time for Christmas so that I can play Left 4 Dead (even if it is a present from me to me)…

1 2 3 5  Scroll to top