Development

If unit testing is the cake, functional testing is just the icing

I’ve heard a lot of different view points on software testing over the years. Especially when it comes to unit testing. Some people swear by it and treat code coverage metrics like it’s their greatest achievement in life (maybe it is). Others think it’s an unnecessary burden. I’ve heard arguments like “how can you refactor code if you have to update all of your tests each time” and “how can you refactor code if you don’t have sufficient unit test coverage” or “you’re flying by the seat of your pants without good coverage” vs “your tests are just a burden, you’re slowing me down!”.

I’d like to give a view from the trenches as it were. Albeit a view my trench working on a REST web service.

Some background

Before I begin I’d like to clarify what I mean by “unit test” and “functional test”.

  1. Unit testing – This is breaking down the application code into individual units that can be tested in isolation at a code level.
  2. Functional testing – This is more of a higher level, black-box test (normally from a user point of view) where the test doesn’t care about how the application functions. In this context functional are REST API calls simulating user behaviours and flows.

Over the past 18 months we’ve tried to balance our approach to testing. To do this we have taken the view that the end user is the most important piece of the puzzle. So we should focus our testing efforts on user scenarios. This means more functional tests, and less focus on unit testing.

The problem

For quite a while this approach worked well. We have suits of functional tests that make REST API calls to our web service and check the responses coming back. The tests walk through specific scenarios (creating user accounts, configuring the state of user accounts, testing scenarios with these accounts etc) and assert that the system behaves as expected. If something breaks (like you change the location of a field, or something that was an integer in the response is now a string, then the test fails, or a field is removed from the response.

Fast forward 18 months. We now have several hundred of these tests. They run every time code in the “dev” branch is updated (for us that means every time a feature branch is merged in and the code is pushed to our test environment). “That’s good right?” Well, here are some down-sides we’ve seen

  1. Tests are slow! – These kind of functional tests aren’t quick to execute, they are testing real systems with complex logic behind them.
  2. They are dependent on things I don’t care about – Should I really care if our user account team has a server down in our test environment? Should it block my work? What if the storage platform is down? In this instance if my test can’t create a user account then it fails and I have to investigate why.
  3. They test my colleagues work – Remember when I said these tests run every time code is merged in from feature branches. That means that not only are these codes testing my work, but they are testing other peoples work too. If I’m testing that value A = B and someone else is working on a feature where A = B+C then we have a race to see who gets there first with both code changes and test changes (granted this could be avoided by managing the work but it’s not always so simple in the real world).
  4. They are slow! – I’ve said it twice because it’s important. When testing changes I want to be confident that my changes work as expected, but I also want to be sure that I haven’t broken anything. Functional tests often run late in the development process (they need to run on a test environment after the code is built & deployed) which means you get feedback late too. If you did break something, finding out at this stage normally means you’ve broken your service / app for someone else in the testing environment.

The solution

I honestly think that both unit testing and functional testing have their place in web app development. It’s critical to test your application from the inside (via unit tests) and the outside (via functional tests). After all, your users are the most important thing so you have to test that they get the behaviour they expect.

However, a lot of people ignore unit testing, or at least treat it like second class citizen. Maybe it’s because they associate it with TDD or other approaches they dislike. Maybe it’s a lack of knowledge of how to write testable code (dependency injection, inversion of control, TDD all help with this).

Personally, I think that unit testing should be the primary focus of your testing efforts. Why?

  1. Unit tests are fast (well, if written properly) – Yes, it’s possible to write slow unit tests, but, if you’re breaking your code down into small individual units and writing testable code in the first place, then the tests should be quick to execute.
  2. Feedback is isolated- Unit tests run against my local changes. So if a test breaks I know it’s almost certainly because I’ve broken something and I need to fix it. No need to go on a wild goose chase only to find out its someone else’s change.
  3. You can refacor with confidence – I can look at code coverage reports and other metrics and assess if the code I’m going to refactor has test coverage and then make changes with a higher degree of confidence. If something breaks I get feedback straight away.
  4. They enforce good patterns and practices – Just like it’s possible to write a slow test, it’s also possible to write a monolithic test to test monolithic code. Unit testing makes you think about how you’re going to test your code (like how can I test what happens if dependency X throws an exception). Unit testing nudges you towards practices like dependency injection, inversion of control (etc) all of which makes code easier to maintain and understand. Everyone wins.

Having a solid set of unit tests means you catch issues early and can refactor with confidence, leaving the functional tests there as a safety net rather than first line of defence. Like I said in the title, unit tests should really be the cake, and functional tests are just the icing.

Progress on the Kingsgate Media Player

Just a quick update on the progress of the Kingsgate media player. Lots of things have been completed, the main feature being that video playback is now supported! Here’s a video showing the app in action on my Amazon FireTV.

There have been a lot of other internal changes too, including:

  1. Implemeted dependency injection using Dagger 2
  2. Added automated builds using Travis-CI
  3. Added new HttpClient, HttpRequest and HttpResponse object using DI

Lots more to come!

Solving a problem, with technology

I’ve been attending a fantastic church, Kingsgate, for over a year now. The level of technology that the church uses is excellent, but there is one problem.

Podcasts.

For ages podcasts have been a pain to use. Some platforms make it easy, like iOS who have a dedicated podcast app where you can search for Kingsgate and you see the podcast. It’s not a bad experience but it’s not great either. For one, you only get the audio feed not the video feed. The logo is also missing and it just looks pretty crappy.

My main issue is that I want to watch the video of the sermons from Church not just listen to the audio. I have a fairly nice TV and I can stream a tonne of great programs from Netflix and Amazon Prime, and BBC iPlayer. So why is it so hard to watch the sermons from church on my TV?

I know there are RSS apps for a lot of different platforms (even the Amazon Fire TV) but I’m not really the target audience. What about my parents, how can they watch the podcast of the sermons on their TV:

  1. Go to the app store of choice
  2. Search for RSS (no “dar-ess”, the letters “R”,”S”,”S”)
  3. Now install one (if you’re lucky there is only one and you don’t have to make a choice)
  4. Now go to settings, and remove any default subscriptions (CNN, BBC etc)
  5. Click add subscription
  6. Now type “http://”
  7. Give up

There must be a better way…

I have an Amazon Fire TV which is essentially and Android box with a better interface. So I’ve decided to create an app that lets you easily access the sermon podcasts on the Amazon Fire TV.

12661840_921537694627093_5182743669736874803_n.jpg

Right now this is totally unofficial, and in no way linked to Kingsgate. The app is currently fetching a list of available sermons. The next steps are:

  1. Full-screen media playback
  2. Keeping track of played / un-played sermons
  3. Porting to iOS (iPad, Apple TV) and Windows 10 (desktop, tablet, windows phone and Xbox One)

All of the code for this is on GitHub so if you want to help out then feel free to fork the repo and send a pull request!

Firebug not running in Firefox on Linux (solution)

Firebug LogoSince switching to Linux I’ve had one issue that has really caused me grief over the past few weeks, and it’s been with Firebug, which to a web-developer is a pretty indispensable tool!

I compiled Firefox 2.0.0.11 using the ebuild from Gentoo Portage, and then installed the Firebug extension, but once Firefox had restarted to complete the installation, Firebug wouldn’t open at all. Whenever I tried to open it I received various error messages in Firefox’s error console including ‘permission denied to call method location.tostring’. Upgrading to Firefox 2.0.0.12, and Firebug 1.1.0b10 didn’t help.

After some digging I found a bug report that described exactly what’s happening to me. So I added ‘mozdevelop’ to my Portage USE flags, and then recompiled Firefox and now Firebug works a treat!

Synergy – The ultimate desktop sharing tool

Synergy logoI’ve been doing some internal testing at work this week using Safari, and I’ve moved our Mac Mini onto my desk rather than using VNC. I’ve been using some awesome software called Synergy which lets you share your keyboard and mouse over the network. So I currently have my main work PC (running Windows) which has dual-monitors, next to a third monitor which is hooked up to the Mac Mini. With synergy I can move my mouse cursor between all three monitors and work on either machine without having to switch keyboards, or use a KVM.

Obviously I can’t drag/drop files or move windows between machines, but it’s made life a lot easier for performing Mac testing. It even supports cut/copy/paste from one machine to the other. Here’s a picture of my desktop setup at work (click to see the Flickr version that has notes)

Safari – Must try harder

Well Apple have released the public beta of Safari 3 for Windows, claiming it has “blazing performance” and an “elegant user interface”, judge the last point for yourself!

Safari Screenshot 1

This is a screen shot of the default home page for Safari. There’s no text on the menus, or in the page, and the majority of the buttons haven’t loaded correctly either. I would include a link to this page so you could check it in other browsers, but the address bar doesn’t work, at all!

Apple say “Safari loads pages up to 2 times faster than Internet Explorer 7 and up to 1.6 times faster than Firefox 2.” which might be true, but I’d rather wait a few more seconds to have a page render correctly, especially if it means the Amazon home page doesn’t look like this

Safari Screenshot 2

Here’s a summary of the issues I’ve experienced

  1. Text not being displayed – This includes in menu items, pages, web-page buttons, address bar, search box
  2. Buttons not being displayed – The back, forward and refresh are the only buttons that have any icons
  3. No tool tips – Anywhere. This is especially annoying when buttons and menu items aren’t displayed
  4. Broken address bar – This doesn’t work at all, you can’t see what page you’re currently viewing, or type anything into it, making the browser unusable
  5. Random crashes – The browser keeps crashing for no reason at all

This is a shame, I’d love to have a fully functional version of Safari available on Windows, it would make development and testing a hell of a lot easier, but judging by the experiences of others I’d say it’s a little too early for this beta

Flash Rendering on top of HTML Elements

I’ve had few problems recently with Flash content rendering over the top of HTML elements, especially when using CSS based fly-out menus. Here’s an example of the problem, showing how Flash content is rendered over the top of the HTML drop-down menu.

Flash Wmode (Windowed)

This is quite annoying, especially after we’ve spent a lot of time developing a Flash intro movie for a client, only for them to ask if they can have drop-down menus on their home page too!

After a bit of digging around I came across the WMODE parameter which controls how the browser renders Flash content. There are three possible options, ‘window’ (default), ‘opaque’, or ‘transparent’. The example above shows the effect of the default option (‘window’).

Setting WMODE to ‘opaque’ has this effect

Flash Wmode (Opaque)

You can see the effect this has, but the background of this Flash movie is set to gray, so content underneath the Flash movie won’t be visible, so we’re 80% there…
Setting WMODE to ‘transparent’ fixes both issues, as you can see.

Flash Wmode (Transparent)

For more details embedding flash content you should check out the article over at A List Apart. There’s also an article on the Adobe website about wmode and how Flash is rendered.

Optimizing Page Load Time

I came across an interesting article today (via Slashdot) entitled Optimizing Page Load Time written by Aaron Hopkins a Software Engineer for Google.

He’s put together quite an interesting study on how to improve the load time of a web page, using a variety of methods, covering everything from the obvious “Load fewer external objects”, to using HTTP Pipelining at the browser end. There’s also some interesting points about how host names affect the speed of your website like:

By default, IE allows only two outstanding connections per hostname when talking to HTTP/1.1 servers or eight-ish outstanding connections total. Firefox has similar limits. Using up to four hostnames instead of one will give you more connections. (IP addresses don’t matter; the hostnames can all point to the same IP.)

And a few, simple, yet often overlooked tips

Set an Expires header on everything you can, with a date days or even months into the future. This tells the browser it is okay to not revalidate on every request, which can add latency of at least one round-trip per object per page load for no reason.

It’s well worth a read.

 Scroll to top