Tech

If unit testing is the cake, functional testing is just the icing

I’ve heard a lot of different view points on software testing over the years. Especially when it comes to unit testing. Some people swear by it and treat code coverage metrics like it’s their greatest achievement in life (maybe it is). Others think it’s an unnecessary burden. I’ve heard arguments like “how can you refactor code if you have to update all of your tests each time” and “how can you refactor code if you don’t have sufficient unit test coverage” or “you’re flying by the seat of your pants without good coverage” vs “your tests are just a burden, you’re slowing me down!”.

I’d like to give a view from the trenches as it were. Albeit a view my trench working on a REST web service.

Some background

Before I begin I’d like to clarify what I mean by “unit test” and “functional test”.

  1. Unit testing – This is breaking down the application code into individual units that can be tested in isolation at a code level.
  2. Functional testing – This is more of a higher level, black-box test (normally from a user point of view) where the test doesn’t care about how the application functions. In this context functional are REST API calls simulating user behaviours and flows.

Over the past 18 months we’ve tried to balance our approach to testing. To do this we have taken the view that the end user is the most important piece of the puzzle. So we should focus our testing efforts on user scenarios. This means more functional tests, and less focus on unit testing.

The problem

For quite a while this approach worked well. We have suits of functional tests that make REST API calls to our web service and check the responses coming back. The tests walk through specific scenarios (creating user accounts, configuring the state of user accounts, testing scenarios with these accounts etc) and assert that the system behaves as expected. If something breaks (like you change the location of a field, or something that was an integer in the response is now a string, then the test fails, or a field is removed from the response.

Fast forward 18 months. We now have several hundred of these tests. They run every time code in the “dev” branch is updated (for us that means every time a feature branch is merged in and the code is pushed to our test environment). “That’s good right?” Well, here are some down-sides we’ve seen

  1. Tests are slow! – These kind of functional tests aren’t quick to execute, they are testing real systems with complex logic behind them.
  2. They are dependent on things I don’t care about – Should I really care if our user account team has a server down in our test environment? Should it block my work? What if the storage platform is down? In this instance if my test can’t create a user account then it fails and I have to investigate why.
  3. They test my colleagues work – Remember when I said these tests run every time code is merged in from feature branches. That means that not only are these codes testing my work, but they are testing other peoples work too. If I’m testing that value A = B and someone else is working on a feature where A = B+C then we have a race to see who gets there first with both code changes and test changes (granted this could be avoided by managing the work but it’s not always so simple in the real world).
  4. They are slow! – I’ve said it twice because it’s important. When testing changes I want to be confident that my changes work as expected, but I also want to be sure that I haven’t broken anything. Functional tests often run late in the development process (they need to run on a test environment after the code is built & deployed) which means you get feedback late too. If you did break something, finding out at this stage normally means you’ve broken your service / app for someone else in the testing environment.

The solution

I honestly think that both unit testing and functional testing have their place in web app development. It’s critical to test your application from the inside (via unit tests) and the outside (via functional tests). After all, your users are the most important thing so you have to test that they get the behaviour they expect.

However, a lot of people ignore unit testing, or at least treat it like second class citizen. Maybe it’s because they associate it with TDD or other approaches they dislike. Maybe it’s a lack of knowledge of how to write testable code (dependency injection, inversion of control, TDD all help with this).

Personally, I think that unit testing should be the primary focus of your testing efforts. Why?

  1. Unit tests are fast (well, if written properly) – Yes, it’s possible to write slow unit tests, but, if you’re breaking your code down into small individual units and writing testable code in the first place, then the tests should be quick to execute.
  2. Feedback is isolated- Unit tests run against my local changes. So if a test breaks I know it’s almost certainly because I’ve broken something and I need to fix it. No need to go on a wild goose chase only to find out its someone else’s change.
  3. You can refacor with confidence – I can look at code coverage reports and other metrics and assess if the code I’m going to refactor has test coverage and then make changes with a higher degree of confidence. If something breaks I get feedback straight away.
  4. They enforce good patterns and practices – Just like it’s possible to write a slow test, it’s also possible to write a monolithic test to test monolithic code. Unit testing makes you think about how you’re going to test your code (like how can I test what happens if dependency X throws an exception). Unit testing nudges you towards practices like dependency injection, inversion of control (etc) all of which makes code easier to maintain and understand. Everyone wins.

Having a solid set of unit tests means you catch issues early and can refactor with confidence, leaving the functional tests there as a safety net rather than first line of defence. Like I said in the title, unit tests should really be the cake, and functional tests are just the icing.

Progress on the Kingsgate Media Player

Just a quick update on the progress of the Kingsgate media player. Lots of things have been completed, the main feature being that video playback is now supported! Here’s a video showing the app in action on my Amazon FireTV.

There have been a lot of other internal changes too, including:

  1. Implemeted dependency injection using Dagger 2
  2. Added automated builds using Travis-CI
  3. Added new HttpClient, HttpRequest and HttpResponse object using DI

Lots more to come!

Solving a problem, with technology

I’ve been attending a fantastic church, Kingsgate, for over a year now. The level of technology that the church uses is excellent, but there is one problem.

Podcasts.

For ages podcasts have been a pain to use. Some platforms make it easy, like iOS who have a dedicated podcast app where you can search for Kingsgate and you see the podcast. It’s not a bad experience but it’s not great either. For one, you only get the audio feed not the video feed. The logo is also missing and it just looks pretty crappy.

My main issue is that I want to watch the video of the sermons from Church not just listen to the audio. I have a fairly nice TV and I can stream a tonne of great programs from Netflix and Amazon Prime, and BBC iPlayer. So why is it so hard to watch the sermons from church on my TV?

I know there are RSS apps for a lot of different platforms (even the Amazon Fire TV) but I’m not really the target audience. What about my parents, how can they watch the podcast of the sermons on their TV:

  1. Go to the app store of choice
  2. Search for RSS (no “dar-ess”, the letters “R”,”S”,”S”)
  3. Now install one (if you’re lucky there is only one and you don’t have to make a choice)
  4. Now go to settings, and remove any default subscriptions (CNN, BBC etc)
  5. Click add subscription
  6. Now type “http://”
  7. Give up

There must be a better way…

I have an Amazon Fire TV which is essentially and Android box with a better interface. So I’ve decided to create an app that lets you easily access the sermon podcasts on the Amazon Fire TV.

12661840_921537694627093_5182743669736874803_n.jpg

Right now this is totally unofficial, and in no way linked to Kingsgate. The app is currently fetching a list of available sermons. The next steps are:

  1. Full-screen media playback
  2. Keeping track of played / un-played sermons
  3. Porting to iOS (iPad, Apple TV) and Windows 10 (desktop, tablet, windows phone and Xbox One)

All of the code for this is on GitHub so if you want to help out then feel free to fork the repo and send a pull request!

Stunnel & Apache (Invalid method in request x80gx01x03)

Here’s a really quick post about an issue I’ve encountered recently when using stunnel to connect through to Apache via HTTPS.  I set up the connections and then tried to view the end-point using ‘links’ (https://localhost) and received an SSL error. The apache logs listed:

Invalid method in request x80gx01x03

The stunnel config that I was using looked something like this

[web]
accept=443
connect=someserver.com:1234 # Apache SSL listing on a non-standard port
cert=/path/to/my/cert.pem

It turned out to be a really simple fix. Because I was connecting to stunnel using SSL it was being encrypted by my browser then encrypted by stunnel. At the other end it was being decrypted by stunnel and then left with my original browser encrypted data which Apache couldn’t do anything with and couldn’t understand the request.

The fix was to simply change the config to:

[web]
accept=80
connect=someserver.com:1234 # Apache SSL listing on a non-standard port
cert=/path/to/my/cert.pem

Then test the connection via ‘links http://localhost’ and let stunnel handle the encryption and certificate negotiation on its own.

An end to the importance of domain names?

Don’t worry about the title, I’m not proclaiming and end to domain names altogether, but I’m noticing a new trend in the way websites are advertised in the mainstream media. 

There have been a number of adverts on TV that publicise a search term rather than the URL of a site.  The most recent advert that I’ve seen is for the Madeleine Peyroux album Bare Bones, which asks viewers to search “who is she” for more details. There’s also the advert for the More 4 pub quiz where they say “search online for more4 pub quiz” at the end of the advert (the video is below).

There’s been a few other adverts too, including government adverts for services, and health care. 

I think this is an interesting trend, and shows that companies are starting to realise that having the perfect domain isn’t the most important factor in having a strong web presence. After all just because you have a great domain for your product or service doesn’t mean people are going to know about it. People use search engines every day to find things online, and that’s certainly nothing new, so companies are starting to realise how important search engine optimisation really is. 

The trend towards advertising a search term rather than a domain adds a new twist. In traditional SEO you would identify what search terms people use, and then adjust the content of your website accordingly, in effect you take your website to your potential visitor. But larger companies are now advertising the search term, and creating the popularity and driving traffic towards an already optimised site, for what would otherwise be a mediocre search term.

There’s also the factor of the human memory. Take for example the Madeleine Peyroux website, if I had seen the advert and they advertised, say, madeleinepeyroux.com then I wouldn’t have a clue how to spell it, and would probably  struggle to locate the site. But advertising with the term “who is she” is nice and simple and easy to remember, so chances are I’ll actually find the site I’m looking for.

This isn’t the first time where people have taken an ‘unpopular’ search term and created hype around it to drive traffic. A few years ago somone did just that with the term “miserable failure“, which gained a lot of popularity. That was mainly for a joke, but we’re starting to see commercial companies taking the same approach.

In principle it works, you take an unpopular search term, and create the popularity, but it has it’s down sides. For a start you have to be a fairly large company to have access to the kind of media coverage that’s required to create the popularity in the first place, although it won’t be long before we start to see some kind of guerrilla marketing campaign that achieves the same result. There’s also the issue of SEO itself, you always run the risk that someone else with have a better search engine rank than you. Which undoes all of your hard work in generating the popularity in the first place and could quite easily direct traffic to a competitors website (if it’s better optimised than yours), meaning you’ve spent hard earned money and someone else is reaping the rewards.

This could be the achelies heel in this kind of marketing, but it will be interesting to see if advertising in the mainstream press starts to move away from domain names in favour of search terms.

Optimizing Page Load Time

I came across an interesting article today (via Slashdot) entitled Optimizing Page Load Time written by Aaron Hopkins a Software Engineer for Google.

He’s put together quite an interesting study on how to improve the load time of a web page, using a variety of methods, covering everything from the obvious “Load fewer external objects”, to using HTTP Pipelining at the browser end. There’s also some interesting points about how host names affect the speed of your website like:

By default, IE allows only two outstanding connections per hostname when talking to HTTP/1.1 servers or eight-ish outstanding connections total. Firefox has similar limits. Using up to four hostnames instead of one will give you more connections. (IP addresses don’t matter; the hostnames can all point to the same IP.)

And a few, simple, yet often overlooked tips

Set an Expires header on everything you can, with a date days or even months into the future. This tells the browser it is okay to not revalidate on every request, which can add latency of at least one round-trip per object per page load for no reason.

It’s well worth a read.

 Scroll to top