testing what we produce

Fri, Jan 25, 2008

As a group, the Integrated Technologies folk here have started thinking about how to test what we produce and what sort of options are open to us, both as software engineers and learning technologists. Having worked in software testing for a time many years ago and finding it interesting, I thought I’d write down some of the techniques I’ve used and am still using to test the software and systems I produce.

There are basically three levels of testing:

  • Unit tests. Where low level code is tested, down to class level.
  • Functional tests. Often called integration tests or black box tests. These exercise the technical functions of the system from the point of view of a user.
  • Acceptance tests. This is where the user signs on the dotted line, or not. It’s where the look and feel of the system is tested by the user.

Unit tests

Only programmers do unit tests. Only they know the system in enough detail to probe it for weaknesses and only they have the knowledge and skill to choose an appropriate framework and write the testing code. Most of the unit tests I do make use of JUnit. The tests are very focussed and small. They pick one area of low level functionality and make sure it works. e.g. read in a known file and compare the contents, to exercise a file I/O function of a class. Grouping unit tests into suites is verging on functional testing, except that the environment in which it’s done in no way resembles the real world. It might make use of mock objects to simulate databases or directory services. What they provide is insulation from real world problems such as network failure or bandwidth constraints. A suite of unit tests, grouped to test a wider area of functionality, will give you confidence that the application will work in an idealised environment.

Functional tests

These are often called integration or black box tests. They simulate a user’s interactions with the completed system, in a real world environment. So where unit test suites work with an idealised view of the world, functional tests are exposed to real world systems. They are susceptible to bad data, network problems and a host of other interesting scenarios. There are a few frameworks out there, such as Selenium, JFunc and HttpUnit. You don’t have to be a programmer to write functional test scripts. You can use expect as well, or you can even write your own suite of tests in any language you like. My favourite is HttpUnit and I used this to test CLAN functionality. There’s a nice tutorial on functional testing with HttpUnit here. When I used it in CLAN, I wrote some tests to simulate a user logging in to the VLE. I then navigated the HTML to their My Modules area and checked the module codes and names with what was expected from the student information system. Knowing what is expected, HttpUnit provides a way to programmatically extract what is actually there. So you can build up a suite of tests for specific areas of functionality. For example I could also, if I was still testing CLAN, use HttpUnit to login as a known user, navigate to the language selection page, change it to Gaelic and check the translations were working ok.

Acceptance tests

This is where the look and feel is tested by the people who will use the system. It’s here that anomalies with the spec will be revealed. “I didn’t mean that” or “can you change it to do that?” statements arise in this testing scenario. So you would make the changes, unit test them, integrate and run them through functional tests, then return to acceptance testing with the user. I wouldn’t bother with CSS and javascript stuff in functional tests. I’d leave that to the acceptance tests.

A big organsiation will have a dedicated test department, with detailed test plans that a test engineer will sit down and follow to the letter, recording anomalies. It’s also possible to express test plans in code or test scripts which can be run automatically.

For a programmer though, you can make it easier on yourself by adopting an agile methodology such as Test Driven Design (TDD). I use this quite a lot and a modern IDE will make it easier to work in this way as it will prompt you to create classes that don’t exist or methods that need to be created, as you craft your unit tests before you write any system code.

There’s a whole load more tests that sit at the level of functional testing, such as load and stress testing, regression testing etc but for me, these are the three main testing areas that I need to keep on top of.

comments powered by Disqus