In Praise of Characterization Testing

Many sources on testing imagine that the developer, or someone close-by, knows exactly how the system works and how it behaves, and imagines that the writing of tests naturally means supplying known inputs to some routine, and checking the results against the expected outputs. But the truth is that for some code, what it does and how it works is effectively unknown … not only do we not know what it does, we don’t even know if it does it correctly!

A ‘Characterization Test’ is a test that you write not knowing what the code under test does or what the correct operation is.  The tests will be written in conjunction with examining the code, writing test expectations in response to seeing the actual results; turning the normal expectation for writing tests on its head.

Continue reading

The One Missing Feature

Sometimes the success or failure of a new system function can end up depending on one feature; and if that feature or function is missing the whole reason for the system existing can be undermined.

This system we are looking at today is one which can track vehicles, and one feature is the ability to plot journeys on a map in a web-browser; it’s actually very cool, but the plotting of the route was a little slow. Continue reading

Software Inventory – Joel on Software

While the Software Inventory post by Joel Spolsky is a thinly-veiled advertisement for Fog Creek’s new (at the time) ‘project management’ tool Trello; the points Joel makes are nevertheless deeply meaningful to me. Points like this:

“The trouble is that 90% of the things in the feature backlog will never get implemented, ever. So every minute you spent writing down, designing, thinking about, or discussing features that are never going to get implemented is just time wasted.”

Or this:

“… the desire never to miss any bug report leads to bug bankrupcy, where you wake up one day and discover that there are 3000 open bugs in the database, some of which are so old they may not apply any more, some of which can never be reproduced, and most of which are not even worth fixing because they’re so tiny.”

…though you probably won’t need any more than 20 bug reports to find outdated or incorrect tickets!

I’ve just had another reason to hunt-out and refer to this blog post professionally, in an effort to encourage a client that having a bunch of years-old tickets assigned to someone but ‘on-hold’ when the original ticket requester has left the business is a sure-sign that really those projects are never going to happen and are probably not a good idea anyway!

– Make it consisent – Make it consisent – Make it consisent

We made a mistake recently, breaking one of our own rules; Be Consistent.  Now, of course it is not always possible to ‘be consistent’, sometimes because you are doing something truly new; but often because one incorrectly sees differences – when you may be better off seeing patterns and similarities (and thus implementing something to fit an existing pattern)!

Continue reading

Making Use-Case Diagrams Useful

We have found that communicating our system designs with clients is most usefully done with diagrams rather than large chunks of text.  Some years ago, we looked at using UML Use Case Diagrams for this communication – but see what Martin Fowler has to say about them in  ‘UML Distilled’:

“But almost all the value of use cases lies in the content, [of the textual cases]  and the diagram is of limited value.’

In other words, in his opinion, you should use the textual Use Cases, not the diagrams (which just map those texts visually).

‘Pure’ UML also seems to disappoint in terms of producing very dry monochrome diagrams with stick figures, and simple primitives such as boxes and ovals.  Is this really the best we can do to convey the use of a system?

Continue reading