An early morning last month (October 2016) a mix of software developers from several teams of Bonnier Broadcasting gathered in the cosy offices of Agical in Gamla Stan to spend a full day listening to the wise words of Mr. J. B. Rainsberger on the topic of TDD — Test-Driven Development.
Here follows an attempt to distil some of my reflections and take-aways from the event in the form of a combined summary and short intro to the topic.
What is a test; what is TDD?
As developers we are human, and as humans, we err. Writing tests is a way for us to eliminate, or at least reduce, errors by ensuring that what we build functions in the way we expect. If we write code that can add two numbers, we might write a test that asserts that given one plus two, we get three.
TDD takes the idea of testing a step further by in essence letting the test define correct behaviour, as opposed to verifying correct behaviour after the fact. Writing tests early on turns out to have positive effects not only on the correctness of our code, but also its design, with reduced coupling and better separation of concerns — perhaps this is the greatest benefit of TDD.
Many things were covered during the event and it would have been odd if everyone went home with the exact same insights. Here are just a couple of take-aways that struck me as having particular value.
Design is hard. Done well it brings many benefits and can be intellectually rewarding. Done wrong it can be the cause of high costs both financially and in the form of lowered productivity and motivation.
TDD is not a silver bullet that magically solves all design problems, and I will not claim that TDD is essential to designing things well, but it is a valuable tool that if used right allows us to find design mistakes early so that they may be corrected, before they become expensive. Rainsberger relates to the concept of risk exposure.
E = ∑([pm * cm, …])
Exposure is the sum of the probability of each possible mistake multiplied by the cost incurred should it happen.
To reduce exposure we may choose to reduce the probability of a mistake — in essence figuring out the potential mistake before it happens. If it is true that design is hard, then predicting future mistakes is not a trivial task.
Alternatively, we could assume that mistakes will be made and instead try to reduce their cost.
Code that is easy to test tends to be better designed, since — and one could almost get away with giving the inverse as the reason — well-designed code tends to be easier to test. It is like a virtuous circle, and also a concept within TDD; write a test, implement, test, refactor (improve the design), loop. This way TDD helps keep a clean design. If we only take care to watch for the tell-tales indicated by how straightforward and clear the tests are, it is easier to, at an early stage, spot potential flaws in our design.
When designing we make certain assumptions. That code that adds two numbers might assume that its arguments are numbers. In a statically typed language this particular case is of course automatically enforced. In a dynamically typed language, it has to be assumed or checked. This is a precondition for the code to work correctly and essentially becomes part of its contract, whether explicitly documented or not.
Another example is the Read method as defined by the interface io.Reader of the Go standard library. Read is used to read bytes from a data source by copying it into a given buffer p. Its contract states “if some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more”. It is the responsibility of the caller to be aware of this behaviour.
Assumptions always exist about how the code we write will be used, and about how code we use behaves. A clear contract not only defines what can and cannot be expected, but it helps define to what extent the code should be tested. While some things are difficult to capture in a test (such as unexpected behaviour), writing tests can be a nice way to explicitly demonstrate what kind of usage we expect and support.
Unit tests vs. integration tests
A unit test serves to test a small part of a system in isolation.
A unit test:
- …is isolated
- …is simple
- …is small
- …runs fast
- …is clear on what is being tested
- …makes it easy to pinpoint the cause of specific failure
- …drives design
An integration test serves to test how multiple layers of a system work together as a whole.
An integration test:
- …tests many parts of the system as a whole
- …might make calls to things outside of our control
- …can fail in many different ways
- …is slow
Side note: Rainsberger would correct the term “integration test” replacing it with “integraTED test”, defining it such that an integraTION test tests isolated points of contact between two parts, whereas an integraTED test is, quote, “any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behaviour“. While these definitions are clear in themselves, the use of two similar-sounding terms to mean different things tends to cause confusion (cf. authentication vs. authorization — both conveniently abbreviated ‘auth’), so I choose to simply walk around the definition swamp by using the term “integration test” to mean what Rainsberger calls “integrated test”, and for the time being avoid this other definition of “integration test”. How to name things: one of the hard problems in programming.
Rainsberger metaphorically compares the two methods of testing saying that the former is like painting a wall with a brush, while the latter is more like trying to cover all of it by throwing buckets of paint at it.
He describes the latter kind of tests as a scam and drew for us a vicious circle, illustrating that the more we start to rely on these tests, the more we lose the ability to iteratively improve our design, which in turn negatively affects the ability to write isolated unit tests, in turn resulting in even more end-to-end tests being added.
Perhaps one can extend the paint metaphor by saying that the more buckets of paint we throw at the wall, the harder it gets to move closer to it without getting our feet covered in paint.
Above being said, it should be pointed out that acceptance tests for the purpose of verifying behaviour from the end user’s point of view are perfectly fine, as long as they are not used as a substitute for tests which are usable in driving the design.
Even for someone already at least somewhat familiar with the idea of TDD I found it refreshing to during this event be able to reiterate on some of the concepts.
For the past two-or-so weeks we tried forcing ourselves to use TDD at one of our mob stations. In the context of mob programming it turned out to be quite useful in defining something of a road map of what to do next, and also as a means to keep focus on a particular task.
The most important insight that I gained from this day deserves its own line:
TDD is not about writing tests, it is about writing better code
Lastly, some things noteworthy.
- – Prepare for design change from any direction.
- – Not prepared for any particular change, but prepared for change
- – Test integrations with external parties in isolation.
- – Abstract external parties using interfaces, exposing only what is needed.
- – Avoid mocking types that you don’t own.
- – Do not pass nil to things intentionally.
- – Complex mocks can be an indication of bad design.
- – Make the documentation clear, entertaining, and small, with code examples.
- – 100% test coverage does not necessarily mean that all logic is being tested.
- – A worker should never talk directly to the client in multi-threaded code.
- – Test the worker in isolation; it should not know whether or not it is being called asynchronously.
- – Avoid large anonymous functions, because they are hard to test in isolation.
I shall also add that the above are my own interpretations, and as such do not necessarily correspond exactly to the ideas presented during the event.
Thank you for reading.