Wasting Time TDDing The Wrong Things

When learning TDD, you usually start by testing small, individual classes. Exercises like these are designed to get you into the workflow quickly, without much hassle.

When you transition from small exercises to real-world work, however, things quickly get hairy.

A typical scenario goes something like this. You don’t know where to start, so you guess which classes you will need, and randomly pick a small one to begin with. A few red/green/refactor cycles later, and everything looks good! You successfully TDD’d a bunch of little classes. You just need to tie them all together, and you’ll be done – except they don’t fit together properly. You had a rough idea about the overall structure you were aiming for, but now you realise that isn’t going to work. Maybe you didn’t fully understand what you were trying to build, and now that you’ve built it, it technically works but doesn’t do what it was supposed to. You probably need to throw away a bunch of code that you just spent hours implementing and testing, and write it again. Or even worse, it might be 5:26pm on a Friday, tempting you to hack it all together and call it a day.

How can you “test first” if you don’t know exactly what you’re testing? If TDD is supposed to help with design, how can it lead you down the wrong path like this?

Bottom-Up Design, The Culprit

“Bottom-up” is also known as “inside-out.”

Classical TDD has a heavy focus on unit tests, and a distaste for mocks. When you TDD with unit tests and no mocks, you are naturally inclined to write the implementation from the bottom up. That is, you tend to write the small, independent classes first, because the higher-level classes depend upon them. Classical TDD encourages bottom-up design.

Unfortunately, bottom-up design is a contributor to the pains mentioned earlier. It is guesswork. You are guessing what code you will need, before you actually need it. This leads to waste if your guess isn’t exactly correct. If your guess is way off, you’ll be throwing your code away and rewriting it. Even if your guess is mostly correct, you might overshoot the mark, implementing unnecessary methods that are never used by the higher-level classes.

And yet plenty of respectable developers prefer this approach, despite its potential for waste. I think this is because your guesses become more accurate as you become more experienced. Through years of trial and error, experienced programmers develop a sort of sixth sense that allows them to avoid design pitfalls and dead ends most of the time. This is cold comfort for less-experienced programmers, since “spend a few years developing a sixth sense” is not particularly helpful advice.

Try Top-Down Design

“Top-down” is also known as “outside-in.”

Experience aside, what can you do to avoid these common problems in your TDD workflow? I suggest that you try the reverse approach: top-down design.

Here is a basic top-down workflow:

1. Start by asking what the whole thing is supposed to do. Paying specific attention to the inputs and outputs. This encourages you to clarify requirements – an important step in any design process.

2. Write a test for the top-level interface by using code that you wish existed. Think to yourself: “if this thing was already finished, what would the usage code look like?” You decide what interface you want before writing the implementation. This is key to good design.

3. Experiment with different designs. At this point, you have a test containing imaginary code. Change whatever you want, with no repurcussions. The best time to change your mind is when none of the implementation has been written yet.

4. Run the test, fix the errors, and repeat until the test passes. Once you’ve settled upon a design you like, it’s time to start the implementation. The imaginary code is going to crash, of course, but each crash tells you what you need to write next. If it crashes because a class doesn’t exist, write the class. If it crashes because a method doesn’t exist, write the method – and so on.

5. Repeat. Once you have a passing test, it’s time to write another one. This could be another high-level test, to help you design another part of the implementation. Alternatively, if the overall design is done, you can switch to unit tests.

The idea is that you implement only the code that you actually need, instead of guessing. This cuts down on the waste that classical TDD can produce.

These tests are not technically unit tests. Unit tests focus on a single unit, typically a class. These tests focus on groups of objects working together – more like a system than a unit.

High-level tests tend to gloss over the fine details and edge cases. For those, it is best to drop down into lower-level tests, like unit tests. When you get to the unit tests, you can at least be confident that the classes won’t need to be rewritten.

Example: Designing A Brainf*#k Interpreter

Brainf*#k is a tiny programming language with eight instructions, each represented by one character. Let’s look at designing a Brainf*#k interpreter from the top down.

The first step is to study the requirements, paying special attention to the inputs and outputs. The interpreter needs to accept the instructions to run, so that is one of the inputs. The , instruction reads a byte from an input stream such as $stdin, so that is another input. The only output is the . instruction, which writes a byte to an output stream, such as $stdout. Everything else appears to be an implementation detail – it can all be shoved under the hood, while we design the interface.

The usage code I would like to see is something like this:

instructions = ",[.,]"
BrainFunk.run(instructions)

Let’s put that into an RSpec test:

RSpec.describe BrainFunk do
  it 'runs instructions' do
    instructions = ",[.,]" # copies input to output
    BrainFunk.run(instructions)
  end
end

Notice that the test does not use expect anywhere, to assert that the results are correct. That’s because there is no easy way to get the results with this design. There is also no way for the test to provide input. The design doesn’t cover the input and output streams, so let’s redesign it:

require 'stringio'

RSpec.describe BrainFunk do
  it 'runs instructions' do
    input = StringIO.new('abc')
    instructions = ",[.,]" # copies input to output

    expect {
      BrainFunk.run(instructions, input)
    }.to output('abc').to_stdout
  end
end

This design allows us to specify the input. In the tests, we provide canned input with a StringIO. Outside of the tests, we can provide $stdin for user input, or maybe provide a File object to read input from. In both cases, the output is printed to $stdout.

If the input stream is being passed in now, why not pass in the output stream too? That way the implementation isn’t forced to use the $stdout global.

require 'stringio'

RSpec.describe BrainFunk do
  it 'runs instructions' do
    input = StringIO.new('abc')
    output = StringIO.new
    instructions = ",[.,]" # copies input to output

    BrainFunk.run(instructions, input, output)

    expect(output.string).to eq('abc')
  end
end

Now the output can be written to anything – a string in memory, standard output, a file, whatever.

If it tickles your fancy, you could make the interface more object-oriented at this point, but I’m going to skip that.

Now that I’m happy with the design, it’s time to start the implementation. I will leave this as an exercise for the reader.

Top-Down Rails

In Rails, you might start a top-down implementation with an integration test like this:

class BlahTest < ActionDispatch::IntegrationTest
  test "blah" do
    get "/blah"
  end
end

It’s just a single line of code in a single test, yet it covers routing, the controller, view rendering, and everything in between.

Once you’ve got the test to pass, you will want to add assertions, to verify the HTTP response and the contents of the rendered page. After that, you can add more integration tests, or drop down into something lower-level, like model tests.

Rails code is actually a bad example of top-down design, because a lot of Rails work doesn’t require much design. The framework provides the design for you, and new code tends to just slot into it. Even so, top-down testing can guide you towards what needs to be done next, at every step.

Further Resources

Top-down TDD is complicated, and this article is only a brief introduction to the topic. Here is a list of resources that go into more depth:

• My favorite way to TDD by Justin Searls. This presentation contains a nice overview of top-down vs bottom-up TDD, along with its historical context. Justin also demonstrates his top-down approach by live-coding in Java.

• Growing Object-Oriented Software Guided by Tests by Steve Freeman and Nat Pryce. This popular book teaches a top-down style of TDD called “London school” or “mockist.”

• Mocks Aren’t Stubs by Martin Fowler. The title of this article is a bit misleading. You might guess that it’s about the definition of two words, but it’s much longer and deeper than that. This also gives an overview of top-down vs bottom up, and some historical context.

• Introducing BDD by Dan North. While BDD remains somewhat unpopular, it’s interesting to see how it tries to address the shortcomings of bottom-up TDD. This post explains what BDD is, and how it evolved out of TDD.