TDD is dead?

Ah, the good old “Is TDD dead?” debate. Such fun.

I was listening to the same old arguments going round in circles, and I’ve heard them so often I was able to sit back and ponder, and had a couple of new thoughts.

(By the way, I should declare myself as a committed TDDer. I’ve seen the benefit and the pleasure.)

First, a lot of the benefits of TDD are actually the avoidance of bad practices. For example, TDD helps make code testable, because it leads you down a design path that has to be testable, because you’ve started with a test. However, once you’ve learned the sorts of things that make code testable or hard-to-test, and you’ve formed habits and instincts that avoid untestable code, is it necessary to write tests first in order to avoid untestable code? No.

That isn’t an argument against TDD. It’s an observation that the world of programming has moved on since the early days of TDD and maybe it’s time for the TDD books and tutorials and guides to stop presenting TDD as the solution to a problem but more as a mechanical means to solve a problem. An approach that, through practice, teaches you the skills you need to avoid that problem through insight rather than rule-following.

Second, rather than recommending that you should “write tests first”, I think there is a much bigger recommendation that speaks to a wider topic than the TDD debate: You should “put tests first”.

Put tests first in the sense of prioritising tests. Elevate tests to the most important level of significance. Care about tests more than anything else.

Write tests first. When you look at new code to try to understand how it works and what it does, start by reading the tests. When you do a code review or walk-through, start with the tests. When you begin a new feature, start by outlining the feature-level test scenarios. Make test quality the top priority for your team’s improvement process. Consider the tests to be the only reliable and meaningful specification of correct behaviour for your system. Give test code the same care and attention to clarity, expressiveness, robustness, precision as you would give to production code (though there are distinct differences in the range of important factors and the approach to satisfying them in tests versus code).

As one of my favourite Uncle Bob examples puts it, if all your production code was on one disk, and all your tests on the other, and the building is on fire and you can only save one disk, which one would you save?

The tests.

Never mind the “should you write tests first” debate, the really important principle is that you should “put tests first”, above all else.

When you elevate tests to that priority, then you will lead and drive your entire development activity from tests at every level, in every aspect and in every stage.

Posted in Uncategorized

Unit Testing Freemarker Templates

I encountered a defect arising from a Freemarker template, fixed it, and then wanted a better way of testing it than firing up the app and checking the effects visually in a browser. Ignoring for a minute whether or not a Freemarker template should ever have anything in it that needs unit testing, let’s see how to do it if we want to.

Code for this is available in GitHub at

Feel free to copy and do what you want with it.

To unit test a Freemarker template you need to configure Freemarker, load the template, process the template against some model, get hold of the output as text or HTML, and make assertions against it.

That’s a lot of engineering that isn’t relevant to the test, so let’s hide it with a JUnit Rule. Now we can specify a JUnit Rule that will load a given template, eg:

public FreemarkerTestRule template =

new FreemarkerTestRule("src/test/java", "sample.ftl");

And in our test we can get hold of an XML representation of the output like this:

Source theXml = template.xmlResponseFor(model);

Finally, we can make assertions against the XMl, using Hamcrest XML matchers, like so:


The test case can now focus on setting up the model for this scenario, and asserting against the output, with all the engineering work out of sight. Using Hamcrest XML matchers means that your test can be reasonably well-protected from future changes to styling and layout aspects of the output, homing in on specific elements with whatever your XPath skills are. This is important because we don’t want our unit tests to break if page layout changes.

Unit tests? Well, OK, no, not really. They are after all loading Freemarker template files, but close enough. Loading the templates and then processing them may be noticeably slow though, so these tests should either be marked as integration tests or grouped with slow tests so that they don’t cause your commit build to get slower.

The JUnit Rule will also let you get hold of the template output as an HtmlUnit HtmlPage, so that you can make use of all the support HtmlUnit offers for testing HTML.

An interesting bonus is that an HtmlPage has an asText() method, which gives you a text representation of what the HTML would look like. This is pretty cool. Eg for some HTML like this

<table id="exam-dates-table">
<tr class="odd">
<th>Extra Info</th>


<tr class='disable'>
<input type="radio" value="2014/06/01" disabled> Sunday
<td>Jun 01, 2014</td>
<td>'Extra Fee Charged'</td>

<tr class='enable selected'>
<input type="radio" value="2014/06/02" checked="checked" > Monday
<td>Jun 02, 2014</td>
<td>'Extra Fee Charged'</td>

<tr class='enable'>
<input type="radio" value="2014/06/03" > Tuesday
<td>Jun 03, 2014</td>


You’d see a text representation like this:

          Day     Date          Extra Info
unchecked Sunday  Jun 01, 2014 'Extra Fee Charged'
checked   Monday  Jun 02, 2014 'Extra Fee Charged'
unchecked Tuesday Jun 03, 2014 

Being able to do that while you’re developing, without having to fire up the app and poke it through browsers, is potentially very useful.

One thing this sort of testing will do is force you to decompose your FTLs into smaller independent macros, because to execute a test against one requires constructing every aspect of the model that your FTL expects to have available. A smaller macro means a smaller model, which means you can write tests without loads of unnecessary faff.

So, is it ever a good thing to be testing your Freemarker templates?

Well, if you have complex logic that is very clearly related to the presentation, then yes it probably is. The thing is to be sure that your logic really is all about presentation and wouldn’t be better pulled down into code. Pulling the business rules into code and adding suitable presentation-agnostic attributes to your model, is likely to mean that your presentation logic loses any complexity that would require testing.

If you find that you do have complexity and would rather test it in isolation, automatically, then this approach will help.

If you want to unit test Freemarker templates, then use this JUnit Rule and a bit of Hamcrest, and your test cases will be a few expressive lines that focus just on the test scenario, with all the engineering work of loading and processing templates and converting output won’t cause any distraction to the readers of your tests.

With XPath, your tests can be reasonably resistant to unimportant changes to the HTML.

The fact that it is likely to encourage a clearer decomposition of templates into macros etc is also a good thing.

Get the working code here

NB I should really have made this, at least the template loading part of it, a ClassRule rather than a TestRule. “Exercise for the reader”…

Posted in Clean Test Code, Hamcrest, JUnit, JUnit Rule

Hamcrest JSON Matchers

I couldn’t find a Hamcrest matcher for JSON that worked how I wanted, so I adapted the JsonAssert library.

This allows you to use dot notation to specify the path to a node in a json string, and use any Hamcrest matcher to check the contents of the string at that node.

For example, given a Json string like:

 {"someThing": {
    "innerThing": {
        "partOne": "abc",
        "partTwo": 123


You could call

  hasJsonPath("someThing.innerThing.partOne", equalTo("abc")));

  hasJsonPath("someThing.innerThing.partOne", greaterThan(100)));

That’s not massively different to the syntax you get with JsonAssert, but I prefer it. The equivalent with JsonAssert would be

  .assertThat("someThing.innerThing.partOne", equalTo("abc"));

which doesn’t seem quite so fluent.

Note that the matcher you use for the value of the required node must be a matcher for the unmarshalled type.

In other words, if the node contains “abc” then a Matcher<String> can be used, and if it contained 123 then a Matcher<Integer> could be used.


import static com.jayway.jsonassert.JsonAssert.with;

import org.hamcrest.Description;
import org.hamcrest.Matcher;
import org.hamcrest.TypeSafeMatcher;

public class JsonMatchers {

  public static <T> Matcher<String> hasJsonPath(
    final String jsonPath, final Matcher<T> matches) {

    return new TypeSafeMatcher<String>() {
      protected boolean matchesSafely(String json) {
        try {
          with(json).assertThat(jsonPath, matches);
          return true;
        } catch (AssertionError ae) {
          return false;
        } catch (Exception e) {
          return false;

      public void describeTo(Description description) {
          .appendText(" JSON object with a value at node ")
          .appendText(" that is ")

OK, that’s a bit of a quick hack, and it looks very ugly formatted to fit this page, but it does the job. It gives you easy to read assertions without a load of awkward engineering just to deal with the json, or clumsy string matching that isn’t precise enough and causes tests to be brittle and not expressive.

Hamcrest is cool.


PS you’ll need the json-path-assert dependency


Posted in Clean Test Code, Hamcrest

Why Clean Code Matters

An old favourite:

For three days and nights the Java master did not emerge from his cubicle. On the fourth day the monks of the temple sent a novice to inquire after him.

The novice found the master at his whiteboard, contemplating a single dataflow diagram. The novice recognized it as a minor component of the vast system that the monks were contracted to maintain. Politely, the novice asked what the master was working on.

The master replied: “There is a defect, and I am considering the best way to repair it.”

The novice said, “You preach often about the importance of setting priorities. How, then, can you obsess about something so tiny and unimportant?”

continue reading at

Posted in Uncategorized

Dangerous Words

A department trialing the use of agile in a predominantly waterfall organisation sent a status update to the traditionalists in program management.

It contained the phrase “backlog of technical debt”.

To us in the Scrum world, that means one thing, but if you imagine you know nothing about “Technical Debt” or the idea of a “backlog”, then it sounds like quite a bad thing, doesn’t it?

Posted in Uncategorized

Scrum Task Board Tips

Call it your story wall, call it your agile board, call it your scrum wall, call it what you like. I call it a task board, and I like to keep it simple.

If you’re in a situation where you can have a physical task board, then you’re winning. Feel free to do stuff electronically in Jira or whatever, because that’s probably how you do things, but beware that anything you do electronically not only duplicates what you do on the board, it also tends to suck you into doing things in a tool that would be more effective face-to-face.

Make your physical task board the one source of truth, and consider everything else supplementary, optional and secondary.

Do a Google image search for scrum task board and you’ll see a hundred different styles. Which way works best?

Well here’s the most important rule for setting up your task board:

Tip 1 – Don’t add it until you need it

Start simple, with just the essentials. Don’t add something that you think you might need. Wait until you discover that you really do need it, then add it. You might think you need a holding area, or a burndown chart, or a “days remaining” counter, or a standard template for your story cards, or a “QA In Progress” column but you probably don’t. You don’t need to write names of the “assigned” person or the story reference number on task notes. If something isn’t absolutely essential, then leave it out. When you encounter a deficiency and see a way of addressing it by changing your task board, then try it. but only then.

Tip 2 – You only need three columns on your board

  1. To Do.
  2. Doing.
  3. Done.

Either someone is working on part of the story, or no-one is. If no-one is, it’s either finished or it wasn’t started yet. That’s all.

Tip 3 – You only need three things on your board

  1. Story Cards
  2. Task notes
  3. Avatars

Tip 4 – Story cards are just reminders

The story card must have just two things

  1. A meaningful title
  2. The relative size in story points

The story card may often have a reference to a backlog ID, but it’s not essential. The card is just a reminder. If it has a clear, succinct title that means something to the team, then the team will know what that card is all about. It’s current work in progress, it’s fresh in everyone’s mind. It’s a short-lived, throwaway index card. You don’t need to write acceptance criteria (write those in your tests), you don’t need names of developers and product owners (that’s just trivial admin and it doesn’t help anyone). A descriptive title, and a size. The title doesn’t have to be in any particular form, like “As a… I want… So that”. You’ve already gone through the story in detail in your backlog grooming and your sprint planning. On the task board, it’s just a label.

Tip 5 – Use task notes to indicate work done versus work remaining

The task notes are the teams list of little things they have to do to complete the story. Getting the team to produce their detailed task lists in advance is an effective way of revealing any uncertainties or difficulties at an earlier stage. Whereas story cards are index cards, task notes should be post-it notes, because they are numerous, very temporary and subject to change every day. They don’t need to be neat, they don’t need to be kept. incidentally, learn the correct way to remove a post-it from the pad, and they will stay stuck for as long as you want. Do it wrong, curling up the sticky part, and they fall off. To do it right, place the stack on your desk and pull the note almost horizontally towards you while holding down the lower stack, until the note parts from the stack without curling up.

The great thing about using post-it notes for your tasks is that because each note represents a small chunk of work that can be completed in an hour or two, the relative weight of notes remaining “to do” against a story versus “done” indicates progress at a glance, without needing to create a burndown chart.

Get a pack of pink post-its for defects and blocked tasks. It’s a great way for a scrum master to quickly grasp the relative health of the current development, and to home in on problems that the scrum master should be addressing.

Tip 6 – Keep tasks in horizontal swim lanes

The story card goes at the left hand side of the board. Task notes stay in line with that story as they progress. Then you can clearly see which tasks go with which story. When all the tasks are done, the story card moves to done, and the task notes are discarded.

Tip 6 – Put higher value stories higher up the board

The most important story should be at the top, the least at the bottom. Then you can quickly tell if the right things are being worked on. Lots of tasks moving across the board lower down? Problem. High up stories “done” and middle stories “doing”? Great. Lots of stories with tasks in progress but no stories finished? Problem.

Tip 7 – Use magnetic avatars to show who is working on what

If you use a movable magnetic marker to indicate each team member, then you can keep stories and tasks lined up, and the markers are easy to move. If someone isn’t working on something it becomes glaringly obvious. If too many people are working on one thing, it becomes glaringly obvious. As a Scrum Master you can immediately see if people are doing the right things at the right time and mitigate problems before they really start

Tip 8 – Physically move task notes and avatars in the stand up

Make the task board physically accessible and encourage team members to physically move things during the stand up. It makes it clearer for other people, easier to follow what the person has been doing. It helps the person working on that card keep track of what is to do. It keeps the progress and status clear for the scrum master.

Tip 9 – You don’t need a burndown

If your high value stories are at the top, the relative weight of tasks remaining is visually represented by post-its, and the position of post-its shows what is being worked, and it is updated in every stand up, then at a glance you can understand progress. The burndown is just admin, it gives a false sense of accuracy, and it too often doesn’t reflect reality because the task notes are too high-level and not updated. It’s much more effective to encourage team members to use more detailed task notes and to update the regularly. Using post-it notes is so easy to do that it doesn’t put people off. Let them own their task notes, let them do what they want, don’t question their estimates, just let them spend a little time and a little thought to plan slightly ahead of where they are and not be taken by surprise. As a scrum master, it’s much more important that you understand how the developer seems to be getting on, than it is to add up some numbers for an unrealistic chart that nobody looks at. So often the “actual” line on a burndown shows such a remarkable degree of match to the required slope that it suggests a certain degree of sub-conscious “cooking the books”. Starting with the answer you want and working backwards doesn’t give any predictive power.

Some say that the task board is only a snapshot and the burndown gives a trend, but if your stories are small enough then you have all the insight you need at a glance. Put more effort into properly decomposing stories to smaller, independent pieces, and forget the burndown.

The burndown, as usually done, is calculated from remaining task hours. This means it can show apparently desirable progress even though no stories have been completed. Ten stories all “nearly finished” with one or two days to go is dangerous, yet the burndown may look healthy. Smaller stories moving quickly to “done” gives a much better view of progress.

Most experienced scrum teams have given up on burndowns and don’t miss them.

Tip 10 – Use consistent language for your story cards

Story estimating is relative. When estimating a story it helps if you can compare it to similar stories. Your capacity planning is based on the premise that with this sort of work, and this team, in this environment ,with the sort of things that tend to get in our way, and the estimates we tend to give, then we can do this any points per sprint. That becomes easier and more reliable when you can identify similar stories.

One thing that helps is if you use consistent language for similar stories. You tend to get recurring types of stories that relate to similar parts of the product and involve similar work. If you name them consistently, it becomes easier to categorise. Eventually, sizing a story in sprint planning becomes a 30 second effort because you’ve internalized the categorisation of similar work items. Whether the words you’re using are screen or form or page or dialog or preference or option or whatever, if you can quickly classify a story and say “it’s a new screen like the xyz screen last sprint and the abc screen from last month”, and you know they were both 5 pointers, then you have another 5 pointer.


Scrum task board layout – see an example of this layout at






Posted in Agile

Bean Mapping With ModelMapper

I’ve never found a satisfactory method for mapping bean properties. They always add too much ugly set-up code, especially for conditional mapping or type conversions.

Perhaps ModelMapper is the best so far.

Let’s say you have a source bean with a these properties plus getters and setters:

public class SourceBean {

    private String firstName;
    private String lastName;
    private int age;

And you have a destination bean with these:

public class TargetBean {

    private String fullName;
    private Boolean oldEnoughToVote;

Now you want the bean mapping to do the equivalent of

      source.getFirstName() + " " + source.getLastName());


    if(source.getAge() >= 18) {

Then the ModelMapper solution looks like:

    private PropertyMap<SourceBean, TargetBean> mapping =
      new PropertyMap<SourceBean, TargetBean>() {

      Converter<SourceBean, String> concatWithSpace =
        new Converter<SourceBean, String>() {
        public String convert(MappingContext<SourceBean, String> context) {
          SourceBean src= context.getSource();
          return src.getFirstName() + " " + src.getLastName();

      Converter<Integer, Boolean> over18 =
        new Converter<Integer, Boolean>() {
        public Boolean convert(MappingContext<Integer, Boolean> context) {
          return context.getSource() >= 18;

      protected void configure() {

    ModelMapper modelMapper = new ModelMapper();

    TargetBean result =, TargetBean.class);

I’m still not so sure that’s gained us very much, and some of it is rather uncomfortable to look at. For example ,can you work out why we pass null to “setOldEnoughToVote” and “setFullName”, without reading the manual and the developer mailing list?

But if you were to extract your re-usable common converters, and if you had a load of properties that just map with the default property mapping, then you have something a bit simpler and perhaps saving some tedious code:


    private PropertyMap<SourceBean, TargetBean> mapping =
      new PropertyMap<SourceBean, TargetBean>() {

      protected void configure() {

    ModelMapper modelMapper = new ModelMapper();

    TargetBean result =, TargetBean.class);

….and perhaps, ultimately, at the point of use, where you really don’t want a load of low-level mapping code obscuring the expression of what is really happening in the higher-level process, you would just have:


TargetBean result =, TargetBean.class);


Hmm. It has promise, but I need to do further experiments with this….

For example, how to use ModelMapper’s “not null” condition to handle null firstName or lastName, without adding so much clutter that you don’t gain over just doing it manually.

If you’re using ModelMapper extensively, I’d love to hear your insights into doing it cleanly.

Posted in Java

Loading Resources In Tests

Some level of tests, let’s call it integration testing because it’s not technically unit testing, often need to load resources like XML samples from the file system.

A talented developer I worked with recently, came up with this nice use of a JUnit Rule to clean up the resource loading, making it reusable and removing the irrelevant detail from the test classes. He also made it cope with different types of resources from simple Strings to XML documents.

Simply include your rule like this:

public TestResourceRule resources = new TestResourceRule(this);

Creating a Rule like that is pretty simple. See

Then you can annotate the fields that should be created from some resource, like this.

private Document xmlDocument;


Now you can just use that Document in your test, and neither you nor your future readers have to even see any file loading code. Nice.

Posted in Clean Test Code, JUnit, JUnit Rule

Names Should Be Expressive When Writing Tests

It is still surprisingly common to see test methods named “test” followed by the name of a some method that’s going to be called. Sometimes you get a few words mentioning some other internal detail, like

public void testLogin_Null(){...

This sucks. I’m amazed people still do it.

It doesn’t say anything (or only very little) about the scenario.

It doesn’t say anything about the behaviour required in that scenario.

It says “test” when I already know it’s a test. The annotation, for example. And the fact that it’s in a test case class.

It says the name of one method that this test calls, ignoring others, and sometimes that method name doesn’t even exist any more.

It tells me that some test data is going to be null, but it doesn’t tell me anything about what should happen in that case, or why.

There’s no excuse for this. It’s so easy to pick a name that expresses what scenario you’re trying to construct, and what behaviour you expect from your code in response. Given an expressive name, others can confirm that the scenario is as intended,, that the behaviour is as intended, and they’ll be more confident that they’re complying with the original intent as they make changes in related code, or fix bugs.

The First Rule Of Test Naming – Say What The Code “Should” do

public void shouldThrowExceptionWhenUserNameIsNull(){...

This tells me, the reader, what the component should do in a particular circumstance. I can read all the test method names to understand the full range of responsibilities of the unit-under-test.

public void shouldCompleteTimeSheetWhenItIsFridayMorning(){...

public void shouldDrinkBeerWhenItIsFridayNight(){...

public void shouldNotGetOutOfBedBeforeTenWhenItIsSundayMorning()...

public void shouldNotWorryAboutLongNamesInTestMethods(){...

The Second Rule Of Test Naming – Describe The Behaviour, Not The Methods Called

This is the same as the first rule, just stated differently.

There is not a one-to-one relationship between methods in the unit-under-test and the test cases. Don’t write test cases by writing one for each method that can be called.

Anyway, you can’t do that if you’re writing the test first, and you are writing the tests first, aren’t you?

Instead, write methods that describe the behaviour of the unit-under-test. When you’re thinking about what the component should do, when you’re thinking about the edge-cases and awkward conditions you’ll have to code for, when you’re discussing requirements with stakeholders, document all these things as test methods with readable names that express what should happen in particular circumstances.

The Third Rule Of Test Method Naming – Imagine Being The Reader

This is the same as the first and second rules, just stated differently.

Imagine you’re the person who will be reading these tests for the first time in the future. What will they want to know? How can you write it so that you are telling that reader, very clearly, what the unit-under-test is meant to do in all the various circumstances it has to contend with. What business rules does it have to obey? What are the requirements? What are the special cases and edge conditions?

Don’t try to tell the reader how the component fulfils these responsibilities – do that in the implementation code.

Tell the reader what the responsibilities are.

Don’t write tests from the perspective of things you know about your implementation and want to check, write them from the perspective of the reader wanting to know what behaviour they have to preserve to make sure this thing still behaves correctly.

Instead of:

public void testSignUp_null(){...

public void testSignUp_blank(){...

How much more expressive and clear is it to say:

public void shouldThrowExceptionWhenUsernameIsNotProvided(){...

public void shouldCreateAuditEntryWhenUsernameIsNotProvided(){...

public void shouldStillAllowSignupWhenPhoneNumberMissing(){...

The word “when” is should’s ally when it comes to test names. shouldDoThisWhenThat. Some say it reads better if you start with the “when”, and that’s just fine.

public void whenTestNameIsBadShouldReplaceIt(){...

I prefer to put should first, because it sounds more forceful, more direct, less passive:

public void shouldReplaceTestNameWhenItSucks(){...

The Only Real Rule Of Test Method Naming – Always Use “should”

This is nothing new. I thought the “testMethodX” style died out in the last decade. Why do I still see it everywhere?


John Ferguson Smart in his excellent book “BDD In Action”, writes:

Language influences thought patterns. The words you use influence the way your message is perceived. Consider the following subconscious dialogue that our brains go through when reading a requirement:

Requirement: A new Frequent Flyer member status must be Bronze.
Brain: Yes Sir, very good Sir, right away Sir!


Requirement: A new Frequent Flyer member status should be Bronze.
Brain: Should it? Are there times when it might not be Bronze? What about special deals from marketing where they start out as Silver? What about if they’re transferring points from a partner airline?

Only one word has changed, but the way we receive the information is quite different. The word “should” invites a question: “should it?” “Should” helps us justify why we want a class to provide a particular service, or to perform a particular task. Should also implies that any requirement, at any level, can be questioned, leaving communication channels much more open.

See a good discussion of different naming strategies here –

Posted in Clean Test Code, JUnit

Great Expectations

What’s the best way to handle expected exceptions in JUnit?

If you want to inspect the contents of the exception and make assertions against it then there is only one clean way; Use the ExpectedException JUnit Rule.

I’ve seen it used badly, though. Think about how to name the field. I see no better name than simply “thrown”. When you use it in your tests it reads perfectly sensibly as




is a bit clumsy, and


is not expressive.

The other mistake people make is to not tidy things up with reusable custom Hamcrest matchers. They’re so easy to create (a future post on that issue coming soon), for your common domain-specific exceptions they can be easily reused, and they make the code expressive and simple. You can simply use any Matcher<?>.

One of the simplest options is to use the Hamcrest “hasProperty” matcher to assert against properties of the exception.

public ExpectedException thrown = ExpectedException.none();

public void shouldGoWrongSometimes() throws Exception {

  thrown.expectMessage("That was bad input");


  thrown.expect(hasProperty("xyz", equalTo("oh dear")));

  testSubject.use("bad data");
Posted in Clean Test Code, Hamcrest, JUnit, JUnit Rule