Flexible Mocking

Unit tests, in particular the assertions within, should be no more and no less specific than they need to be. Make a test too loose, and it will be possible for someone to make changes to the code in a way that violates the specification but doesn’t actually break the test. Too tight, and changing a hidden internal detail can break the test when that detail isn’t actually part of the externally visible specification.

Sometimes this sort of thing is a simple mistake like forgetting to call verify after using mocks, so that they aren’t protecting the behaviour you defined. Seeing that mistake made so often was one of the motivations behind the EasyMockRule project https://code.google.com/p/easymockrule/ which automatically verifies the mocks you have used so it can never be overlooked.

Sometimes it’s a trivial Java mistake (or is it in fact just laziness and lack of care?) like this example:

String[] expected = new String[] { "1", "2", "3" };
String[] actual = doSomethingWith(expected);

assertTrue(expected == actual));

Obviously I’ve simplified that for illustration, but I came across an example like this when the test failed after I’d made a purely internal change like copying the incoming array instead of passing it through. The original author didn’t really mean to assert that the input and output arrays were the same array instance, what they really meant was to check that the contents were the same. So is this more appropriate? :

assertTrue(Arrays.equals(expected, actual));

Actually, this is still more specific than made sense for this code. What was actually significant for specifying the required behaviour was that only the contents should be the same. The order doesn’t matter.

So let’s use a Hamcrest matcher like this:

assertThat(actual, containsInAnyOrder("1", "2", "3")));

Now we have a test that precisely specifies the required behaviour. Not too loose, not too tight. What is more, it presents it in a single line of code that clearly, obviously, unambiguously states the author’s intention, so that the person making changes doesn’t have to dig through the code and try to interpret what the original intention really was, like I had to when that test failed.

As a side-note, look back at that original test. Shouldn’t it have been assertEquals(expected, actual) instead of assertTrue(expected == actual)? Or how about assertArrayEquals()? Or if the identity check was really intended, how about assertSame()?

In that test, this one line, 

assertTrue(expected == actual));

is wrong twice. Whether you’re using JUnit or Hamcrest, use the right assertion and the right matcher to better express yourself.

Under-specified tests allow changes that lead to defects. Over-specified tests are fragile. Ever done some refactoring (changing the internal structure without affecting externally visible behaviour) only to find lots of test failures due to over-specified behavioural verification in mock objects?

In general, if you’re using a mock object to supply indirect inputs to the class under test then you should avoid over-specifying the indirect outputs by verifying the sequence and number of calls and the arguments in those calls. (By the way, if indirect inputs and indirect outputs are unfamiliar terms, I recommend the book X-Unit test Patterns.)

If you use the right kind of mock and set expectations the right way, then you can avoid a too rigid specification that makes for brittle tests. With EasyMock, this means choosing between default mocks, strict mocks, and nice mocks:

  • Strict mock – the order of calls is checked
  • Default mock – the order of calls is not checked
  • Nice mock – the order of calls is not checked, and it will reply with 0, null or false for unexpected invocations

This means that unless there are aspects of the precise sequence of calls that are a part of the specification of the class under test, then you should prefer “Nice” mocks.

Use “Strict” mocks only when you really do intend to specify the indirect outputs of the class.

Similarly, there are different ways to set expectations, and you should generally prefer the most flexible version unless you deliberately intend to be specific. In increasing order of flexibility:

  • .andReturn() – allows one call only
  • .andReturn().times(x) – allows x number of calls
  • .andReturn().atLeastOnce() – allows one or more calls
  • .andReturn().anyTimes() – allows any number of calls including none
  • .andStubReturn() – allows any number of calls including none

Same principle applies to the matchers you use in your expectation setters. Be as specific as required, no more, no less. This is where Hamcrest matchers can come in very useful, which is why I added support in EasyMockRule for really easy use of Hamcrest matchers in expectation setters.

I also really dislike using IMocksControl unless the order of invocation is very specifically and intentionally part of the behavioural contract that your test is specifying and verifying. Using IMocksControl causes brittle tests that fail when inconsequential internal implementation details change without affecting the important externally visible behavioural contract of the component. Some people strongly object to that, but I find that it usually seems to have been used for convenience rather than an explicit intention to specify a precise sequence.


By the way, here’s one approach to using Hamcrest matchers in EasyMock expectation setters. First, you can create a static utility method like this:

 public static <T> T with(Matcher<T> matcher) {
return EasyMock2Adapter.adapt(matcher);

Now you can use it in an expectation setter like this:

expect(code.doStuff(with(hasProperty("blah", equalTo(something)))).andStubReturn(result);

I first learned about this from Dan Haywood – http://danhaywood.com/2009/12/04/easymock-junit-rules-and-hamcrest/


Posted in Clean Test Code, EasyMock

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: