When I first learned unit testing and Test Driven Development, test frameworks were very light and mocking frameworks were just coming into the picture. I was writing a lot of C++ back then, and if I wanted a mock I would implement my class interface and link my test to the mock version instead of the production version. Fast forward 15 years and we have extraordinarily advanced mocking frameworks (and I mostly write Java these days). While new libraries have come and gone, Mockito remains my favorite. It has a good balance of a straightforward syntax and a very complete feature set. In fact, a bit too complete if you ask me.
While there are a few ways to use Mockito (or other similar frameworks), there are two patterns that I see most frequently. The first uses Mockito to generate a dynamic mock object with a call like:
when(mock.method()).thenReturn(expectedValue).
The test interacts with the mock object and uses an assertion library to validate the result of some logic under test. The other pattern uses
verify(mock, times(1)).method()
as an assertion to determine if a specific method was (or was not) called using the provided parameters or parameter surrogates. This second form, which I call “verify”, is what I would like to discuss. In general, verify based tests do not meet the criteria of good tests and should be avoided.
What are we really testing when we use verify?
Verify asserts that the test author and developer are in agreement about what code should be written. In many cases, a single developer writes both the test and code, and it is at best a check against a typo in either the test or the code. In pairing situations, it can serve as a stand-in for the conversation that should be taking place in the pair. I find more value in testing “business things” than “code things”. To break it down, let’s take a look at the typical reasons that we write tests and whether or not verify satisfies them:
- Validate that the product meets the business requirements
- Give cover for future change
- Document intent
Validate Business Requirements
The basic purpose of most tests is to validate correctness, and correctness is defined by meeting the business requirement. The only reliable way to accomplish this is by asserting that the result of a piece of code is the right business outcome. Verify does not meet this requirement in most cases. If the call being verified does the right thing for the business requirement, then the test will be correct, but there is nothing in the test that guarantees it. Assumptions could be missed, the behavior could be different, or the cumulative effects of calls produce a very different effect than intended. Verify calls validate that the code was written as expected, but not that any business logic is actually correct.
Cover for Future Change
Heavy use of verify results in one of the most pathological cases of unit testing, and one often cited as an argument against it. Detractors say that every time they refactor their code, they also have to rewrite all of their tests, causing them to work twice as hard rewriting twice as much code. Since refactoring should only change the structure of code but not the function, this should not be the case. Verify calls are tightly coupled to implementation, so they do not provide functional coverage which is essential for refactoring. Having to make significant change to both the code and the test is more of a rewrite than a refactor, and comes with increased risk.
Documentation of intent
Similar to “Validate Business Requirements”, documentation of intent is only effective when the tests demonstrate the expected business outcomes of the code. Tests that rely on verify show what other code we intended to call, but not why or how it impacts the business outcome. A developer new to the codebase cannot determine why those choices were made or whether the code would be appropriate for reuse in other circumstances; only what was expected to be called directly.
Kevin Hickey's Blog 