Comparing Java Mock Frameworks - Part 1: The Contenders

Update 2010/10/06:  I removed rMock from the list because it hasn’t been maintained for a while. I might take a look at it some other time.

Update 2010/09/24: I removed the comparison of spying capabilities. Indeed, I believe that such frameworks are not intended for state verification but behaviour verification and stubbing only.

This is the first instalment in the series Comparing Java Mock Frameworks. Here, I present the frameworks that I intend to compare. The whole series consists of the following posts:

• Part 1: The Contenders
• Part 2: Creating Mock Objects

---

Since a while ago, I wanted to line up the major mock frameworks in Java. I am used to Mockito, which I must say I utilize a lot when I have to deal with Test Doubles.

 

I must confess that I very often use a mock framework instead of writing my own test doubles, even in trivial cases. I find it quite handy because, I don’t have to care about those mocks if I later change the interface of the mocked class. Others prefer writing their own mocks in many cases.

I use mock frameworks when I need to impersonate a class for the purpose of testing the behaviour of the class under test. I mostly follow the following patterns:

• Test Stub: I need to replace a class in order to return values when the class under test calls some methods on the object. I can also use the test stub to throw an exception to test error handling in the class under test.
• Mock Object: I need to replace a class in order to verify how the class under test interacts with the mock object.

In a mini-series of posts, I will compare mock frameworks for Java to determine which ones best suits me. Here is the list of the different ways I will use them for the comparison:

Instantiation and overriding	Obtaining a test double for an interface Obtaining a test double for a class Obtaining a test double for a final class Overriding of final methods in a test double Overriding of static methods in a test double
Test Stubs	Returning a single value Returning multiple values orderly Throwing an exception in a non-void method Throwing an exception in a void method
Mock Objects	Verify the number of invocations of a method Verify the order of invocations of a method Verify the arguments of an invocation of a method
Fluency of the API	The closer to natural language, the better Ease of set up
Specifics	Features that make the framework stand out of the crowd

Here are the frameworks that I will compare:

Framework	URL
Mockito	http://mockito.org/
EasyMock	http://easymock.org/
Mockachino	http://code.google.com/p/mockachino/
PowerMock	http://code.google.com/p/powermock/
jMock	http://www.jmock.org/
JMockit	http://code.google.com/p/jmockit/
Unitils	http://unitils.sourceforge.net/

I will evaluate the frameworks based on my needs. I won’t claim that everyone should, from now on, use the framework I’ll choose

References:

• xUnit Patterns site
• Mocks aren’t stubs by Martin Fowler: Discussion on state and behavior verification styles
• Mocking mocking and testing outcomes by Uncle Bob: Discussion on the (over)utilisation of mock objects

About these ads