Mocking Frameworks Considered Harmful

Article written by: Michael Boldischar
December 1st, 2011    Posted in Developer Resources, Java, Uncategorized

I’ve been on several projects where developers use Java mocking frameworks.  The most popular of these frameworks is called Mockito.  I’m pretty open to giving new technology the benefit of the doubt.  I used it for several months and found reasons to consider mocking frameworks harmful.  I constantly get asked why I don’t use them by other developers.  I always say that one day I’ll write a blog article on the subject.  Here it is.

Here are the reasons I consider mocking frameworks harmful:
1. When you need to use mock objects , you are saying that you are unable to use standard unit testing on your code.
2. The tests you develop with mocking frameworks are tightly coupled to the internals of the code.
3. The code you test with mocking frameworks is hard to refactor.

1. When you need to use mock objects, you are saying that you are unable to use standard unit testing on your code.

I’ve seen systems where testing using mock objects was the only option.  This is usually a sign there is a major design problem in the system.  The most common cause is too much coupling in the code base.  This can mean too much coupling in the domain layer or between layers (e.g. domain and infrastructure).  Or, classes have references to “god” objects (i.e. objects that have too much responsibility).

The most important part of Object Oriented Programming is build a solid domain model; it should be independent of other layers.  See Eric Evans diagram for a good example of domain-driven-design architecture.  This domain model needs to limit the number of points where it interacts with infrastructure services.  I’m going to use the example of a database infrastructure service.  The domain model should have an interface database through some type of “repository” object or facade.  A common use case for mocking is to test classes that interface with a database.  This becomes a major obstacle in systems where JDBC objects are sprinkled around in the domain layer.  These objects need to be bundled in a cohesive class or package to reduce the coupling between the domain layer and infrastructure layers.  The repository objects that communicate with the database need to be interfaces. This allows you to change database technologies later in the project because all the JDBC code is in one area.  When these repository interfaces are trivial, you can even do things like create in-memory concrete instances of repositories for testing.

You might say, “Okay that’s great, now my database calls are all in the same class.  But I still need to unit test them!”  Here’s the deal my young apprentice.  Testing interactions between larger systems/components is not unit testing.  It’s called integration testing.  That’s why it’s important to have a suite of integration tests to exercise all the major functions of your system.  Your integration tests should be run regularly during development.  Consider using a Linux cron job with automatic email reporting.  When you have a hammer, every problem looks like a nail.  Remember that you have a toolbox.  One of my favorite tricks in integration testing is using Python scripts to call interfaces and aggregate results.

2. The tests you develop with mocking frameworks are tightly coupled to the internals of the code.

The first point wasn’t enough to convince you that mocking frameworks are considered bad practice, eh?  So you went to work the next day and create the class called “SuperAwesomeHelperUtility”.  Yes, it’s more or less a god class that knows everything about sandwiches, email systems, and databases.  But you don’t care because your mocking framework will help you test this utility.  Please don’t attempt this type of coding at home or work.

Here is one of your unit tests:

@Test
public void testMakeTurkeySandwichEmailTomAndStoreSandwichData() {
   Connection mockConnection = mock(Connection.class);
   EmailServer mockEmailServer = mock(EmailServer.class);
   SandwichMaker mockSandwichMaker = mock(SandwichMaker.class);
   SuperAwesomeHelperUtility utility = new SuperAwesomeHelperUtility(mockConnection, emailServer, sandwichMaker);
   utility.makeSandwichEmailFriendAndStoreSandwichDataInDatabase("tom@tomiscool.com", "turkey", "turkeydb");
 
   verify(mockConnection).executeUpdate("insert into sandwich values (\"turkey\")");
   verify(mockSandwichMaker).createTurkeySandwich();
   verify(mockEmailServer).email("tom@tomiscool.com");
}

First, the test above isn’t a complete unit test.  It’s just an example of the types of calls that cause tight coupling between code and unit tests.  In the example above, the internal calls in the method “makeSandwichEmailFriendAndStoreSandwichDataInDatabase” have been exposed by the mock objects passed in the constructor of the “utility” instance.  The concept of encapsulation is being broken by verifying the calls to external components are made.

Again, this is an example of bad design.  But this is the trap that mock framework users fall into.  It becomes their crutch when implementations are hard to unit test.  Soon the practice becomes common in a group because it somewhat solves the problem.  This type of viral malpractice needs to be identified early and stopped.

3. The code you test with mocking frameworks is hard to refactor.

Have you ever tried refactoring code that has all the calls to other components mock tested?  It’s impossible.  Especially when there are thousands of verify calls to other components in a unit test class.  That doesn’t sound very agile, does it?  You might have to throw away the entire unit test and start over if you decided to refactor the “Connection”, “EmailServer”, or “SandwichMaker” classes or method signatures found in item 2 above.

Everyone knows how important unit testing is during development.  The point of unit testing is to test the functionality of one unit of code.  That usually means testing at the method level.  In my experience, good code can be easily tested at the unit level because it doesn’t depend on lots of other services for testing.  Here’s another example.  Say you want a function that tests adding two numbers together:

// adds two numbers together and returns the result
public int add(int x, int y) {
...
}

That’s as simple as it gets in programming.  And that’s the whole idea!  You want to write simple methods in Object Oriented Programming.  You also want to reduce dependencies between classes.  Your class for adding two numbers together should not have a dependency on the “SuperAwesomeHelperUtility”.  If it does and you need to mock it for testing, you’re doing something wrong.

2 Comments