Mockito and strange UnfinishedStubbing problems

Strange errors

Have you ever had an UnfinishedStubbingException or some other Mockito problem, that seemed to be wrong? The code written properly was having some random problems. Like this user:

https://groups.google.com/g/mockito/c/uDKs5cKh61g

org.mockito.exceptions.misusing.UnfinishedStubbingException:
Unfinished stubbing detected here:
-> at SomeClass.someMethod(SomeTest.java:90)

doThrow(new InterruptedException()).when(someCollaborator).someVoidMethod();

The cause

We can expect it to happen during integration tests. I am a strong advocate for integration tests instead of mocking single services and making so called "unit tests". Spring helps us to write such tests with @SpringBootTest annotation for example. Spring will create whole application context and we will be able to interact with its services.

And now, by having the application context running with its async processes and at the same time wanting to provide behavior for some external API clients or @SpyBean services, we can observe the issue.

I have prepared the example. Problem can be reproduced by running "testNotSafe()". The "object" in a test could be a Spring bean annotated with @SpyBean

GitHub: https://github.com/PiotrDuz/mutlithreaded-mockito

Following error will appear:

org.mockito.exceptions.misusing.UnfinishedStubbingException: 
Unfinished stubbing detected here:
-> at org.pidu.MultithreadedMockitoTest.testNotSafe(MultithreadedMockitoTest.java:25)

E.g. thenReturn() may be missing.
Examples of correct stubbing:
    when(mock.isOk()).thenReturn(true);
    when(mock.isOk()).thenThrow(exception);
    doThrow(exception).when(mock).someVoidMethod();
Hints:
 1. missing thenReturn()
 2. you are trying to stub a final method, which is not supported
 3. you are stubbing the behaviour of another mock inside before 'thenReturn' instruction is completed

But the code looks as follows:

Mockito.doAnswer(ans -> 13).when(object).getValue();

We are not stubbing a mock inside a mock, nor we did not forget to specify the return value.

What happens is that while mocking/verifying mockito makes some changes to underlying static context. And when other thread is trying to use the spied bean method, it results in mockito getting lost and throwing errors.

This seems to be known by Mockito team but no fix is yet available. Plain, single-threaded unit tests are fine, so maybe they do not feel much need for additional work. But I would argue that currently integration tests are simple to write (we can setup DB, Message broker and others with Docker and TestContainers) and we will need to run Mockito in async environment more than before.

https://github.com/mockito/mockito/issues/1151

Solution

We can create our own synchronization for mockito method calls.

First lets identify phases of our tests. One is mocking the methods, and second is running the app and waiting for outcome. Of course, once the application context is created, our app starts "running". But we are only interested about the app once we have setup our mocks. So

1. We do not want to obstruct normal operation of our app. Somehow wrapping objects' methods with "synchronized" keywords is too restrictive. It will work, but the app during tests will not behave the same way as on production

2. We need to stop access to objects' methods during the mocking phase (when we are specifying new behavior)

I have chosen the approach with ReentrantReadWriteLock. To use the object, we obtain readLock. ReadLocks do not block eachother, so parallel processes can execute just fine. And only during the mocking phase, we do obtain the WriteLock. This will halt all calls to the Spied or Mocked methods. So we stop the app just to assign new behavior, and then resume.

It can be tested by running "testSafe()":

GitHub: https://github.com/PiotrDuz/mutlithreaded-mockito

Mockito can be extended by providing the plugin class name in mockito-extensions folder. I couldn't find a lot of documentation about it, but glad that some people mentioned it: https://davidvlijmincx.com/posts/writing_higher_quality_tests_with_mockitos_inline_mock_maker/

Notes

• is mocking necessary? External apis can be put behind interfaces, and those could be just implemented for the test.

  • I agree that this is one of the solutions. But then that would be an additional work when preparing test fixture, for every client a test implementation would have to be prepared. Also Mockito has some handy methods to define behavior and then verify it.