This is an interesting problem I've been running into at work. Sometimes we would try to write a test against the system or a component and we would literally have too many choices of how to write it or where to "intercept" or "inject" our stubs or mock objects. In a way, the software is almost "too testable" in that there is usually at least 3 ways to accomplish a similar level of isolation.
What it comes down to is the fact that for a real system, there are different "depth" layers where indirection can be injected. For example, Component X needs object Y that relies on object Z. If we were testing component X we could "replace" either Y or Z. Some would argue that replacing one level too deep is making this an integration test, but as long as its all running in memory I don't see a real problem with that. I gave up on trying to do "pure" unit tests for 100% of the time a long time ago.
Many times, it is much more complex than this. For example, replacing object Y instead of Z (at the closes layer to the object under test) can also lead to problems if:
- The contracts between X and Y are very chatty, than stubbing out Y could lead to a very hard to read or very fragile test
- You are trying to test that interaction occurs between Y and X than that might be an over specified test. If replacing Z would lead to a more state-base test that I'd choose that over replacing X. (shortly: I'd rather use stubs over mocks so that my tests become less fragile. some layers require mocking and some are very simple to just stub out)
Here's a thought or two:
if you one one layer farther away than the one you are using, and it does not require mocks to test, that try to use that layer and forget the inner layer?
If you treat mocks as a smell for over specified tests that will force you to find a way to test the same thing using a stub. I find that sometimes you have no other choice but use mocks (when your app under test has a one way out communication with a component) but in all other cases, I am starting to look at it like a design smell for my tests (over specified tests)
getting back to the original problem (too many ways to test the system) I am trying to think if this is a good or bad thing. The only bad here is that someone who is testing the system for the first time has no real guidance on where to replace things (except look at other tests) which can make writing the first few tests really hard to figure out.
The fact that we are using a sort of auto mocking container in our tests is both a blessing and a bit of a curse. Currently i find that it saves more time that it takes away. the tests and their dependencies are relatively more readable since we are using a more explicit version of a test container that requires you to explicitly state what will be replaced (it just doesn't tell you who requires that component).
I can already hear the pundits say "well that means you have a design smell". The only thing that smells here to me is that because we followed strict DI and interface based injection rules, we ended up with a system that is not just made of lego parts (a good thing), the lego parts or too darn small!