Preventing fragile tests – how can isolation frameworks help or hinder your goal?
One of the things that inhibits unit tests in organizations is the idea of fragile tests. a Fragile test is a test that can easily break when the production code changes. That’s not to say that tests should never break as you change production code, but there are ways to minimize this effect so that when tests do break, they break for the right reason.
the right reason is when a feature in production code isn’t working.
the wrong reasons can be many and varied. Of the most common ones is the idea of over specification in tests. The more your test expects of your code’s internal implementation, the more “specified” it is, the more “expectations” it has on the internals of the code. since internal code is more proone to change, so will the test break more often. so the situation can occur that the production code changed, but still gets the job done, but since it’s doing it differently internally, the test breaks even though the end result is still OK.
One of the main ideas in the new Isolator API was to reduce the fragility of tests by default. there are several features in Typemock Isolator that remove lots of fragility from tests by default, compared to other tools.
What do you think they are? (look here for hints) I found at least 8 separate things.
what do other tools like Rhino Mocks, or Moq do to help or hinder the fragility of your tests? *here is a search that will help*
I’ll post my own answers tomorrow. but I’d like to see if people actually “get” how important the API of an isolation framework is to this cause.