This guideline is much more aggressive and recommended if you work in a test driven manner rather than write the tests after the code has been written. The main goal here is better code coverage or test coverage.
Here s a canonical example:
Given a method:
void Sum(int x, int y, int z)
Which returns the sum of the three numbers, you are given the requirement that if any number is bigger than 1000, it is not calculated.
Assuming you are going to perform this in a test driven manner you d happily go an write a test such as this:
void Sum_NumberBiggerThan1000IsNotSummed() { Int sumResult = Sum(1001,1002,1003); Assert.AreEqual(0,sumResult); }
One has to consider: what is it exactly that I am testing here? The test sends as input all three parameters as "bad". That means that it will pass even if a check on just one of these will be performed on the production code s Sum method. In effect, code coverage of this test is possibly only a third of what it appears to be testing.
Writing such a test and making it pass might lead us to the conclusion that perhaps no more tests are necessary. But is that really the case?
There are a number of options that arise as to how you would make this test pass:
Option one:
int Sum(int x, int y, int z) { If(x>1000) x=0; If(y>1000) y=0; If(z>1000) z=0; Return x+y+z; }
This is not the simplest thing that could work, as is prescribed by TDD advocates. The reason being that you can essentially remove any two of the IFs, be left with just one of them, and the test will still pass. However, when you are faced with such a test that sends three bad values, you might be inclined to solve it this way even though it is wrong, because the test appears to be testing more than it really is.
This function makes the test pass just as well:
Option two:
int Sum(int x, int y, int z) { If(z>1000) z=0; Return x+y+z; }
Yet we know that this code is problematic. If this code is not good enough, and all the tests pass, we have a code coverage problem on our hands. We need more tests.
A better test might look like this:
void Sum_1stParamBiggerThan1000IsNotSummed() { Int sumResult = Sum(1001,1,2); Assert.AreEqual(3,sumResult); }
While we still could potentially make this test work not as simply as possible, at least the test is not presenting a false sense of security. It essentially tells us that it only tests one simple thing and that we need to write more tests if we want to be sure al of the required functionality works.
Multiple asserts in one test
How about this test:
Void Sum_AnyParamBiggerThan1000IsNotSummed() { Assert.AreEqual(3, Sum(1001,1,2);
Assert.AreEqual(3, Sum(1,1001,2);
Assert.AreEqual(3, Sum(1,2,1001);
}
Is this bad? Depends.
If I was writing this test after the production code was already written, I might not have a problem with this. But if this test is written before the tested code is written, it might prove less optimal of several reasons:
- You ll spend a longer a mount of time writing code to make the test pass. The longer you spend without running tests, the less confident you become.
- Sometimes you get a better idea of what's wrong with the code if you have multiple failures. When you have multiple asserts in one test, only the first one fails and the rest never get checked - you're losing some more points of view that you might consider important in orer to quickly discover where the problem stems from.
- You re not focusing on a small problem anymore. You re focusing on larger problems, more generic problems. It s easy to get stuck on large problems. It s easier to keep going while continually solving small problems one after the other.
- The more time you spend not running the tests, the more code you write that has not been tested, and which will probably depend on more code that has not been tested. You re more likely to present unforeseen bugs.
- The previous paragraph should not come as a surprise because the more code you write between each unit tests, the smaller the likelihood that your test actually covers that code. That s because:
- Trying to make a test with three ASSERTs pass, is just like trying to make three separate test cases pass in a single bounce: you ll have to write more code that handles more cases and you ll probably end up writing it more generic that you would have if you were just trying to fix one test case at a time. You won t really know if your tests cover all the generic code you ve just written as you would have known by making the tests pass one by one, in the simplest way possible.
- Also, one you have multiple ASSERTs in a test case, it s very easy and human to keep adding more asserts into the existing test than it would be to add them using new tests. Even if those were totally different asserts in nature, the broken window theory works very well here, and leads to less maintainable and less readable tests.
- You re working more time without the success feedback. Like it or not, the psychological effect here is important. It leads to a better coding experience and more confidence and happiness at work.
So, while it may make it seem like you re working faster, you re essentially increasing the possibility for hidden bugs in your code.