--- title: "Over Specification in Tests" url: "https://osherove.com/blog/2008/7/12/over-specification-in-tests.md" canonical: "https://osherove.com/blog/2008/7/12/over-specification-in-tests.html" published: "2008-07-12T03:23:07" updated: "2008-07-12T03:23:07" tags: ["Agile", "Agile Related", "Art Of Unit Testing", "Unit Testing"] author: "Roy Osherove" description: "I've gotten the chance to visit and consult various companies and I see this all the time: Over specification in tests, especially when doing mocks and stubs . One main reason for that is that people use their stub as a…" usage: "Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original page)." license: "https://creativecommons.org/licenses/by/4.0/" --- # Over Specification in Tests By Roy Osherove July 12, 2008 In [Agile](https://osherove.com/blog/category/agile), [Agile Related](https://osherove.com/blog/category/agile-related), [Art Of Unit Testing](https://osherove.com/blog/category/art-of-unit-testing), [Unit Testing](https://osherove.com/blog/category/unit-testing) **Historical article** · Originally published July 12, 2008; technical details may be outdated. I've gotten the chance to visit and consult various companies and I see this all the time: Over specification in tests, [especially when doing mocks and stubs](http://weblogs.asp.net/rosherove/archive/2007/09/16/mocks-and-stubs-the-difference-is-in-the-flow-of-information.aspx). One main reason for that is that people use their stub as a mock object (stubs are fake objects that we will not assert on so that we can test something else). here is a small example from something posted on the alt.net mailing list: \[Test\] public void Can\_Return\_All\_Plans() { IPlanDao mockPlanDao = MockRepository.GenerateMock(); mockPlanDao.Expect(x => x.AllPlans(\_fakeAdmin)).Return(new List{\_fakePlan}); PlanController planController = new PlanControllerForTesting(\_mockCommonDao, \_mockCommonService, mockPlanDao); ViewResult result = planController.All(false); mockPlanDao.VerifyAllExpectations(); result.AssertViewResultNameAndViewDataType(typeof(PlanContainer), "Plans"); } in this test the following line is used to "stub" out a return value into the system under test. mockPlanDao.Expect(x => x.AllPlans(\_fakeAdmin)).Return(new List{\_fakePlan}); no problem here. The problem lies in the last two lines of the test: mockPlanDao.VerifyAllExpectations(); result.AssertViewResultNameAndViewDataType(typeof(PlanContainer), "Plans"); the second line (the assert) is probably the thing you really want to test. but the call to "verifyAllExpectations" is the one that makes the test very fragile. It is "asserting" that someone has actually called "GetAllPlans()" to get the results, when it clearly does not matter how one got the results. all that matters is that: "Given a set of results returned into the application under test, the view in the controller is correct" (the translation of the last line in the test). If by any chance the application later does several more calls to the stubbed out dao, or uses a different method to get results, the test could break, even though the application would still ultimately work with the same end result. It's like me ordering a pizza delivery and then asking the delivery guy "did you get here by car or motorcycle?" . I realy don't care, as long as the end result: PIZZA, is correct. So the general rules for me are: - If you see both "Verify" and "Assert" in the same test, it is usualy a smell of over specified tests - if you see "expect" and "Return" on a fake object, make sure that you name it "stubXX" or "mockXX" so that you can distinguish whether you want to call verify on it later or not(most cases should be "not") - Try to test on the end result or end state rather than verify interactions.  The only time you _absolutely have not choice_ but to test an interaction using verify is when calling _void_ methods on external objects. that is clearly a one-way communication and _is (or part of)the end result_ of what you are trying to accomplish. A lot of people think that adding that extra "verify" just means it is a good thing since they are doing more assert. Well, they sure are testing more things, but they are _internal_ things to the app's behavior and that is usually leading to brittle tests. Try to test on the end result or end state rather than verify interactions. ## Lessons on this topic - [Test Review Practices](https://courses.osherove.com/courses/the-art-of-unit-testing-and-tdd-master-class-in-net/test-review-practices/) · The Art of Unit Testing & TDD Master Class in .NET - [Maintainability Practices](https://courses.osherove.com/courses/tdd-in-net-in-hebrew/maintainability-practices/) · TDD in .NET in Hebrew ## More in Agile [All posts in Agile](https://osherove.com/blog/category/agile) - Sep 20, 2008 ### [Goodbye mocks, Farewell stubs](https://osherove.com/blog/2008/9/20/goodbye-mocks-farewell-stubs.html) - Jul 24, 2008 ### [Typemock Isolator with AAA syntax](https://osherove.com/blog/2008/7/24/typemock-isolator-with-aaa-syntax.html) - Jul 10, 2008 ### [Poll: What threading features do you use?](https://osherove.com/blog/2008/7/10/poll-what-threading-features-do-you-use.html) - Jul 8, 2008 ### [A sneak Preview of Typemock Racer : Thread Deadlock Finder](https://osherove.com/blog/2008/7/8/a-sneak-preview-of-typemock-racer-thread-deadlock-finder.html) --- ## About the author **Roy Osherove**: Senior Solutions Architect at AWS working on AI for production and AI-driven development; author of The Art of Unit Testing and Elastic Leadership, with about 20 years of consulting and training on unit testing, TDD, team leadership and continuous delivery. ### Sites - [RobotPaper](https://robotpaper.ai/): This blog: AI patterns and anti-patterns, AI and coding agents, and engineering leadership. - [osherove.com](https://osherove.com): Roy's personal site. Author of The Art of Unit Testing and Elastic Leadership; consulting and training on unit testing, TDD, team leadership and continuous delivery, plus about 20 years of blog posts. - [The Art of Unit Testing](https://www.artofunittesting.com): The Art of Unit Testing, 3rd Edition (Manning): readable, maintainable, trustworthy unit tests, plus TDD workshops in C#, JavaScript, Java and Kotlin. - [5whys.com](https://www.5whys.com): Roy's writing on software team leadership: self-organizing teams, surviving chaos, learning mode and the Elastic Leadership model. - [Pipeline Driven](https://pipelinedriven.org): Continuous delivery and DevOps practices: build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Elastic Leadership](https://www.elasticleadership.com): Elastic Leadership, Roy's framework for software team leaders (survival, learning and self-organizing modes). ### Social - X: https://x.com/royosherove - LinkedIn: https://www.linkedin.com/in/osherove/ ### Contact - Contact form: https://forms.gle/jQSx5KoXvrVA1JCK7 ### License Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original page). License: https://creativecommons.org/licenses/by/4.0/