Is 100% Code Coverage the Best Use of Your Time?
How can that time be re-allocated?
In software development, achieving 100% code coverage is often seen as crucial. However, it may not always be the best use of an engineering team's limited resources. While code coverage is valuable, it's important to recognize its limitations and explore other approaches that could lead to better outcomes.
Code coverage measures the percentage of code that is executed during testing. However, it's important to note that simply running each line of code once doesn't ensure that all potential issues will be detected. We also need to test different combinations of values. It's possible that someone might have 15 tests for one method and zero tests for the next 5 methods, which might seem surprising, but not all code paths are equally important.
Martin Fowler, a renowned software developer, aptly puts it:
"Test coverage is a useful tool for finding untested parts of a codebase. Test coverage is of little use as a numeric statement of how good your tests are."
I strongly believe in focusing on the quality and effectiveness of tests, rather than just the quantity. Instead of aiming for 100% coverage, I am confident that investing time in designing comprehensive test cases that cover critical paths and edge cases is more beneficial.
I believe that striving for complete code coverage can often result in diminishing returns. The effort required to increase coverage from 95% to 100% can be disproportionately high, leading to a significant rise in the number of test cases. This expansion can make the test suite more fragile and challenging to maintain over time.
As noted by Steve McConnell in his book "Code Complete":
"Trying to achieve 100% coverage often results in a significant amount of additional testing effort for minimal gain."
Moreover, I must always consider the long-term maintenance cost of a fully covered codebase. As our software evolves, keeping all tests up-to-date can become a time-consuming task that may slow down our development and hinder our ability to ship new features quickly. This brings me to a crucial question: What do our customers truly care about? In most cases, they prioritize a stable, feature-rich product delivered in a timely manner over a theoretical measure of code quality.
In my opinion, while code coverage is certainly helpful, it must not overshadow other vital aspects of software development. I advocate for a well-rounded approach that incorporates strategic test coverage alongside other quality assurance methods.
As suggested by Glenford Myers in "The Art of Software Testing":
"The objective of testing is to find errors. Therefore, a good test case is one that has a high probability of detecting an as-yet undiscovered error."
I believe that by focusing on creating effective tests that address potential problem areas and essential functionalities, my team and I can optimize our time and resources, ultimately providing more value to our customers.