How do you keep your weakly tested legacy code from dragging down your coverage numbers for your new code?
If you have the luxury of excluding all the legacy code, you can have your coverage numbers reflect only the new code. You might try running two coverage calculations - one for the legacy and one for the new code. This is doable but cumbersome.
If you want to have a single coverage calculation in your build, dealing with legacy code might mean diluting your coverage numbers. Nobody is going to pay your team to go back and retrofit unit tests to match the standards of the new code.
What to do, what to do?
On my previous team, we ran into this situation. We were using JaCoCo to do coverage calculations and some of the code in our code base was user interface logic and was hard/expensive to test, so it had low coverage. All the server-side logic and shared logic had really good code coverage. We really wanted to treat them differently but did not want to open the door for developers to get lazy and go to the "lowest common denominator".
So what I did was build a tool that allowed us to express flexible coverage limits based on individual or groups of packages and/or classes per coverage metric. The tool took a set of rules as input, the current JaCoCo coverage report, and applied the rules to the metrics, signalling an error if any of the rules were violated.
The simplest rules file contents is shown below. Since this is CSV-based, the first row shows the static column headers. For these examples, we will only talk about LINE coverage, but obviously other coverage metrics are possible.
This single rule says that all packages in-aggregate, with all classes in each package in-aggregate, should have a minimum 70% line coverage.
PACKAGE, CLASS, ABC_MISSED, ABC_COVERED, LIMIT
*, *, LINE_MISSED, LINE_COVERED, 0.7
The following rules show how you can special-case a single class for its own coverage limit of 40%. The important thing here is that the special class follows one rule and all the rest of the code follows the other rule.
PACKAGE, CLASS, ABC_MISSED, ABC_COVERED, LIMIT
com.acme, FooBar, LINE_MISSED, LINE_COVERED, 0.4
*, *, LINE_MISSED, LINE_COVERED, 0.7
As a final example, we will specify that all the classes in the special apex package should in-aggregate, have a 50% coverage.
PACKAGE, CLASS, ABC_MISSED, ABC_COVERED, LIMIT
com.acme, FooBar, LINE_MISSED, LINE_COVERED, 0.4
com.apex, *, LINE_MISSED, LINE_COVERED, 0.5
*, *, LINE_MISSED, LINE_COVERED, 0.7
I will be working on and open sourcing a coverage rules tool very soon that will follow rules like those above (and a few more).
