Anders Norås' blog

The inner works of code coverage instrumentation in VS Team System

A Norwegian bank account number is 11 digits long. This means that there are 10¹¹ possible values for such a number. If you were to test an account number validation method you would not feed it all possible numbers since only few of those number are likely to be valid. Instead you pick a few numbers that should pass the test and some that should not. A banking domain specialist is able to pick these numbers because she knows how an account number is built. In Norwegian banks these consist of a four digit bank id, a two digit account type and a five digit account number. From the account number a checksum can be calculated to validate whether the number is a possible account number or not. It is this knowledge you are building into your validation method.
When implementing such a validation method you're likely to use quite a few if statements or case switches and maybe a couple of for loops as well. Each of these instructions gives your method an additional execution path for which a test case should be written.
Structured basis testing is a technique for testing all possible paths through a method. The basic concept in structured basis testing is that you need to test each statement at least once. If the statement is a conditional statement you need to vary the test cases to test all possible outcomes to ensure the statement is fully tested.

In the 1976 Thomas McGabe pioneered a metric for cyclomatic complexity. Cyclomatic complexity is in essence the number of execution paths for a section of code.
The formal equation for the metric is CC=E-N+P where E is the number of edges in a graph, N the number of nodes and P the number of connected components.
Nodes are decision statements such as if and edges are paths between nodes. In object-oriented languages P has a constant value of 1 representing the method entry point.
You can calculate the number of test cases needed to cover the entire method and code coverage tools help you determine which parts of your code your test cases cover and which parts they don't.

1private accountType GetAccountType

2{

3 get

4 {

5  if (Balance > 15.00)

6   return accountType.Platinum;

7  else

8   return accountType.Gold;

9 }

10}

11

Consider the above method from the Woodgrove Bank walkthrough project.  This method has one decision point; the if statement. Adding the constant 1 for the methods entry point yields a cyclomatic complexity of 2. From this we can determine that at least two unit tests are required to thoroughly test the method.
When running the test from my previous post on private method testing, the code coverage results show that 25% of the GetAccountType property is not covered by the test case. When the "Show code coverage" switch is on, Visual Studio even shows you which lines are covered (the green ones) and which aren't (the red ones). How does Visual Studio 2005 Team System know this?
To determine which parts of a method a test covers code coverage tools instrument the test target. The Visual Studio 2005 Team System code coverage analyzer injects probes into the executable to monitor which lines the execution pointer stops by and which are never visited.

1private BankAccount.accountType get_GetAccountType()

2{

3      BankAccount.accountType type1;

4      Init_0e94efa84444b4ee4b7ec08941916a9b.Register();

5      Init_0e94efa84444b4ee4b7ec08941916a9b.m_vscov[9][14] = 1;

6      Init_0e94efa84444b4ee4b7ec08941916a9b.m_vscov[9][15] = 1;

7      bool flag1 = this.Balance <= 15;

8      if (!flag1)

9      {

10            Init_0e94efa84444b4ee4b7ec08941916a9b.m_vscov[9][0x10] = 1;

11            type1 = BankAccount.accountType.Platinum;

12      }

13      else

14      {

15            Init_0e94efa84444b4ee4b7ec08941916a9b.m_vscov[9][0x11] = 1;

16            type1 = BankAccount.accountType.Gold;

17      }

18      Init_0e94efa84444b4ee4b7ec08941916a9b.m_vscov[9][0x12] = 1;

19      return type1;

20}

The above code is the GetAccountType property from earlier with the probes inserted. Init_0e94efa84444b4ee4b7ec08941916a9b is a generated class. This class is accompanied by a structure named Header_0e94efa84444b4ee4b7ec08941916a9b. As you might have guessed, the awkward name ensures that the class and structure names are unique. The Init class has a static internal UInit32 array named m_vscov.
The first call registers the method for instrumentation. This is done by passing a reference to the m_vscov array to a method called VSCoverRegisterAssembly on an unmanaged dynamic link library named vscover.dll.
The subsequent injected lines are the actual probes. The m_vscov map that was passed to the external method is used to keep track of execution. The first dimension of the array seems to always have 9 as its index. The second dimension corresponds with an address in an instance of the header structure (m_RvaVscov).
Each time a probe is passed by the execution pointer, a flag is set to 1 on the probes unique index. The number 1 is a numeric representation of a true boolean.
When a test run completes the Visual Studio code coverage analyzer can retrieve the code coverage metrics by analyzing the flags in the m_vscov map.
The number of lines not covered is the total number of probes minus probes with the flag set to true and so on.
The code coverage probes do not end up in neither debug nor production code. Instead Visual Studio compiles a separate executable used solely for code coverage instrumentation. This file will be located in Documents and Settings\%user%\Local Settings\Application Data\VSEqtDeploymentRoot\%guid%\Out\.
Since the actual executable is modified, Visual Studio also provides an option for re-signing the assembly after a code coverage metric have been collected. This option is found on the "Code Coverage" tab in the "Select a Test Run Configuration"dialog.

To make sure the entire method is covered by the test we have to extend the test case to include a test where the balance is over the minimum for the platinum account; $15.00.

1[TestMethod()]

2public void GetAccountTypeTest()

3{

4 string customerName = null;

5 double balance = 0;

6 BankAccountNS.BankAccount target = new BankAccountNS.BankAccount(customerName, balance);

7 BankAccountNS.BankAccount.accountType val = BankAccountNS.BankAccount.accountType.Gold;

8 BankAccount.BankAccountAccessor accessor = new BankAccount.BankAccountAccessor(target);

9 Assert.AreEqual(val, accessor.GetAccountType);

10 balance = 16.00;

11 target = new BankAccountNS.BankAccount(customerName, balance);

12 val = BankAccountNS.BankAccount.accountType.Platinum;

13 accessor = new BankAccount.BankAccountAccessor(target);

14 Assert.AreEqual(val, accessor.GetAccountType);

15}

Volia, all probes get hit and we have 100% code coverage.