Al's Game Programming Blog

Code coverage analysis is a way of verifying that every code path in a program is exercised at least once by a collection of test cases. Consider this simplified example:

public class MathUtils

{

  public static int Sign(int x)

  {

    if (x<0)

    {

      return -1;

    }

    else if (x>0)

    {

      return 1;

    }

    else

    {

      return 0;

    }

  }

}



[TestFixture]

class TestMathUtils

{

  [Test]

  public void TestNegative()

  {

    Assertion.AssertEquals(MathUtils.Sign(-5), -1);

  }



  [Test]

  public void TestPositive()

  {

    Assertion.AssertEquals(MathUtils.Sign(5), 1);

  }

}

There are three code paths through the function Sign(). Of those, two are exercised by unit tests and one is untested. Test Driven Development (TDD) demands that the programmer writes unit tests first and then only as much code as is necessary to make those tests pass. In fact it also demands that she first deliberately write code that makes the test fail! An introduction to TDD can be found in a series of articles by Robert C. Martin, starting here.

If the intent is that all code paths should be tested, for example when using TDD, it is useful to have an automated tool that can detect untested code paths. In a sense, this tests the tests and provides confidence that the TDD principles are being followed. It is even possible to integrate code coverage analysis into the normal build process and run it to ensure the tests are complete (or more accurately, not incomplete) prior to running the tests themselves.

I would like to build support for code coverage analysis into my C# to C++ translator. I will do it by generating additional C++ code that will track which blocks of code have been executed and which haven't. There is no need to track each individual C# statement or MSIL instruction. It is sufficient to track entry or exit from each basic block. A basic block is a sequence of instructions ending with a control flow transfer instruction and containing no control flow targets. For example, the body of an inner loop containing no other control flow constructs would be a basic block.

A simple approach would be to associate a bit-field with each C# method. Each basic block in the method's MSIL would be represented by a bit in the bit-field. The translator would prepend each basic block with a C++ statement to set the corresponding bit in the bit-field.

Before running the unit tests, the bit-fields would be cleared. Then afterwards, in order to verify that the unit tests fully exercised the tested code, it is a simple matter of checking that all the bits are set. If any are still clear, it should be possible to display an error message indicating the source file, method and line number that is untested.