January 16, 2005

Code coverage analysis

Code coverage analysis is a way of verifying that every code path in a program is exercised at least once by a collection of test cases. Consider this simplified example:

public class MathUtils

{
public static int Sign(int x)
{
if (x<0)
{
return -1;
}
else if (x>0)
{
return 1;
}
else
{
return 0;
}
}
}

[TestFixture]
class TestMathUtils
{
[Test]
public void TestNegative()
{
Assertion.AssertEquals(MathUtils.Sign(-5), -1);
}

[Test]
public void TestPositive()
{
Assertion.AssertEquals(MathUtils.Sign(5), 1);
}
}
There are three code paths through the function Sign(). Of those, two are exercised by unit tests and one is untested. Test Driven Development (TDD) demands that the programmer writes unit tests first and then only as much code as is necessary to make those tests pass. In fact it also demands that she first deliberately write code that makes the test fail! An introduction to TDD can be found in a series of articles by Robert C. Martin, starting here.

If the intent is that all code paths should be tested, for example when using TDD, it is useful to have an automated tool that can detect untested code paths. In a sense, this tests the tests and provides confidence that the TDD principles are being followed. It is even possible to integrate code coverage analysis into the normal build process and run it to ensure the tests are complete (or more accurately, not incomplete) prior to running the tests themselves.

I would like to build support for code coverage analysis into my C# to C++ translator. I will do it by generating additional C++ code that will track which blocks of code have been executed and which haven't. There is no need to track each individual C# statement or MSIL instruction. It is sufficient to track entry or exit from each basic block. A basic block is a sequence of instructions ending with a control flow transfer instruction and containing no control flow targets. For example, the body of an inner loop containing no other control flow constructs would be a basic block.

A simple approach would be to associate a bit-field with each C# method. Each basic block in the method's MSIL would be represented by a bit in the bit-field. The translator would prepend each basic block with a C++ statement to set the corresponding bit in the bit-field.

Before running the unit tests, the bit-fields would be cleared. Then afterwards, in order to verify that the unit tests fully exercised the tested code, it is a simple matter of checking that all the bits are set. If any are still clear, it should be possible to display an error message indicating the source file, method and line number that is untested.


Comments:
IFAIR you use MSIL->model->C++ path of compiling. Yesterday I have compiled C# compiler from mono. It takes less then 2 hours to understood how to do it.

Mono C# compiler emits MSIL instructions through Reflection.Emit API. Compiler is written on pure C# and have about 50 000 lines. To compile it under windows you need only Visual Studio .NET and nothing more.

In order to build Mono compiler from sources you should
1) download sources from
http://www.mono-project.com/downloads/

2) Compile cs-parser.jay to cs-parser.cs
jay.exe -ctv cs-parser.jay <skeleton.cs >cs-parser.cs

jay.exe comes as precompiled program within packeges from download page from step 1, but if you wish you could build it with visual studio from sources (I did it).

3) When cs-parser.cs file exists you can open compiler.sln which contains only compiler.csproj.

Make compiler and try to compile. My attempt to compile and run 'hello, world' under windows was successful :)

serra, level3@mail.ru
 
Serra, thanks for your comment. I have also successfully compiled the Mono C# compiler from source using Visual Studio. But I think you misunderstand my goal. I am trying to translate C# into C++ in order to run it on platforms with no .NET virtual machine, specifically video game consoles like PS2 and Xbox.
 
i want jay.exe can anyone tell me which file i have download from this url http://www.mono-project.com/downloads/
thnaks
 
Hi ,
i need jay.exe to get cs-parser.cs
and i want compiler construction Tool using c# which generates c# based code for lexer and parser.
thanks
Renugopal
 
You need to build jay.exe from the source code in mono-1.1.x.x/mcs/jay using a C compiler. I did this once with Visual C++ but I no longer have jay.exe.
 
Hi,

My code is built using C language and I am using a Rational Purecoverage tool. Can anyone help me to give details on the code coverage? If there are any existing documents on Code coverage.Please help me to find the path.

Thanks and Regards
Shruthi
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?