<?xml version="1.0" encoding="ISO-8859-1"?>
<document>
<head>
    <name>Code Coverage Design (V2)</name>
    <doc-version>$Date: 2003/08/06 20:02:10 $</doc-version>
    <author>Matt Albrecht</author>
</head>
<body>

<section>Overview</section>
<P>
The goal of the codecoverage package is to capture the lines of code that were
executed in a Java VM.  Traditional Java-based code coverage tools fall into
four categories:
<TABLE BORDER="1">
    <TR>
        <TH>Type</TH>
        <TH>Speed</TH>
        <TH>Needs Source</TH>
        <TH>Special In-Process Setup</TH>
        <TH>Examples</TH>
    </TR>
    <TR>
        <TD>Monitor VM</TD>
        <TD bgcolor="pink">Slow</TD>
        <TD bgcolor="lime">No</TD>
        <TD bgcolor="pink">VM must have monitoring enabled</TD>
        <TD>GroboCodeCoverage V1</TD>
    </TR>
    <TR>
        <TD>ClassLoader Dynamic Class Analysis</TD>
        <TD bgcolor="aqua">Moderate</TD>
        <TD bgcolor="lime">No</TD>
        <TD bgcolor="pink">Running VM must use specific classloader, and
        code can't load covered classes in a new classloader.</TD>
        <TD>Quilt, Hansel</TD>
    </TR>
    <TR>
        <TD>Source Recompilation</TD>
        <TD bgcolor="lime">Fast</TD>
        <TD bgcolor="pink">Yes</TD>
        <TD bgcolor="lime">No (just extra classes in classpath)</TD>
        <TD>Clover</TD>
    </TR>
    <TR>
        <TD>Bytecode Recompilation</TD>
        <TD bgcolor="lime">Fast</TD>
        <TD bgcolor="lime">No</TD>
        <TD bgcolor="lime">No (just extra classes in classpath)</TD>
        <TD>Gretel</TD>
    </TR>
</TABLE>
</P>
<P>
The V2 design extends the standard ClassLoader analysis tools and moves the
bytecode analysis and recompilation to a step outside the actual runtime
analysis.  The disadvantages include:
    <UL>
        <LI>there's an extra step in the coverage analysis;</LI>
        <LI>there are two copies of the class files (the differences from the
            outside being the size of the classes), which can lead to
            confusion;</LI>
        <LI>the pre-runtime recompilation generates extra files which must
            be used with the exact class files generated;</LI>
        <LI>additional system properties must be specified in the runtime
            engine, along with an additional class library in the
            classpath.</LI>
    </UL>
Using an automated build tool such as Ant along with a structured build process
can lead to considering the first three disadvantages as inconsequential;
additional disk-drive space becomes the only worrysome factor (which, more and
more, this is a minimal factor).  The fourth factor requires adding additional
information to the runtime setup scripts.
</P>
<P>
The perceived advantages far outweigh these manageable deficiencies:
    <UL>
        <LI>the run-time processing is sped up, since the analysis of the
        class files and bytecode restructuring is performed outside the
        execution;</LI>
        <LI>there is no more need to have a specially setup execution
        environment to ensure the proper class loader is generated, and
        no special care is needed to ensure that the executed code doesn't
        use its own class loader;</LI>
        <LI>the additional classes needed to accompany the altered classes
        is minimal, since they do the equivalent of logging.</LI>
    </UL>
</P>
<P>
<I>The reason why the author decided to create his own code coverage tool,
rather than use Gretel, is that Gretel requires a Gretel-compiled "main" method
to be executed first.  This is rarely possible in application servers or
other such environments.</I>
</P>
<section>Architecture</section>
<P>
This package breaks the general code-coverage problem into four parts:
<UL>
    <definition term="post-compilation engine">
    performs post-source code
    compilation on the bytecode (a "post-compiler", if you will), and
    dynamically allows the analysis modules to pre-process the class
    files.
    </definition>
    <definition term="analysis modules">
    parse the bytecode methods for all classes,
    mark bytecode instructions as under analysis, and generate data
    necessary for creating a report.
    </definition>
    <definition term="channel logger">
    the generated bytecode adds logging statements
    for each analysis module.  The logging is directed towards one
    "channel" per module during the runtime execution.  These loggers
    must be minimal in runtime overhead, and must be separately jarred
    for easy inclusion into the runtime.
    </definition>
    <definition term="report engine">
    parses the post-compilation data along with the
    runtime channel logs to generate reports for each analysis module.
    If the modules and post-compilation engine were constructed correctly,
    this part does not need to interact with the analysis modules
    directly.
    </definition>
    <definition term="data store">
    organizes the post-compilation meta-data and
    channel logs.  Allows for easy and efficient query of per-analysis
    module and per-class data.
    </definition>
</UL>
</P>
<P>
This package does not need to be classloader aware.  If the recompiled classes
log the classname with a checksum, then each part can uniquely identify
each class file.  Even though, technically under the JVM spec, that if two
class files are loaded by different class loaders then they are different
classes, the actual coverage numbers do not split with this differentiation.
</P>

<section>Details of Analysis Modules</section>
<P>
Analysis modules provide the actual value to the coverage tool.  They can
report number of bytecodes covered, number of lines covered, or branch
coverage.
</P>
<P>
A proper module knows how to decode the BCEL object format for methods,
marks individual bytecode instructions as "check for coverage", and generates
associated meta-data for each mark, so that the report engine can easily
create a report without having to refer back to the module.
</P>
<P>
The post-compilation engine will provide objects that simplify the matter of
inserting the marks.  It will also generate a data store for each method for
the module to record all the meta-data.
</P>
</body>
</document>

