Building ConQAT Blocks

From Conqat

As of ConQAT 2.0 the preferred way for creating and editing blocks is to use cq.edit. This page is only required, if you want to understand the underlying file format in more detail.


This page gives a description of the XML format used for defining ConQAT blocks (CQB files). Basically a block is a description of which processors and other blocks to use and how to combine them.

Contents

ConQAT root element

The root element of each ConQAT configuration is the conqat element. Additionally you require the usual XML "decorations" such as the namespace used. A bare configuration file looks like this:

 <?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
 <conqat xmlns="http://conqat.cs.tum.edu/ns/config">
     <!-- Include configuration here ... -->
 </conqat>

Block Specification Element

Blocks are specified using the block-spec element, which takes the name of the block as attribute (this is the name which is used from block elements in the spec attribute). You can put exactly one of those block specifications in a block configuration file (for details see below).

Processors

Processors are the main elements of ConQAT (see Architecture). To include them in the configuration, the processor element is used, which requires two attributes: A unique name and the Java class implementing the processor. Processors usually have additional parameters. Which parameters a processor supports is documented in the ConQATDoc reference. For example the processor for reading a bunch of Java files looks like this:

 <processor name="my-scope" class="edu.tum.cs.conqat.java.scope.JavaScope">
     <source dir="c:\myproject\src" />
     <byte-code dir="c:\myproject\build" />
 </processor>

Here the name of the processor is my-scope and the JavaScope processor is used. The processor-specific parameters indicate where the source and byte code of "myproject" can be found. Note that the name of a processor may only consist of alphanumeric characters, the dash and underscore.

As seen the parameters of a processor can be filled with constant expressions given as a string. Alternatively the results (output) of another processor can be used as input. In this case the @ character followed by the name of the processor is used. The next example shows how to count the lines of code in the files read in the processor above using the LOCAnalyzer.


 <processor name="loc-counted" class="edu.tum.cs.conqat.filesystem.analysis.LOCAnalyzer">
     <input ref="@my-scope" />
 </processor>

As using the output of one processor as input for another one also determines their execution order, no cycles may be introduced using these references. So the following example is invalid and will cause an error:

 <processor name="a" class="...">
     <input ref="@c" />
 </processor>
 
 <processor name="b" class="...">
     <input ref="@a" />
 </processor>
 
 <processor name="c" class="...">
     <input ref="@b" />
 </processor>

Blocks

Using only processor the configuration files for a larger analysis can become quite complex and contain redundancy. To make the configuration more manageable so called blocks are used, which are just pieces of configuration files (i.e. more processors and blocks). You either can write your own blocks or use some of the blocks included in the bundles coming with ConQAT, e.g the ConQAT blocklib.

From a user's perspective blocks are quite similar to processors. They are declared using the block element, which also requires the name attribute and additionally the spec giving the name of the block specification used. As with processors, blocks may accept parameters. In the following example the StdAnalysis is used, which is a block for performing analysis of Java projects.


 <block name="my-analysis" spec="edu.tum.cs.conqat.blocklib.StdAnalysis">
     <input scope="@my-scope" junit="@my-junit" name="My Project" />
     <db connector="@my-db" />
     <output dir="c:\temp\output" />
 </block>

So we use the output of the my-scope processor from the earlier example and the results of some other processors not shown here as input to the block. The result of the block can be accessed just as the result of a processor using the @ character. However blocks may have multiple outputs and thus the name of the output used must be given (separated by a dot). So to output the results of the block using the HTMLPresentation, we could write

 <processor name="output" class="edu.tum.cs.conqat.html_presentation.HTMLPresentation">
     <output dir="c:\temp\output" />
     <project title="Analysis of MY project" />
 
     <result ref="@my-analysis.html-unused" />
     <result ref="@my-analysis.html-immutable" />
     <result ref="@my-analysis.html-javadoc" />
     <result ref="@my-analysis.html-rating" />
     <!-- ... -->
 </processor>

However as you see, this is a lot of work if many outputs are referenced. Thus ConQAT also allows to use * as the last character, indicating to use all outputs starting with the string before. So the previous example can be written as

 <processor name="output" class="edu.tum.cs.conqat.html_presentation.HTMLPresentation">
     <output dir="c:\temp\output" />
     <project title="Analysis of MY project" />
 
     <result ref="@my-analysis.html-*" />
 </processor>

The order of the results used in this case is determined by the block's specification. Also note that we included only outputs starting with "html-", as the StdAnalysis also has outputs not suitable for the HTMLPresentation. To get all outputs we would have used "@my-analysis.*".

Parameters and Outputs

Inside of the block-spec element you can use both the processor and block elements as in a normal configuration file. However the names of processors and blocks are local to the current specification, i.e. you can not reference these processors from outside or reference the results of processors not in this specification. To allow exchanging data with the surrounding of the block, parameters and outputs have to be specified. To specify a parameter, the param element is used, which must include attr elements defining the attributes of the parameter. Both require the name attribute. For outputs the out element is used, which also requires the name as well as a ref attribute referencing the processor or block output which should be made available on this output. Referencing processors and block output is performed as usual using the @ followed by the name of the processor or block and for block the output's name. To reference an attribute of the input we again use the @, but this time followed by the name of the parameter, a dot, and the name of the attribute. All this is put together in the next example.

 <block-spec name="my.own.Block">
     <param name="input">
         <attr name="scope" />
         <attr name="name" />
     </param>
     <param name="keys">
         <attr name="read" />
         <attr name="write" />
     </param>
         
     <processor name="aggregated" class="edu.tum.cs.conqat.commons.aggregation.SumAggregator">
         <read key="@keys.read" />
         <write key="@keys.write" />
         <input ref="@input.scope" />
     </processor>
   
     <processor name="layouted" class="edu.tum.cs.conqat.html_presentation.layouters.TableLayouter">
         <input ref="@aggregated" />
         <info name="@input.name" description="Fixed description" />
     </processor>
  
     <out name="html" ref="@layouted" />
     <out name="raw" ref="@aggregated" />
     <out name="scope" ref="@input.scope" />
 </block-spec>

In this (artificial) example the block accepts two different parameters: input with attributes scope and name, and keys with attributes read and write. The input scope will be put through the SumAggregator which in turn is layouted using the TableLayouter. Both the layouted and the "raw" result of the aggregation are made available as outputs, as well as the provided scope. We could now use this block as follows:

 <block name="my-block" spec="my.own.Block">
     <input scope="@my-scope-processor" name="MyProject" />
     <keys read="A" write="B" />
 </block>
 
 <processor name="user" class="...">
     <input ref="@my-block.*" />
 </processor>

This together is equivalent to the following configuration:

 <processor name="my-block.aggregated" class="edu.tum.cs.conqat.commons.aggregation.SumAggregator">
     <read key="A" />
     <write key="B" />
     <input ref="@my-scope-processor" />
 </processor>
   
 <processor name="my-block.layouted" class="edu.tum.cs.conqat.html_presentation.layouters.TableLayouter">
     <input ref="@my-block.aggregated" />
     <info name="MyProject" description="Fixed description" />
 </processor>
  
 <processor name="user" class="...">
     <input ref="@my-block.layouted" />
     <input ref="@my-block.aggregated" />
     <input ref="@my-scope-processor" />
 </processor>

To better understand block specifications it also is a good idea to look at existing blocks coming with the bundles.

Of course the parameters given to a block can be also repeated. For example

 <block-spec name="Spec1">
     <param name="p">
         <attr name="a" />
         <attr name="b" />
     </param>
 
     <processor name="Proc" class="MyClass">
         <config value="@p.a" />
         <config value="@p.b" />
     </processor>
 </block-spec>
 <block name="my-block" spec="Spec1">
     <p a="A1" b="B1" />
     <p a="A2" b="B2" />
 </block>

is equivalent to

 <processor name="my-block.Proc" class="MyClass">
     <config value="A1" />
     <config value="A2" />
     <config value="B1" />
     <config value="B2" />
 </processor>

Inference mechanism

As seen before, the parameters, attributes and outputs of a block specification are not annotated with types or multiplicities. This is because these can be calculated from the way they are used in the blocks and processors inside of the block specification. Usually this inference mechanism will just do what is expected, and the easiest way to understand what is happening is to run ConQATDoc on the block to see what the inference algorithm decided. However for some cases a deeper understanding might be useful, which is provided by this section.

Type inference

The type used for an attribute is the lowest type in the type hierarchy which fulfills all needs of the processors using it. It is best seen by an example:

 <block-spec name="TypeInferenceExample">
   <param name="p"><attr name="a" /></param>
 
   <processor name="P1" class="...">
       <input ref="@p.a" />
   </processor>
 
   <processor name="P2" class="...">
       <input ref="@p.a" />
   </processor>
 </block-spec>

If both P1 and P2 use a Double as input, the type of p.a will be Double. If P1 takes a Double and P2 a Number, the type will again be Double, as Number would not be enough for P1. If the type for P1 is List and that used by P2 is File, the type required for p.a would be a "subclass of File which implements List", which is unusual but possible. And if P1 wants a String and P2 a File, this would cause an error because there can be no class being both a subclass of File and String.

What was shown here for two processors can be generalized to multiple processors and blocks, taking pipelining into account. For outputs the type is inferred in a similar way, but here only one processor has to be taken into account.

Multiplicity inference

While for the attributes we infer the types, for the entire parameter we have to decide on the allowed multiplicity. Again we use the most general multiplicity possible.

 <block-spec name="MultiplicityInferenceExample">
   <param name="p1"><attr name="a" /></param>
   <param name="p2"><attr name="a" /><attr name="b" /></param>
 
   <processor name="P1" class="...">
       <input ref="@p1.a" />
   </processor>
 
   <processor name="P2" class="...">
       <input ref="@p1.a" />
       <config ref="@p2.a" />
       <config ref="@p2.b" />
       <config ref="@other-processor" />
   </processor>
 </block-spec>

We will denote multiplicities by [a,b] indicating at least a and at most b occurrences. The multiplicity for parameter p1 is just the intersection of the multiplicities for parameter input in both P1 and P2. So for [1,1] and [1,1] this will result in [1,1] for p1, too. Also [1,inf] and [0,1] will result in [1,1]. For [3,7] and [4,42] we would get [4,7] and for [2,4] and [7,9] we get an error, as there is no multiplicity for p1 making it usable for both P1 and P2.

Things get more involved for parameter p2, as it is used it twice in P2. So if P2's config parameter has multiplicity [0,8] this would result in a multiplicity for p2 of [0,3] because for each of these value the multiplicity of P2.config is fulfilled. If we had 4 occurrences of p2, there would be 9 config parameters for P2.

Documenting blocks

It is good practice to document the block specifications, to make their purpose clear without having to read the entire block specification. Basically there are two methods for doing this. One is to use XML comments (&lt!-- --&gt), but similar to plain Java comments they will not be visible to the user. The other possibility is to use the doc element, which is similar to JavaDoc comments in that everything written there will be shown in ConQATDoc.

The doc element just contains the documentation text and can be appear as the first element in all of the block-spec, param, attr, and out elements. It will then be shown in the corresonding section of ConQATDoc. For reference see the following example:

 <block-spec name="my.documented.Block">
     <doc>This block serves for demonstration purposes only.</doc>
   
     <param name="p1">
         <doc>Using p1 you can influence the behavior of the block.</doc>
         <attr name="a"><doc>A reference to an A-processor.</doc></attr>
         <attr name="b"><doc>The most critical parameter</doc></attr>
     </param>
  
     <!-- Here should be the implementation of the block using processors ... -->
  
     <out name="result"><doc>This is the main result</doc></out>
     <out name="html"><doc>This is a HTML table showing some statistics.</doc></out>
 </block-spec>

Where to place block specifications

Block specifications have each to be placed in a separate file with extension cqb (ConQAT Block) and come in two flavors. Blocks which are executed directly may be placed in the top level directory of a bundle and usually have a simple name without any dots.

Blocks which are to be accessed from other blocks and bundles are placed in the blocks directory of the bundle. The name of the block has to start with the ID of the bundle it is located in and the remaining name has to match the location of the file it is located in (where dots are used to separate directories). For example the block specification with the name org.foo.my.tests.BigBlock in the bundle org.foo.my as referenced from the following snippet

 <block name="myBlock" spec="org.foo.my.tests.BigBlock" />

should be located in the file blocks/tests/BigBlock.cqb in the main directory of the org.foo.my bundle.

Properties (no longer supported)

As of ConQAT 2.0 properties are no longer supported. Instead the parameters of a block can be set when executing a block.