You are on page 1of 29

Self Tester

An application for regression testing of the

PVS-Studio and CppCat analyzers

Implementing the mechanism of regression
testing of the static code analyzers PVSStudio and CppCat.
Ensuring that tests are run on a large number
of open-source C/C++ projects.
Ensuring testing of the analyzers operation
under all the supported Visual Studio
Providing convenient handling of the
differences list, quick view feature, apply
changes feature, and so on.

Platform: .NET Framework 4.5;


Programming language: C# 5.0;

Technology: Windows Presentation
Foundation 4.5

Projects to launch
The interface presents a net, its first
column containing a set of projects to
Projects to be launched are stored in an
archive that will be extracted into the
installation folder by the program
This column contain names of solutions
with file extensions for short.
To find out the exact destination of a
solution file, move the cursor over the
solution name.

Choosing a default
analysis launch
When launched for the first time, the
program automatically chooses a column
with a configuration supposed to be
launched most often.
In the current program version, the
dominant configuration is the one
implying launching all the projects for the
PVS-Studio analyzer under Visual Studio

Other launches
The program keeps track of launch
sessions. For example, you can launch all
the projects for all the installed Visual
Studio versions for the PVS-Studio
analyzer in one session.
To do this, just click on the column
If you need to choose several Visual
Studio versions, hold CTRL key while
clicking on the column headers.
The standard shortcuts are also valid
when handling cells. For example, if you
want to select a number of single
projects, hold CTRL key when clicking on
the cells.

Other launches
The program also supports launches for
the CppCat analyzer

States of cells when

analysis is not running
The cells of the main project table can be
in different states.
Depending on the analysis launch status,
cell states may intersect or not.
For example, regardless of the analysis
launch status, the Not supported state
may be displayed, which means that the
analyzer is not supported for a certain
Visual Studio version in the current
project configuration.

States of cells when

analysis is not running
Also, you can see that some Visual Studio
versions may be not installed even if the
project configuration implies that all the
launches are supported.
It is just logical that the VS not installed
state will lap over the Not supported
state even if the configuration of a
particular solution was meant to be
launched in the Visual Studio 2013

States of cells when

analysis is running
After launching analysis of the projects
selected, you will see the cells switch into
the analysis running state.
The initial state for each project is
Pending, which means that the project
was selected and scheduled for analysis
in the current launch session.

States of cells when

analysis is running
It is logical to presume that the
Pending state is followed by the In
progress state.
It means that the project is being
analyzed at the moment.
Note also that several projects can be
analyzed simultaneously.
This behavior cannot be handled from the
user interface. It is specified in the
applications configuration file and set to
value 4 by default, which means that 4
projects at most can be launched

States of cells when

analysis is finished
Once analysis is completed, the projects
can take one of several analysis complete
A normal analysis complete state is
marked as OK. It means that no
differences have been found for the
project with the current reference settings
of the analyzer.

States of cells when

analysis is finished
It is also possible that analysis may reveal
differences in the project.
In this case, its state is marked as Diff.
Differences mean that a previous
reference configuration did not match the
current one during the analysis.
In such cases, you need to find out the
source and reasons of the differences.
If you were expecting these differences,
you can make this configuration a new
current reference one for future analysis
sessions; otherwise you need to fix the

States of cells when

analysis is finished
The least definite project state after the
analysis is finished is the Fail state.
It does not indicate if any differences
have been found or not.
The Fail state means that analysis has
failed because of internal exceptions that
sometimes occur in Visual Studios
installed plugins.
In this case, you can resolve the issue by
relaunching the analysis.

Note also that each solution has a
configuration that analyzer launches
depend on.
It is usually specified during the
installation procedure and not intended
to be changed further on.
However, you may see projects that can
be launched under certain Visual Studio
versions only.
For example, for the GuiBaker solution,
the only supported version is Visual
Studio 2010.
This is the explanation for the Not
supported state.

The program has the only menu from
where you can select launch sessions.
If there have been no analyzer launch
sessions yet, you will see only the Erase
All button which will be inactive.
If you have already performed analysis
sessions, this button will become active
and clicking on it will remove all the
existing launch sessions.

At the first launch, an item corresponding
to the current analysis launch will be
added into the menu. It will be
highlighted but the menu itself will
become inactive until the current analysis
session is completed or cancelled.
The menu item name consists of the date
and time for the current locale and the
name of the currently logged in user in

When the current analysis session is
completed or stopped, the menu items
become active and can be selected.

Active/inactive menu state

Warning windows
In the application, you can sometimes see
warning windows displayed.
These windows can be of two types:
those appearing when errors occur
related to the analyzers work, and those
related to the application itself.
The picture shows a window generated
by Visual Studio indicating that the
TestRun command is not valid in the
current version. It may signal that the
plugin you are trying to install is not
registered for the current version.

Warning windows
This picture shows a warning window of
the application itself.
It tells you that no solution was selected
for launch.

Graphic launch states

Graphic launch states are provided for in
this application.
They are shown in the bottom left part of
the window and make it easier to monitor
the applications activity.
For example, this picture shows that the
analysis session was passed, stopped, and
launched but revealed no differences as

Graphic launch states

This picture shows that analysis is
currently running but a few projects
different from the current reference
configuration were already detected.

Graphic launch states

This picture shows that the current
analysis session was completed or
stopped and revealed differences for two

Monitoring the progress when analysis is running

Visual Studio call

Double-clicking on a selected cell opens
the project in Visual Studio.

Toolbar buttons
To enable the user to perform various
manipulations with test launches, a
toolbar was implemented. Button
descriptions left to right:
Run button is intended to launch/stop
analysis for the current selected
Log button is used to open an HTMLlog to view differences for the current
selected session;
Diff button allows you to open
selected solutions with differences in
Visual Studio;
Apply Results button allows you to
replace reference configurations with
current ones for selected solutions;

Toolbar buttons
The toolbar can switch into the launch
While it is in this state, you can see the
time passed since the current analysis
launch displayed on it.
It is logical to suppose that the Apply
Results button will be inactive in this state
since not all of the results have been
obtained yet.

Visual Studio launch

The button to open solutions in Visual
Studio was implemented to make it more
convenient to open several solutions at
However, launching several Visual Studio
instances may take quite a lot of time, so
we have implemented a status window
that does not block the user interface
when opening several Visual Studio