You are on page 1of 29

Self Tester

An application for regression testing of the


PVS-Studio and CppCat analyzers
www.viva64.com

Objects
Implementing the mechanism of regression
testing of the static code analyzers PVSStudio and CppCat.
Ensuring that tests are run on a large number
of open-source C/C++ projects.
Ensuring testing of the analyzers operation
under all the supported Visual Studio
versions.
Providing convenient handling of the
differences list, quick view feature, apply
changes feature, and so on.

www.viva64.com

Platform: .NET Framework 4.5;

Technologies

Programming language: C# 5.0;


Technology: Windows Presentation
Foundation 4.5

www.viva64.com

Projects to launch
The interface presents a net, its first
column containing a set of projects to
launch.
Projects to be launched are stored in an
archive that will be extracted into the
installation folder by the program
installer.
This column contain names of solutions
with file extensions for short.
To find out the exact destination of a
solution file, move the cursor over the
solution name.

www.viva64.com

Choosing a default
analysis launch
When launched for the first time, the
program automatically chooses a column
with a configuration supposed to be
launched most often.
In the current program version, the
dominant configuration is the one
implying launching all the projects for the
PVS-Studio analyzer under Visual Studio
2010.

www.viva64.com

Other launches
The program keeps track of launch
sessions. For example, you can launch all
the projects for all the installed Visual
Studio versions for the PVS-Studio
analyzer in one session.
To do this, just click on the column
header.
If you need to choose several Visual
Studio versions, hold CTRL key while
clicking on the column headers.
The standard shortcuts are also valid
when handling cells. For example, if you
want to select a number of single
projects, hold CTRL key when clicking on
the cells.

www.viva64.com

Other launches
The program also supports launches for
the CppCat analyzer

www.viva64.com

States of cells when


analysis is not running
The cells of the main project table can be
in different states.
Depending on the analysis launch status,
cell states may intersect or not.
For example, regardless of the analysis
launch status, the Not supported state
may be displayed, which means that the
analyzer is not supported for a certain
Visual Studio version in the current
project configuration.

www.viva64.com

States of cells when


analysis is not running
Also, you can see that some Visual Studio
versions may be not installed even if the
project configuration implies that all the
launches are supported.
It is just logical that the VS not installed
state will lap over the Not supported
state even if the configuration of a
particular solution was meant to be
launched in the Visual Studio 2013
version.

www.viva64.com

States of cells when


analysis is running
After launching analysis of the projects
selected, you will see the cells switch into
the analysis running state.
The initial state for each project is
Pending, which means that the project
was selected and scheduled for analysis
in the current launch session.

www.viva64.com

States of cells when


analysis is running
It is logical to presume that the
Pending state is followed by the In
progress state.
It means that the project is being
analyzed at the moment.
Note also that several projects can be
analyzed simultaneously.
This behavior cannot be handled from the
user interface. It is specified in the
applications configuration file and set to
value 4 by default, which means that 4
projects at most can be launched
simultaneously.

www.viva64.com

States of cells when


analysis is finished
Once analysis is completed, the projects
can take one of several analysis complete
states.
A normal analysis complete state is
marked as OK. It means that no
differences have been found for the
project with the current reference settings
of the analyzer.

www.viva64.com

States of cells when


analysis is finished
It is also possible that analysis may reveal
differences in the project.
In this case, its state is marked as Diff.
Differences mean that a previous
reference configuration did not match the
current one during the analysis.
In such cases, you need to find out the
source and reasons of the differences.
If you were expecting these differences,
you can make this configuration a new
current reference one for future analysis
sessions; otherwise you need to fix the
error.

www.viva64.com

States of cells when


analysis is finished
The least definite project state after the
analysis is finished is the Fail state.
It does not indicate if any differences
have been found or not.
The Fail state means that analysis has
failed because of internal exceptions that
sometimes occur in Visual Studios
installed plugins.
In this case, you can resolve the issue by
relaunching the analysis.

www.viva64.com

Solution
configuration
Note also that each solution has a
configuration that analyzer launches
depend on.
It is usually specified during the
installation procedure and not intended
to be changed further on.
However, you may see projects that can
be launched under certain Visual Studio
versions only.
For example, for the GuiBaker solution,
the only supported version is Visual
Studio 2010.
This is the explanation for the Not
supported state.

www.viva64.com

Menu
The program has the only menu from
where you can select launch sessions.
If there have been no analyzer launch
sessions yet, you will see only the Erase
All button which will be inactive.
If you have already performed analysis
sessions, this button will become active
and clicking on it will remove all the
existing launch sessions.

www.viva64.com

Menu
At the first launch, an item corresponding
to the current analysis launch will be
added into the menu. It will be
highlighted but the menu itself will
become inactive until the current analysis
session is completed or cancelled.
The menu item name consists of the date
and time for the current locale and the
name of the currently logged in user in
brackets.

www.viva64.com

Menu
When the current analysis session is
completed or stopped, the menu items
become active and can be selected.

www.viva64.com

Active/inactive menu state

www.viva64.com

Warning windows
In the application, you can sometimes see
warning windows displayed.
These windows can be of two types:
those appearing when errors occur
related to the analyzers work, and those
related to the application itself.
The picture shows a window generated
by Visual Studio indicating that the
TestRun command is not valid in the
current version. It may signal that the
plugin you are trying to install is not
registered for the current version.

www.viva64.com

Warning windows
This picture shows a warning window of
the application itself.
It tells you that no solution was selected
for launch.

www.viva64.com

Graphic launch states


Graphic launch states are provided for in
this application.
They are shown in the bottom left part of
the window and make it easier to monitor
the applications activity.
For example, this picture shows that the
analysis session was passed, stopped, and
launched but revealed no differences as
yet.

www.viva64.com

Graphic launch states


This picture shows that analysis is
currently running but a few projects
different from the current reference
configuration were already detected.

www.viva64.com

Graphic launch states


This picture shows that the current
analysis session was completed or
stopped and revealed differences for two
projects.

www.viva64.com

Monitoring the progress when analysis is running

www.viva64.com

Visual Studio call


Double-clicking on a selected cell opens
the project in Visual Studio.

www.viva64.com

Toolbar buttons
To enable the user to perform various
manipulations with test launches, a
toolbar was implemented. Button
descriptions left to right:
Run button is intended to launch/stop
analysis for the current selected
solutions;
Log button is used to open an HTMLlog to view differences for the current
selected session;
Diff button allows you to open
selected solutions with differences in
Visual Studio;
Apply Results button allows you to
replace reference configurations with
current ones for selected solutions;

www.viva64.com

Toolbar buttons
The toolbar can switch into the launch
state.
While it is in this state, you can see the
time passed since the current analysis
launch displayed on it.
It is logical to suppose that the Apply
Results button will be inactive in this state
since not all of the results have been
obtained yet.

www.viva64.com

Visual Studio launch


The button to open solutions in Visual
Studio was implemented to make it more
convenient to open several solutions at
once.
However, launching several Visual Studio
instances may take quite a lot of time, so
we have implemented a status window
that does not block the user interface
when opening several Visual Studio
instances.

www.viva64.com