A CONTROLLED EXPERIMENT IN

ASSESSING AND ESTIMATING

SOFTWARE MAINTENANCE TASKS
Pondicherry University
By:
SADIQUE
NAYEEM
Outline
Motivation and Background
Experiment Design
Results and Explanatory Models
Conclusions
2
MOTIVATION AND
BACKGROUND
3
Software Maintenance
Software Maintenance
Modification of a software product after
delivery to correct faults, to improve
performance or other attributes, or to adapt the
product to a modified environment [IEEE 98]
Changes are made in response to change
requirement but the fundamental software
structure is stable.


4
Maintenance is crucial in software
engineering
Systems are tightly coupled with their environment
Environment changes require changing its software systems
Technologies and requirements are continuously changing
Software systems are outdated quickly
Software systems must be updated and upgraded to maintain their
values
Maintenance is important in market competition
New software has more advantages than the existing one
Software system must be upgraded to keep its market share
5
Maintenance cost is usually 2x to 100x as much as new
development cost (Sommerville, 2006)
K$
100 200 300
System 1
System 2
(Summerville, 2006)
New Development
Maintenance
400
Development and Maintenance
cost
6
For software maintenance, the modeling process
is even
more challenging
The maintenance effort is affected by a large number of
factors such as:
Size and types of maintenance work
Team stability
Personnel capabilities
The level of programmers familiarity with the system
being maintained
Processes and standards in use
Complexity
Technologies
The quality of existing source code and its supporting
documentation
7
Software estimation community has paid
little attention to software maintenance
Most estimation models regard maintenance estimation
as secondary
Models were built using mainly data of new development projects
Use of SLOC metrics for models is inconsistent
Some models use SLOC added, modified, deleted
Others use only SLOC added and modified

8
Types of Software Maintenance
Types of Maintenance
Different author proposed different classification of Software
Maintenance
Swanson 76 : Adaptive, Corrective, Perfective
IEEE 98: all Swansons plus Preventive
Chapin et al, 2001: 12 types, 4 clusters: support interface,
documentation, software properties, and business rules. The last
cluster, includes all activities that alter the business rules of the
software :
Enhancive : Did the activities replace, add to, or extend the customer-
experienced functionality?
Corrective : Did the activities fix the customer-experienced
functionality or make it more correct?
Reductive : Did the activities restrict or reduce the customer-
experienced functionality?

9
Impact of the types of software
evolution and maintenance.
Ned Chapin, et al., Types of software evolution and software maintenance, Jan. 2001
10
Figure 1. Impact of the types of software evolution and maintenance.
The purpose of the study
To assess the size and effort implications
Effort distribution of three different maintenance types
To describe estimation models to predict the
programmers effort on maintenance tasks.

The focus of study is on enhancive, corrective, and
reductive maintenance types according to a
maintenance topology proposed by Chapin et al.
because they are the ones that change the business
rules of the system by adding, modifying, and deleting
the source code.
11
Research Questions and Hypotheses
RQ1: Are there any differences in the
productivity of enhancive, corrective, and
deductive maintenance?
Hypothesis 1 (H1): no difference

RQ2: Are there any differences in the effort
distribution among the maintenance types?
Hypothesis 2 (H2): no difference

12
EXPERIMENTAL
DESIGN
13
Experiment Description
23 students and 1 senior, computer science major
Participants worked on tasks individually in the lab
Time limit of 7 hour.
Participants were given the original source code, a list of maintenance
activities, and a timesheet form.
Participants were required to record time on paper for every activity
performed to complete maintenance tasks. Time information includes start
clock time, stop clock time, and interruption time measured in minute.
Participants used Visual Studio 2005 for maintenance on Windows XP
UCC as a target program
5188 source statements (logical SLOC) in 20 C++ classes
The program was well-structured and well-commented
14
Matrices
Independent variable -- type of maintenance (enhancive,
corrective, and reductive)
Dependent variables -- programmers effort and size of
change.

The programmers effort = total time the programmer spent
to work on the maintenance task excluding interruption time.

The size of change is measured by three LOC metrics, LOC
added, modified, and deleted.
Calculating Maintenance SLOC
Equivalent SLOC =TRCF x AAM

> +
s +
=
=
1 ,
100
1 , ] ) 1 ( 1 [ 1 (
2
AAF f or
UNFM x SU
AAF
AAF f or UNFM x SU x AAF AAF
AAM
TRCF
S
AAF
TRCF = the total SLOC of task-relevant code fragments
S = the size in SLOC (added, modified, or deleted)
SU = the software understandability(10% to 50%)
UNFM = the level of programmer unfamiliarity with the program(0.0 to 1.0)
AAF= adaptation adjustment factor
AAM= adaptation adjustment multiplier


Task-relevant
code fragment
(TRCF)
16
In this experiment, COCOMO II reuse model is used to determine the equivalent
LOC of the maintenance tasks.
RESULTS AND MODELS
17
Resulted Data
Each task requires four maintenance activities
Task comprehension (reading, understanding task requirements,
and asking for further clarification)
Code isolation(locating and understanding code segments to be
adapted)
Editing code(programming and debugging the affected code)
Unit test(performing tests on the affected code)

Timesheet has 490 activity records, totaling 4621 min.
Total of 909 SLOC added, modified, and deleted
402 added
216 modified
291 deleted
18
Effort distribution is different among the
groups (For H2)
Corrective group spent much
time for code isolation
twice as much as that of the
enhancive group
Enhancive group spent
majority of time for editing
code
Effort distribution is statistically
different among three groups
H2 is rejected
19
Productivity
20
Enhancive group produced
1.5 times as many Reductive group deleted and
4 times as many Corrective group modified.
Work done by each group as
per the pattern dictated by
the task design.
Productivity is significantly different
among the groups (For H1)
Box Plot
Corrective group has lowest
productivity
Reductive group has
highest productivity
Productivity between groups
are statistically different
H1 is rejected
Enhancive Reductive Corrective
P
r
o
d
u
c
t
i
v
i
t
y

(
S
L
O
C
/
H
o
u
r
)


40



30



20



10
21
Participant Time Explanatory Models
M1 Effort =78.1 +2.2 * S * EAF
M2 Effort =43.9 +(2.8*Add +5.3*Mod +1.3*Del) * EAF



Effort = Time spent by the participant on all maintenance tasks
EAF = Effort adjustment factor, a product of Programmer capability (PCAP),
Language experience (LTEX), and Platform Experience (PLEX)
Add, Mod, Del = Equivalent SLOC added, modified, and deleted by the participant, respectively
S = Add +Mod +Del
R
2
= The coefficient of determination values
R
2
=0.5
R
2
=0.75
22
R
2
=0.55
R
2
=0.64
M3 Effort =110.0 +2.2 * S * EAF
M4 Effort =79.1 +(2.3*Add +4.6*Mod) * EAF
In this experiment, EAF is defined as the multiplicative of programmer capability (PCAP), language and
tools experience (LTEX), and platform experience (PLEX). The same rating values for these cost drivers
that are defined in the COCOMO II Post-Architecture model is used.
Model M
2
outperforms M
1,
M
3,
M
4
i
i i
i
Actual
Estimate Actual
MRE

=
23
Fig The MRE values obtained by the models in estimating the effort spent by 24
participants
Magnitude of relative error (MRE)
Threats to Validity
Internal design
Groups have imbalanced skills and capability
Imbalanced complexity of tasks on three groups
Incorrect time recorded
Generalizability
Professional programmers are more experienced
Professional programmers are more familiar with
the software maintained
Maintenance process is different in industry
24
Conclusions
Productivity and effort distribution are significantly different
among the maintenance types
SLOC metrics are relevant factors for estimating effort
Three SLOC metrics (added, modified, and deleted) have
different impact on effort
SLOC deleted is an important factor for estimating effort
It is more expensive to modify than to add or delete a statement
Assigning experienced programmers to fixing defect can
save up to 40% of effort
25