Leiden Institute of Advanced Computer Science

System s Development and Project Management Software quality assurance

Prof. dr. Thomas Bck

Leiden Institute of Advanced Computer Science

Dates

Feb. 1 Feb. 2 Feb. 9 Feb. 15 Feb. 16 Feb. 22 Feb. 23 Mar. 1 Mar. 2 Mar. 8 Mar. 9 Mar. 18

14:45 17:30 13:45 16:30 13:10 15:45 14:45 17:30 15:15 18:00 14:45 17:30 13:15 15:45 14:45 17:00 13:45 16:30 14:45 17:30 13:45 16:30 15:00 17:00

Introduction, Project Description STEP WISE Approach to Project Planning STEP WISE Approach to Project Planning, SAVE ENERGY Case Selecting an Appropriate Software Dev. Approach Activity Planning and Resource Allocation Software Effort Estimation Risk management, project escalation Exam Risk Management, Project monitoring and control Software Quality Assurance Managing People; Contract Management Trade Fair
2

Leiden Institute of Advanced Computer Science

STEP WISE overview

Some relate to qualities

1. Identify project objectives

0.Select project 3. Analyze pr. characteristics

2. Identify project infrastructure

Installation standards, procedures. Special quality requirements ? Entry, exit, process requirements ?

Review Lower level detail

4. Identify products and activities 5. Estimate effort for activity 6. Identify activity risks

For each activity

10. Lower level planning 9. Execute plan

7. Allocate resources 8. Review/ publicize plan

Review of overall quality aspects.

Leiden Institute of Advanced Computer Science

Software quality
! " Of increasing concern
!"

E.g. safety critical systems, dependence on core IS Need to make project progress visible Every task has a deliverable Errors accumulate with each stage Errors become more expensive to remove the later they are found It is difficult to control the error removal process (e.g. testing)
4

! " Project control concerns:

!" !" !" !" !"

Leiden Institute of Advanced Computer Science

Software quality
! " Three Specifications:
!" !" !"

Functional: What the system is to do. Quality: How well the functions are to operate. Resource: How much is to be spend on the system.

Leiden Institute of Advanced Computer Science

ISO 9126 software qualities

Functionality Reliability Usability Efficiency Maintainability Portability Does it satisfy user needs? Can the software maintain its level of performance? How easy is it to use? Relates to the physical resources used during execution Relates to the effort needed to make changes to the software How easy can it be moved to a new environment?
6

Leiden Institute of Advanced Computer Science

ISO 9216
! " Defined in 1991 ! " To tackle the question of definition of software quality ! " Also suggests sub-characteristics of the main ones outlined here (outside main standard) ! " See next slides

Leiden Institute of Advanced Computer Science

Functionality sub-characteristics
! " Suitability ! " Accuracy ! " Interoperability
!"

Ability of software to interact with other software components Degree to which software adheres to application-related standards or legal requirements, e.g. audit Control of access to the system

! " Compliance
!"

! " Security
!"

Leiden Institute of Advanced Computer Science

Reliability sub-characteristics
! " Maturity
!"

Frequency of failure due to faults - the more the software has been used, the more faults will have been uncovered and removed

! " Fault-tolerance ! " Recoverability

!"

Note that this is distinguished from security see above

Leiden Institute of Advanced Computer Science

Further quality sub-characteristics

! " Usability sub-characteristics:
!" !" !"

Understandability: easy to understand? Learnability: easy to learn? Operability: easy to use? Time behavior, e.g. response time Resource behavior, e.g. memory usage

! " Efficiency sub-characteristics:

!" !"

10

Leiden Institute of Advanced Computer Science

Further quality sub-characteristics (cont d)

! " Maintainability sub-characteristics:
!" !" !" !"

Analyzability: ease with which the cause of a failure can be found Changeability: how easy is software to change? Stability: low risk of modification having unexpected effects Testability

11

Leiden Institute of Advanced Computer Science

Quality sub-characteristics (cont d)

! " Portability sub-characteristics:
!" !" !"

!"

Adaptability Installability Conformance: standards that have bearing on portability (compare to compliance ) - e.g. use of high-level language Replaceability: factors giving upwards compatibility - downwards compatibility is excluded
12

Leiden Institute of Advanced Computer Science

Quality relationships
! " Indifferent
!"

One has no effect on the other A system can only be good in respect to one quality at the expense of another A system which is good in respect to one quality is likely to be also good in respect to the other

! " Competitive
!"

! " Complementary
!"

13

Leiden Institute of Advanced Computer Science

Internal vs. external qualities

Translate into

External qualities: ! " Changeability

Internal qualities (SQCs): ! " Modularity ! " Generality ! " Expandability ! " Self-descriptiveness ! ! ! ! " " " " Simplicity Modularity Instrumentation Self-descriptiveness
14

! " Testability

Leiden Institute of Advanced Computer Science

Internal vs. external qualities (cont d)

External qualities: ! " Portability Internal qualities (SQCs): ! " Modularity ! " Self-descriptiveness ! " Machine independence ! " Software system independence

15

Leiden Institute of Advanced Computer Science

Using ISO 9126 quality standards

! " Judge the importance of each quality for the application
!" !"

E.g. safety critical systems - reliability very important Real-time systems - efficiency important E.g. mean-time between failures for reliability Response-time for efficiency

! " Work out ways of measuring quality

!" !"

16

Leiden Institute of Advanced Computer Science

Using ISO 9126 quality standards (cont d)

! " Map measurement onto ratings scale to show degree of satisfaction:
!"#$%&#"' (#")*+' ! "# "%$# '%()# ((%($# ('%")# + ")# !,-.&/#'(01 2+' $# &# *# "# (# )#

17

Leiden Institute of Advanced Computer Science

Using ISO 9126 quality standards (cont d)

! " Work out how ratings are to be combined ! " ISO 9126 does not specify how to do that only that some method must be devised
!"#$%&'( !"#$%&$#$'() -..$/$"0/() 34%&$#$'() 8 ,&#$( )*+,-&#./0(1#2( 3#&%.4(152( *) 1) 5) ( +) *) 5) ( 6/,-0(1#(7(52( ,*) ,2) ,6) 9:(

18

Leiden Institute of Advanced Computer Science

Software measurement
! " May apply to:
!" !"

Final products Intermediate products (predictive metrics) Relative or binary (does it/does it not exist?) Direct or indirect Tightly or loosely coupled

! " May be:

!" !" !"

19

Leiden Institute of Advanced Computer Science

Quality specification
! " Each project has three sets of requirements
!" !" !"

Functional requirements: what the system is to do Quality requirements: how well it is to do it Resource requirements: how much it is going to cost

Leiden Institute of Advanced Computer Science

Quality specification, e.g. ease of installation

! " Definition of attribute
!"

The amount of effort needed to install the package for a new customer Hours Time needed to install system at three different sites

! " Measurement scale

!"

! " How tested

!"

21

Leiden Institute of Advanced Computer Science

Quality specification, e.g. ease of installation (cont d)

! " Worst acceptable limit
!"

4 hours 1 hours 30 minutes

! " Planned limit

!"

! " Best achievable

!"

Define these for user-friendliness!

22

Leiden Institute of Advanced Computer Science

How do we achieve product quality?

! " The problem: quality attributes tend to be retrospectively measurable ! " Need to be able to examine processes by which product is created beforehand ! " The production process is a network of subprocesses ! " Output from one process forms the input to the next ! " Errors can enter the process at any stage
23

Leiden Institute of Advanced Computer Science

Product vs. process quality management

! " Errors are more expensive to correct at later stages
!" !"

Need to rework more stages Later stages are more detailed and less able to absorb change Error typically 10 times more expensive to correct at coding stage than at requirements stage 100 times more expensive at maintenance stage
24

! " Barry Boehm

!" !"

Leiden Institute of Advanced Computer Science

For each activity, define!

! " Entry requirements
!" !"

These have to be in place before an activity can be started Example: a comprehensive set of test data and expected results be prepared and independently reviewed against the system requirement before program testing can commence

25

Leiden Institute of Advanced Computer Science

For each activity, define! (cont d)

! " Implementation requirements
!" !"

These define how the process is to be conducted Example: whenever an error is found and corrected, all test runs must be completed, including those previously successfully passed

26

Leiden Institute of Advanced Computer Science

For each activity, define! (cont d)

! " Exit requirements
!" !"

An activity will not be completed until these requirements have been met Example: the testing phase is finished only when all tests have been run in succession with no outstanding errors These requirements may be laid down in site standards, or a quality plan may be drawn up for a specific project
27

! " Software quality plan

!"

Leiden Institute of Advanced Computer Science

Inspections general principles

! " When a piece of work is completed, copies are distributed to co-workers ! " Time is spent individually going through the work noting defects ! " A meeting is held where the work is then discussed ! " A list of defects requiring re-work is produced

28

Leiden Institute of Advanced Computer Science

Inspections advantages of approach

! " An effective way of removing superficial errors from a piece of software ! " Motivates the software developer to produce better structured and self-descriptive code ! " Spreads good programming practice ! " Enhances team-spirit ! " The main problem maintaining the commitment of participants
29

Leiden Institute of Advanced Computer Science

General movement to give software more quality

! " Increase the visibility of software ! " Put method into processes of development ! " Check intermediate stages

30

Leiden Institute of Advanced Computer Science

External standards ISO 9001:2000

! " Ensure that a monitoring and control system to check quality is in place ! " Only certification of development process ! " Not software development-specific ! " Main activities:
!" !" !" !" !"

Determine customer needs and expectation Establish quality policy Design product creation process with responsibilities Measure effectiveness and efficiency Take corrective action
31

Leiden Institute of Advanced Computer Science

ISO 9001:2000 (Criticism)

! ! ! ! " Expensive, time consuming " Putting smaller firms at a disadvantage " Preoccupation with certification " Can distract attention from real problems of producing quality products

32

Leiden Institute of Advanced Computer Science

External standards Capability Maturity Model

! " Levels of Process Maturity:
!" !"

Level 1 Initial: haphazard procedures followed

" Any organization at this level by default !

Level 2 Repeatable: basic project management procedures

" The way individual tasks are carried out will depend largely on person doing it.

33

Leiden Institute of Advanced Computer Science

External standards Capability Maturity Model

! " Levels of Process Maturity:
!" !" !"

Level 3 Defined: how should each task in the softw. Development life cycle be done Level 4 Managed: products and processes are subject to measurement and control Level 5 Optimizing: improvements based on measurement data

34

Leiden Institute of Advanced Computer Science

External standards Capability Maturity Model (cont d)

Level Initial Repeatable Defined Managed Optimizing Key Process Areas Not applicable

Software Configs, Version Control Configuration management, i.e., quality

assurance, project planning, etc. Peer reviews, integrated software management, training program, etc. Quality management, process measurement and analysis Process change management, technology innovation, defect prevention

35

Leiden Institute of Advanced Computer Science

Summary
! " Quality = vague concept. Requirements have to be carefully defined. ! " There have to be practical ways to test relative presence / absence of a quality. ! " Most qualities can only be tested when system is completed. ! " We need ways of checking during development. ! " Some procedures focus on testing products, others on evaluating quality of the development process.

36