MES3053

Software Verification and Validation

Chapter 6
Black-Box Testing
 Learning Objectives
To describe the term Black-box
To describe the importance and role of black-box testing
approach
To describe the black-box testing techniques
To design test cases using the black-box testing techniques
such as:
equivalence partitioning technique
boundary value analysis
decision table

2
 Introduction
Black-box approach view software as black-
box, i.e., without any knowledge of internal
structure of software under test (SUT)
Tester concentrate on software behavior and
its functionality.

Input Output
 Black-box Testing Concept
Also known as functional testing or specification-based
testing.
No knowledge on the internal structure of the SUT are
required.
This approach only depends on the legal inputs and
what are the expected outputs.
Consequently, the tester and the programmer can be
independent of one another.
Advantage: Avoid programmers biasness toward his own work.
 Black-box Testing Concept ..cont.
It can be performed at any testing level.
It occurs from the viewpoint of an end user.
It requires a set of input/output relationships
that verify whether each specification
requirement is implemented correctly.
At least one test case should represent each
entry of the specification requirement
document.
These test cases should test the various
boundary conditions for each entry.
 The Basis of Black-box Testing
BBT is totally based on the testing for requirements and
functionality of the work products.
The objectives of BBT is to uncover the following type
of errors:
Incorrect of missing functions
Interface errors
Errors in data structures or external database access
Performance errors
Initialization and termination errors
 The Basis of Black-box Testing ..cont.

As for all testing techniques, the basis of BBT

lies in the selection of appropriate data set
and the development of its test oracle.
Having a good set of test cases without being
able to differentiate between success or
failure still a useless a test.
Thus, both test case selection and test oracle
must be efficient and effective enough to
detect as many faults as possible.
 Selection Consideration
It should make use of randomly generated inputs but
eliminate any guesswork by the tester.
Input data outside of the specified input range should be
tested to check the robustness of the program.
Boundary cases should be tested to make sure the highest
and lowest allowable inputs to produce proper output.
Whenever numeric data is to be input, the number zero must
be tested.
With real time systems, stress testing should be performed
additionally.
Crash testing should be performed to see what it takes to
bring the system down.
 Selection Consideration ..cont.
Use test monitoring tools to track already
performed tests to avoid repetition and to
aid the software maintenance.
Use finite state machine models to guide the
design of functional tests.
 The advantages of BBT
BBT is more effective on larger units of code than white
box testing.
BBT is normally balance and unprejudiced as tester and
developer are independent of each other.
Tests can be developed earlier as soon as the
requirement specifications are ready.
It helps in identifying the vagueness and inconsistencies
of functional specifications.
 The disadvantages of BBT
Howden (1980) stated that black box approach ignores the
important functional properties of the program, which are
part of its design or implementation and which are not
described in the requirements.
Therefore, many program faults can be overlooked as tests
primarily stresses the user interface and does not consider
the inner structure of the test unit (Moller & Paulish. 1993).
Moreover, it does not provide a basis for coverage measure in
terms of data states.
Goa et al. (2003) have reported that logic errors, which can
account for as many as 32% of all possible faults may not be
identified effectively by black-box approach.
 BBT Techniques
An exhaustive black-box testing requires the software
under test to be executed with all possible input
combinations, which can be seen as impossible (Myers
1979; Whittaker 1997).
For example to test exhaustively a simple program such
as adding only two integer inputs of 32-bits requires 264
distinct test cases to be tested, which would take
hundred years to complete, even if tests were
performed at a rate of thousands per second.
Therefore, it is a goal of a tester by using available
resources to produce test cases that can give a
maximum number of errors.
 BBT Techniques ..cont.
BBT testing techniques that will help to
achieve the goal including:
Equivalence Partitioning
Boundary Value Analysis
Decision Table
Cause-Effect Graph
Robustness Testing
Syntax Testing
Finite State Testing
13
 Equivalence Partitioning
The goals of equivalence partitioning are:
To reduce the number of test cases to necessary minimum.
To select the right test cases to cover all possible scenarios.
Equivalence partitioning divides the input domain of a
program into classes of data that have similar effect on
the program.
These classes of data are said to be equivalence
classes.
A representative test case can be derived from each of
these equivalence classes.

14
 Equivalence Class
An equivalence class is a set of inputs that the program
treats identically when the program is tested.
An equivalence class is used to represent certain
condition on the input domain.
Input conditions are typically used to partition the input
domain into equivalence classes for the purpose of
selecting inputs.
During equivalence partitioning process, both valid (i.e.
return a non-error value) and invalid (i.e. return an
erroneous value) inputs are considered.

15
 General View of Equivalence Partitioning

Invalid inputs
Valid inputs

Input domain Partition

system

outputs

16
 Test Case Generation
A set of guidelines for generating test cases
using equivalence partitioning technique.
The guidelines is based on the input
conditions such as:
If an input condition specifies a range,
If an input condition requires a specific value
If an input condition specifies a member of a set
If an input condition is Boolean
 Test Case Generation ..cont.
If an input condition specifies a range of
values then identify:
One valid equivalence class
Two invalid equivalence class
Example: If a range of values 199, thus three
equivalence classes are derived.
One valid equivalence class: {1,2, ..99}
Two invalid equivalence classes
{x | x < 1}
{x | x > 99 }
 Test Case Generation ..cont.
If an input condition specifies a specific value
then identify:
One valid equivalence class
Two invalid equivalence class
Example: If the specific value is 4055, thus
three equivalence classes are derived.
One valid equivalence class: 4055
Two invalid equivalence classes
{x | x < 4055}
{x | x > 4055}
 Test Case Generation ..cont.
If an input condition specifies a set of values
then identify:
One valid equivalence class
One invalid equivalence class
Example: If a set of values is M = {a, y, n, z}
thus two equivalence classes are derived.
One valid equivalence class {x | x M }
One invalid equivalence classes
{x | x M }
 Test Case Generation ..cont.
If an input condition specifies Boolean value
then identify:
One valid equivalence class
One invalid equivalence class
 Exercise 1
Define the equivalence classes for a program
specification states that the program accepts 4 to
10 inputs and each input represents five digit
integers greater than 10000.

22
 Solution:

Less than 4 Between 4 and 10 More than 10

Less than 10001 Between 10001 and 99999 More than 99999

2 valid equivalence classes + 4 invalid

equivalence classes = 6 equivalence
classes
23
 Test Script
TC ID Input Expected Output Actual output Pass / Fail Remarks

TC 1

TC 2

TC 3

TC 4

TC 5

TC 6

TC 7

TC 8

TC 9
 Exercise 2
A store in city offers different discounts depending
on the purchases made by the individual. In order to
test the software that calculates the discounts, we
can identify the ranges of purchase values that earn
the different discounts. For example, if a purchase is
in the range of $1 up to $50 has no discounts, a
purchase over $50 and up to $200 has a 5% discount,
and purchases of $201 and up to $500 have a 10%
discounts, and purchases of $501 and above have a
15% discounts.
 Solution:

Valid
Invalid Valid Valid Valid
Partition(No
Partition Partition(5%) Partition(10%) Partition(15%)
Discounts)
$0.01 $1-$50 $51-$200 $201-$500 $501-Above
 Exercise 3
Consider a module that is intended to accept a grocery list
with the following specification. The grocery module requires
a user to key-in two types of input which are item-type and
item-name.
The input for item-type is three digit integer values that are between
100 and 999
The item-name, on the other hand is to be characters values and the
length of the characters values is between 5 and 15 characters.
If the user enters valid item-type and item-name, the module will
display the price of the item; otherwise invalid item will be
displayed.
 Boundary Value Analysis (BVA)
It complements equivalence partitioning
Rather than focusing solely on input
conditions, BVA derives test cases from the
output domain as well
Boundary value analysis (BVA) requires selection of
test cases that exercise bounding values
 Boundary Value Analysis (BVA)

Suppose you have a software which accepts

values between 1-1000, so the valid partition will
be (1-1000), equivalence partitions will be like:

Invalid Partition Valid Partition Invalid Partition

0 1-1000 1001 and above

And the boundary values will be 1, 1000 from

valid partition and 0,1001 from invalid partitions.
 Example 1
A store in city offers different discounts depending on the purchases made by the
individual. In order to test the software that calculates the discounts, we can
identify the ranges of purchase values that earn the different discounts. For
example, if a purchase is in the range of $1 up to $50 has no discounts, a purchase
over $50 and up to $200 has a 5% discount, and purchases of $201 and up to $500
have a 10% discounts, and purchases of $501 and above have a 15% discounts.

We can identify 4 valid equivalence partitions and 1 invalid partition as shown below.

Valid
Valid Valid Valid
Invalid Partition Partition(No
Partition(5%) Partition(10%) Partition(15%)
Discounts)
$0.01 $1-$50 $51-$200 $201-$500 $501-Above
 Solution
Boundary values for Invalid partition: 0.00
Boundary values for valid partition(No Discounts): 1,
50
Boundary values for valid partition(5% Discount): 51,
200
Boundary values for valid partition(10% Discount):
201,500
Boundary values for valid partition(15% Discount):
501, Max allowed number in the software
application

 Boundary Value Analysis (BVA) ..cont.

Focuses is on the boundaries of the input
If input condition specifies a range bounded by a certain values,
say, a and b, then test cases should include
The values for a and b
The values just above and just below a and b
If an input condition specifies any number of values, test cases
should be exercise
the minimum and maximum numbers,
the values just above and just below the minimum and maximum
values
Apply guideline (1) and (2) to output conditions.
If internal program data structures have prescribed
boundaries ( e.g : an array has defined limit of 100 entries)
be certain to design a test case to exercise the data
structure as its boundary
Example 2: Equivalence Partitioning
 Valid partitions
The valid partitions can be
0<=exam mark <=75
0<=coursework <=25
 Invalid partitions
The most obvious partitions are
Exam mark > 75
Exam mark < 0
Coursework mark > 25
Coursework nark <0
Exam and course work marks
 Less obvious invalid input EP
invalid INPUT EP should include
 Partitions for the OUTPUTS
EP for valid and invalid OUTPUTS should include
 The EP and boundaries
The EP and boundaries for total mark
 Unspecified Outputs
Three unspecified Outputs can be identified (very
subjective)
Output = E
Output = A+
Output = null
Total PE
Test Cases corresponding to PE exam
mark (INPUT)
Test Case 4-6 (coursework)
Test case for Invalid inputs
Test cases for valid outputs:1
Test cases for valid outputs:2
Test cases for invalid outputs:3
 Exercise 2
Consider a module that is intended to accept a grocery list
with the following specification. The grocery module requires
a user to key-in two types of input which are item-type and
item-name.
The input for item-type is three digit integer values that are between
100 and 999
The item-name, on the other hand is to be characters values and the
length of the characters values is between 5 and 15 characters.
If the user enters valid item-type and item-name, the module will
display the price of the item; otherwise invalid item will be
displayed.
 Possible Solution
Input Domain Valid Partition:
TC1: 100 item-type 999
TC2: 5 item-name length 15

Input Domain Invalid Partition:

TC3: item-type<100
TC4: item-type>999
TC5: item-name length < 5
TC6: item-name length >15
TC7: item-name = integer value

Output Domain Valid Partition

TC8: price of item is induced when valid item-type and item-name are entered.
TC9: invalid item is induced when invalid item-type or/and item-name is entered.
 Test Script
Input Expected Actual
TCID Pass/Fail Remarks
Item-type Item-Name Output Output
99 kacang Invalid type
100 kacang Valid type
101 kacang Valid type
TC1
998 kacang Valid type
999 kacang Valid type
1000 kacang Invalid type
555 kaca Invalid name
555 kacan Valid name
555 kacang Valid name
TC2 555 KacangPanjangI Valid name
555 KacangPanjangIn Valid name

555 KacangPanjangInd Invalid name

TC3 88 kacang Invalid type
TC4 1002 kacang Invalid type
TC5 666 abc Invalid name
TC6 666 abcdefghijklmnoprs Invalid name
TC7 666 122 Invalid name
TC8 666 abcdefg Item price is displayed
TC9 999 abcdefg Invalid Item is displayed
 Exercise: ATM
Consider data maintained for ATM
User should be able to access the bank using
PC and modem (dialing)
User should provide six-digit password
Need to follow a set of typed commands
 Data format (xxx-xxx-xxxx)
Software accepts
Area code:
blank or three-digit
Prefix:
three-digit number not beginning with 0 or 1
Suffix:
four digits number
Password:
six digit alphanumeric value
Command:
{check, deposit, bill pay, transfer etc.}
 Input conditions for ATM
Input conditions
Area code:
Boolean: (the area code may or may not be present)
Range: values defined between 200-905
Prefix:
range specific value >200
Suffix:
range between 1000 - 9999
Password:
Boolean: password may or may not be present
value : six char string
Command:
set containing commands noted previously
 What you have to do?
Discuss this with your group members, and
produce a complete test script based on the
two techniques that we have discuss earlier.

54
 Decision Tables
Represent very complex processes with
multiple decision rules
Decision table: a matrix representation of the
logic of a decision which specifies the possible
conditions for the decision and the resulting
actions.
Best used for complicated decision logic.

Chapter 8 2008 by Prentice Hall 55

 Decision Tables
Condition stubs: the part of a decision table
that lists the conditions relevant to the
decision.
Action stubs: the part of a decision table that
lists the actions that result for a given set of
conditions.

Chapter 8 2008 by Prentice Hall 56

 Decision Tables
Rules: that part of a decision table that
specifies which actions are to be followed for
a given set of condition.
Indifferent condition: in a decision table, a
condition whose value does not affect which
actions are taken for two or more rules.

Chapter 8 2008 by Prentice Hall 57

 Decision Tables
Procedure for Creating Decision Tables
Name the condition and the values that each
condition can assume.
Name all possible actions that can occur.
List all possible rules.
Define the actions for each rule.
Simplify the table.

Chapter 8 2008 by Prentice Hall 58

 Decision Table

Note: for salaried employees the action stub chosen will always
be the sametherefore hours worked is an indifferent condition
 Reduced Decision Table

Because of indifferent condition, the complete decision table

can be reduced to one with fewer rules
Exercise: Simplify the following Decision Table

Conditions/ Rules
Actions
Passed SIM3300 Y Y N N
Passed SIM3301 Y N Y N
Register SIM4200 X
Drop SIM4200 X X X
 Case Study: Payroll System
In one company, the salary for employee is determined by
these rules. The salary is determined through the type
whether salary or hourly. Employee, who has type of salary,
will get a pay base salary. Type of hours worked has three
values. For worker who worked less than 40 hours, the
system will calculate the hourly pay and make an entry in the
absence report. For an hourly worker who has worked
exactly 40 hours, system will calculate the hourly wage. For
employee who worked more than 40 hours, system will
calculate the hourly pay and overtime.
 How to calculate rules?
If all conditions are simply Y/N
Number of Values to the values:
power of the number of 2number of conditions
causes with these values
If 1 condition with 3 values and
3 conditions with 2 values:
31 * 23 = 24
Or, use the values column and
multiply each condition value for
example:
3*2*2*2=24

June 2002 63
 Decision Tables

Chapter 8 2008 by Prentice Hall 64

 Test Script
TC ID Input Expected Output Actual Pass/Fail Remarks
Output
Type Hours
TC1 S 10 Pay-based Salary
Calculate Hourly rate and
TC2 H 30 Produce Absent report
TC3 S 40 Pay-based Salary

TC4 H 40 Calculate Hourly rate

TC5 S 55 Pay-based Salary

Calculate Hourly rate and
TC6 H 60 calculate overtime

65
 Exercise
A salary scale for a system analyst has 3 different scale points. All applicants for
the post are placed on the scale according to the following criteria:
Applicants above age 30 with who have BOTH a degree in Computing Science AND 8
year of relevant experience are placed on the scale of 3 points.
Applicants above age 30 with EITHER a degree in Computing Science OR 8 years
of relevant experience are placed on the scale of 2 points.
Applicants of age 30 and below with BOTH a degree in Computing Science AND 8
years of relevant experience are placed on the scale of 2 points.
Applicants of age 30 and below with EITHER a degree in Computing Science OR 8
years of relevant experience are placed on the scale of 1 points.
All other applicant are rejected

Construct a DECISION TABLE and DECISION TREE for the placing of candidates on
the salary scale.
 Possible Solution

Conditions / Rules
Actions
Age >30 >30 >30 >30 30 30 30 30
Degree CS CS XCS XCS CS CS XCS XCS
8 years Experience Y T Y T Y T Y T
3 Points X
2 Points X X X
1 Point X X
Reject Application X X
 Possible Solution

Conditions / Rules
Actions
Age >30 >30 >30 - 30 30 30 -
Degree CS CS XCS XCS CS CS XCS XCS
8 years Experience Y T Y T Y T Y T
3 Points X
2 Points X X X
1 Point X X
Reject Application X X
 Possible Solution

Conditions / Rules
Actions
Age >30 >30 >30 - 30 30 30 -
Degree CS CS XCS XCS CS CS XCS XCS
8 years Experience Y T Y T Y T Y T
3 Points X
2 Points X X X
1 Point X X
Reject Application X X
 Simplified Solution

Conditions / Rules
Actions
Age >30 >30 >30 - 30 30 30
Degree CS CS XCS XCS CS CS XCS
8 years Experience Y T Y T Y T Y
3 Points X
2 Points X X X
1 Point X X
Reject Application X
 Summary
In this chapter you have learned how to:
describe the term Black-box
describe the importance and role of black-box testing
approach
describe the black-box testing techniques
design test cases using the black-box testing techniques
such as:
equivalence partitioning technique
boundary value analysis
decision table

71