PRESENTED BY: SWAGATIKA SWAIN ROLL NO.

211CS3302

Models are used to understand, specify and develop systems in many disciplines. Models are
used to promote understanding and provide a reusable framework for product development.

In the software engineering process,

models are now accepted as part of a modern object oriented analysis and design approach by all of the major OO methodologies.

Objective:
present an approach for test creation based on a graphical model and a set of heuristics that enable tests to be generated from that model.

RATIONALE FOR MODELLING

Modelling is a very economical means of capturing knowledge about a system and then reusing this knowledge as the system grows. For testing team this type of information is gold,

trying to understand what the System Under Test (SUT) should be doing?
Buried in the test script Or simply lost, waiting to be discovered.

But such type of information are either

By constructing a model of a system that defines the systems desired behaviour for specified inputs to it, a team now has a mechanism for a structured analysis of the system.

Scenarios are described as a sequence of actions to the system with the correct responses of the system also being specified. Hence, test coverage is well understood and test plans are developed in the context of SUT. The largest benefit is reuse

The next cycle of test cycle can start where this one is left.

Quality is improved. If the model is improved, then tests are expanded.

The software to be tested is at integration or system test phase of the development process. Objective is to insure that the system meets its requirement from external point of view. The focus is not on implementation, but how its users will evaluate it. Measure the overall functional compliance of the system rather than code coverage. This tests sometimes known as acceptance tests and are traditional implementation of black box testing.

Developing a specification in the form of a model, even if done late in the process, is a very effective means of:
1. 2. 3.

Discovering defects in the system Rapidly defining the basis for use scenarios of the system. Preserving this investment for future release or other similar systems.

A description of the product or application to be tested. A defined set of features and/or behaviours of the product is needed in order to define the scope of the work. Problem with incomplete description

Incomplete description forces the test engineers to wait until the system is developed or entire context of features known. Sometimes they are ambiguous. The appropriate action is never defined

Specific activities of the practice are (1) Build the model, (2) Generate expected inputs (3) Generate expected outputs, (4) Run tests, (5) Compare actual outputs with expected outputs, and (6) Decide on further actions (whether to modify the model, generate more tests, or stop testing, estimate reliability (quality) of the software)

A variety of techniques/methods exist for expressing models of user/system behavior. These include, but are not limited to: Decision Tables Tables used to show sets of conditions and the actions resulting from them Finite State Machines A computational model consisting of a finite number of states and transitions between those states, possibly with accompanying actions Grammars describe the syntax of programming and other input languages Markov Chains (Markov process) A discrete, stochastic process in which the probability that the process is in a given state at a certain time depends only on the value of the immediately preceding state Statecharts Behavior diagrams specified as part of the Unified Modeling Language (UML). A statechart depicts the states that a system or component can assume, and shows the events or circumstances that cause or result from a change from one state to another.

1. 2. 3. 4. 5.

A Finite State Machine is an ordered quintuple of five sets: A set of inputs, e.g., {Dial/Party Busy, Dial/Party Ready, Hang Up, Party Hangs Up, Party Picks Up, Pick Up} A set of states, e.g., {Busy, Dial Tone, On-Hook, Ringing, Talking} The set of initial states. Let the initial state be {On-Hook} The set of final (terminal) states. Let this set be {OnHook} A partial function mapping from an ordered pair consisting of a state and an input to a subset of states. Figure 2 defines this function for the example. For example, f(Dial Tone, Dial/Party Busy) is {Busy}.

Objective is to verify that the system will behave properly when a sequence of user actions occur. A test script is defined as the entire sequence of actions required to create a complete user scenario for the system-from start-up to shut-down. The test script can be decomposed into a series of individual test primitives that accomplish specific actions. The primitives will be combined in a specific sequence to provide a test script that verifies a unique use scenario for the product.

Test primitives will typically fall into one of the following categories: 1)Provide a stimulus to the system: This is the most obvious - it controls movement from one state to another in the behavioral models. The action can be directly controlled from the test execution environment. 2)Verify that the system responded correctly: Verification can be difficult to accomplish because the systems response must be determined and then compared to an expected response and/or verify that we are in the correct state in our system. The degree that verification is used and the means for performing verification will vary widely with application type and test objectives. Not all actions require direct verification - sometimes the fact that the next stimulus will be accepted is an acceptable means of verifying that the system is currently in the correct state.

3) Set-up the system testing environment: Tests will often need to control the environment so that the next action will follow a predictable path. In many environments there are situations that have several potential outcomes from the same input. These are often due to multiple resources being available for the action. To make this situation deterministic, and therefore easier to test, the environment can be temporarily preallocated or constrained in order to force a specific sequence or response to occur. 4) Report and/or log the results: Depending on the degree of automation in the test execution environment commands can be embedded that will report the test results to a reporting system. These embedded commands range from simple print statements for log files, all the way to sophisticated inter-process communications with a test management system. In any situation, the means to record and analyze results should be planned.

There are several approaches that can be used to develop tests from a model. Central to most of these is the concept of a path. A path is a sequence of events or actions that traverse through the model defining an actual use scenario of the system. Each element in a path, a transition or state, can have some test primitives associated with it. The primitives will define
What test actions are required to move system from current state to next state. Verify that the state is reached or check that the system has responded properly to previous input.

When this script is applied to the system, the system will follow the same path(or sequence). This process can be repeated for another path, which define another scenario and verifies another sequence of actions. The test objective might range from testing basic functionality to a complete product verification suite.

The effectiveness of model-based testing is primarily due to the potential for automation it offers. If the model is machine-readable and formal to the extent that it has a well-defined behavioral interpretation, test cases can in principle be derived mechanically. Often the model is translated to or interpreted as a finite state automation or a state transition system. This automaton represents the possible configurations of the system under test. To find test cases, the automaton is searched for executable paths. A possible execution path can serve as a test case. This method works if the model is deterministic or can be transformed into a deterministic one. Depending on the complexity of the system under test and the corresponding model the number of paths can be very large, because of the huge amount of possible configurations of the system. For finding appropriate test cases, i.e. paths that refer to a certain requirement to proof, the search of the paths has to be guided. For test case generation, multiple techniques have been applied and are surveyed in.

Some of the techniques are:

1.
2. 3. 4. 5.

Test case generation by theorem proving Test case generation by constraint logic programming and symbolic execution Test case generation by model checking Test case generation by using an event-flow model Test case generation by using a Markov chains model

T-VEC

Model-based test automation solutions reduce cost of verification and validation by identifying defects early to reduce rework
It can be used at any stage of the programming process: design, coding, or testing. It is fully integrated into the Microsoft .NET environment: AsmL models can interoperate with any other .NET assembly, no matter what source language it is written in.

AsmL

Model-Based Testing (MBT) can result in the following benefits: Shorter schedules, lower cost, and better quality A model of user behaviour is one major artifact of ModelBased Testing Enhanced communication between developers and testers in conjunction with developing the model Early exposure of ambiguities in specification and design while developing the model Capability to automatically generate many non-repetitive and useful tests Test harness to automatically run generated tests Combination of MBT artifacts eases the updating of test suites for changed requirements (typically, only the model need be updated) Capability to evaluate regression test suites (one can know what level of test coverage they obtain) Capability to assess software quality (if tests are generated from a Markov model, the results of the tests provide inputs to typical software reliability models satisfying appropriate assumptions

Comfort factor

This is not your parents test automation

Skill sets

Need testers who can design Models can be a significant upfront investment Will never catch all the bugs

Expectations

Metrics

Bad metrics: bug counts, number of test cases Better metrics: spec coverage, code coverage

Models are useful abstractions

In specification and design, they help us think and communicate about complex artifacts by emphasizing key features and suppressing details Models convey structure and help us focus on one thing at a time

We can use them in systematic testing

If a model divides behavior into classes, we probably want to exercise each of those classes! Common model-based testing techniques are based on state machines, decision structures, and grammars

but we can apply the same approach to other models also.

http://www.goldpractices.com/practices/mbt/ http://ix.cs.uoregon.edu/~michal/book/slides/p df/PezzeYoung-Ch14-ModelBased.pdf Apfelbaum L., Doyle J.,paper, Model Based Testing, Software Quality Week Conference in May, 1997. Dalal S.R., Jain A.,Karunanithi N.,Leaton J.M., Lott C.M.,Patton G.C. and Horowitz B.M.,paper, Model-Based Testing in Practice, Proceedings of ICSE99, May 1999 (ACM Press).