Testing Practice

Automation Testing

10/07/2013
 10/07/2013
Agenda Testing Practice
Public

1. What is Test Automation

2. Benefits of Automation
3. Automation Fundamental concepts
4. When not to Automate
5. When to Automate
6. Manual vs. Automation testing
7. Factors to be considered – Test Automation
8. How to select Automation tool

2
 10/07/2013
What is Test Automation Testing Practice
Public

▶ What is Test Automation ?

– Software Test Automation is the process of automating the steps of

manual test cases using an automation tool Or utility to shorten the
testing life cycle with respect to time.

– When application undergoes regression, some of the steps might be

missed out or skipped which can be avoided in Automation.

– Automation helps to avoid human errors and also expedite the testing
process.

– To implement the Test Automation detailed planning and effort is

required.

– Automation saves time and effort which results in reduction of the Test
life cycle.

3
 10/07/2013
Benefits of Automation Testing Practice
Public

Benefits of Automation

▶ Reliable : Tests perform precisely the same operations each time they are run,
thereby eliminating human error.

▶ Repeatable : You can test how the software reacts under repeated execution of
the same operations.

▶ Programmable : You can program sophisticated tests that bring out hidden
information from the application.

▶ Comprehensive : You can build a suite of tests that covers every feature in your
application.

4
 10/07/2013
Benefits of Automation Testing Practice
Public

▶ Reusable : You can reuse tests on different versions of an application, even if

the user interface changes.

▶ Better Quality Software : Because you can run more tests in less time with
fewer resources.

▶ Fast : Automated Tools run tests significantly faster than human users.

▶ Cost Reduction : As the number of resources for regression test are reduced.

5
 10/07/2013
Testing Practice
Automation Fundamental Concepts Public

▶ Automation life cycle is a subset of the entire test life cycle.

▶ Automation planning can be initiated in parallel to the test planning

phase.

▶ Factors to be considered in automation planning are as follows:

– Stability of AUT (Application under test)
– No of regression cycles to be performed
– Compatibility of App platform with testing tools
– Cost benefit analysis (ROI)
– Availability of skilled resources

6
 10/07/2013
Testing Practice
Automation Fundamental Concepts Public

▶ Automation will not :

– Eliminate or Reduce Manual testers

• Automation can not test for aesthetics
• Can not design test cases
• Validate the results
• Replace the business knowledge

– Be a substitute for:
• Walkthroughs, inspections, adherence to coding standards, Project
Management and etc

– Find More “bugs” as Test Cases find the bugs

• Automated Test Scripts are only as good as the test case they
replicate

7
 10/07/2013
When not to Automate Testing Practice
Public

▶ Application is not stable.

▶ Regression test is not necessary.
▶ Main Application has lot of interdependency with other Applications which in turn
cannot be automated.
▶ Time and resources are a constraint.
▶ Complexity of Application.
▶ Customized applications with short life span.

8
 10/07/2013
When to Automate Testing Practice
Public

▶ Spending a lot of time by testing manually.

▶ Regression test is essential.
▶ Check the functionality with huge set of data.
▶ Verify database which contains additional data.
▶ Test application with in less time.
▶ Code coverage is vast.
▶ Life span of the product is long.

9

Manual vs. Automation Testing
▶ Manual vs. Test Automation
Pros
Manual 1.Quick to Start
testing 2.Initial Results are provided sooner
3.Complex scenarios are more easily
executed at an earlier time.
4.Lower technical recourse requirement
5.Lower level of technical skills are
required by testers.
Automation 1. Test execution is faster.
Testing 2. Increased accuracy and check point
control provides greater assurance.
3. Easier maintenance.
4.Easier and faster scenario propagation.
5. Automatic reporting.
6. Provides object property checks .
7. Automation can start as soon as screens
are available.
10/07/2013
Testing Practice
Public
Cons
1. Execution time is much longer than
automation.
2. Amount of time to run all tests increases in a
greater magnitude as more tests are added.
3.Regression of existing functionality uses more
of time available for testing as more
functionality is added.
4. Maintenance of test steps is an overhead
especially if functionality changes a lot.
1. Requires more development knowledge and
processes.
2. Requires more knowledge of framework and
management of script development.
3. More cost for tools and higher spec hardware.
10
 10/07/2013
Factors to be considered - Test Testing Practice
Automation Public

▶ Time Plan Approval

Investment of time needed, for delivering the framework
Scheduled time-plan approval

▶ Commitment on Priority
Commitment from the management and senior managers on the priority
assigned to this activity, till the completion of framework development

▶ Cost and Budget

Dedicated Budget A dedicated budget to be allocated, which includes costs
related to test tool, development, deployment, resource and training.

11
 10/07/2013
Factors to be considered - Test Testing Practice
Automation Public

▶ Well-defined testing Process

Well-defined quality control procedures and test execution standards.
No Ad-hoc testing
Define the tests
Define the test-coverage
Define test criteria at each stage

▶ Dedicated Resources
A dedicated team is needed for effective test automation.
Non-dedicated team will execute the test automation with their own limitations

12
 10/07/2013
How to select automation tool Testing Practice
Public

Following factors to be considered

▶ Platform and OS independence
▶ Data driven capability (Input Data, Output Data, Metadata)
▶ Customizable Reporting (DB Access, crystal reports)
▶ Easy debugging and logging
▶ Version control friendly – minimal binary files
▶ Extensible & Customizable (Open APIs to be able to integrate with other tools)
▶ Common Driver (For example, in the Java development ecosystem, that means
Ant or Maven and the popular IDEs). This enables tests to integrate with the
developers' workflows.
▶ Support unattended test runs for integration with build processes and batch
runs. Continuous integration servers require this.
▶ Email Notifications (automated notification on failure or threshold levels). This
may be the test runner or tooling[clarification needed] that executes it.
▶ Support distributed execution environment (distributed test bed)
▶ Distributed application support (distributed SUT)

13
 10/07/2013
Some Popular Automation tools Testing Practice
Public

Tool Name Produced By

HP QuickTest Professional HP Software Division

Rational robot IBM Rational

IBM Rational Functional Tester IBM Rational

SilkTest Borland

Oracle Application Testing Suite Oracle Corporation

Visual Studio Test Professional Microsoft

Selenium Open source

Fitnesse Open Source

14
Thank you

Atos, the Atos logo, Atos Consulting, Atos Worldline, Atos Sphere,
Atos Cloud and Atos WorldGrid
are registered trademarks of Atos SA. June 2011

© 2013 Atos. Confidential information owned by Atos, to be used by

the recipient only. This document, or any part of it, may not be
reproduced, copied, circulated and/or distributed nor quoted without
prior written approval from Atos.

10/07/2013