STEP-AUTO Conference, Bangalore

Test Automation 20th Sep 2007

Emerging Trends in Test

Automation

Presenter: Gunasekaran Veerapillai

Designation: Practice Head, Test Automation

Date:

© Copyright 2007 Wipro Technologies Wipro confidential 1

Introduction to Automation

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 1
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Offshore Testing Services Maturity

Offshore Application Testing – ID G00150394 dated 17th August 2007

CONFIDENTIAL© Copyright 2005 Wipro Technologies 3

Maturity of Automation Testing

Level 1 Level 2 Level 3 Level 4

Managed
Services

Key Word Customizable

Driven Framework

Modular
Approach Hybrid
Framework

Script Based Reusable

approach components

Data
Driven

Capture &
Playback
Manual
Testing

CONFIDENTIAL© Copyright 2005 Wipro Technologies 4

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 2
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Test Automation: Introduction

Automation: Facts
 Manual scripts needed
 Ideal for regression testing
 Benefits come from discipline in analysis and planning
 Causes significant schedule impact at introduction
 Require more programming and design skills from testers
 Frequently require maintenance

Automation: Myths
 High immediate ROI from automation
 One tool that fits perfectly
 Need to substitute skilled manual testers
 Preclude manual testing
 100% Automation of all existing manual tests

CONFIDENTIAL© Copyright 2005 Wipro Technologies 5

RCA Analysis – A Case Study

The challenges that a customer, a leading service provider in EMEA,
EMEA, faced during automation

Automation Team
The Solution Approach
Manual test Team
Identifying the applications for
Or
Sc

Timelines Test Team Automation

ga
ri p

Selection of automation
n iz
ts

Environment Implementation percentage for identified

Re

ati

applications.
us

on

Selection of the compatible tool

ab

Follow-up
Str

Usability/User Training for AUT

ilit

uc

Customization of tool/creation
y

tu

Metrics Analysis QA Team of framework

Application version changes
re

Development of
reusable/modular components

Creation of user-friendly
Application
ls

process for running the scripts

Too

Technology Flaw in Framework

Maintenance and enhancement
m

incompatible
process
Tea

ting
I

Feasibility
RO

Cryptic/outdated test cases
Continuous metrics collection

tion

Te s

and analysis
/

Choice of Integration
gs

ma

Applications
vin

Review Process
uto
t Sa

tA

Frequent
Absence of coordination
Co s

changes
Tes

Traceability

CONFIDENTIAL© Copyright 2005 Wipro Technologies 6

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 3
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Building a Test Automation Framework

Test Automation Framework: Approach

Phase 1 : Discovery

Phase 2: Automation

Understand the current Test
Automation practices, identify
gaps, prioritize, develop
roadmap

Prepare skeleton model
Process Definition
Phase 3: Framework
Development
of the framework based Phase 4: Verification
on appropriate systems,
methods, and
infrastructure
Identify the key
components of the
Framework, create the
scripting standards,
identify the reusable
components, document
the metrics analysis and
Conduct in-process and
approach independent
Verification on the
robustness of the
framework.

CONFIDENTIAL© Copyright 2005 Wipro Technologies 8

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 4
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Test Automation Framework: Considerations

Maintainability Reviewability Reliability

Elements
Testing Manual test Application under
Model Cases test

Modularity Test Environment Automation tools

Relationship

Integrity Independence Performance

CONFIDENTIAL© Copyright 2005 Wipro Technologies 9

Test Automation Framework: Components

Plan
Framework for Test Automation

Directory Scripting Reusable Modular

structure guidelines functions approach

Design
Generic Business Driver/Functions Reporting
functions functions scripts scripts

Develop
Automation/DB Test Data Error handling/ Review and
Scripts tables Recovery Traceability

Maintain
Execute and Impact Analysis Metrics Collection Continuous
Report enhancements And analysis Process Imp

CONFIDENTIAL© Copyright 2005 Wipro Technologies 10

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 5
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Test Automation Framework: Design

Application Assessment (Existing Applications)

KT + Verification BASELINE
Assessment Of Test Cases REGRESSION

AUTOMATION TEAM
AUTOMATION BRANCH (Existing Applications)
Existing Applications

Updated Test
FRAMEWORK Library
AUTOMATION cases
DESIGN & DEV Functions

Test Processes
Baseline 1
Test data

Test Documents
AUTOMATION BRANCH (Enhancements)
CONFIGURATION
UPGRADE MANAGEMENT
AUTOMATION
FRAMEWORK

INPUTS TO UPGRADE THE FRAMEWORK

APP-
APP- 5
Enhancement APP-1
APP- 4 APP-
Approach ----
---- 1
6
APP-2 3
APP-
5 APP-
7

Proposed model for Regression Test Automation development

CONFIDENTIAL© Copyright 2005 Wipro Technologies 11

Wip-
Wip-CAFÉ: Wipro’s Framework

CONFIDENTIAL© Copyright 2005 Wipro Technologies 12

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 6
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Test Automation: Managed Services Model

SERVICES / SUPPORTS

Application 1
Client Test Center
DELIVERABLES
Application 2 Wipro PERFORMS
Client Automation Automation
Requests QA Managers Testers
Application 3 Manager CONTINUOUS
IMPROVEMENT
SERVICE
Application 4 REQUESTS

… Customized Tool Vendors

Automation
Framework

Development Team

CONFIDENTIAL© Copyright 2005 Wipro Technologies 13

Test Automation: The emerging areas

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 7
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Automation in Capacity Modeling

Test Results Data Model What-

What-If Analysis
Data Collection Analysis Construction Experiments Of Results

PT Tool Capacity Modeling Tool

Other Performance Benefits:

Testing Tools

Determine System throughput by Business Function

Understand Impact of change in Infrastructure,
Performance application workflow, Workloads, etc.,
Monitoring Tools

Understand implications of adding new services to the
System Resource infrastructure
Monitoring Tools
Identify Network connectivity effects of User Performance

Avoid Outages and Performance problems by planning
ahead

Model Multi-tiered Application Systems

CONFIDENTIAL© Copyright 2005 Wipro Technologies 15

Automation in SOA Testing

• Individual component testing • Interoperability Testing

• Integration, and End-
End-to-
to-End • W3C, WS-
WS-I Standards
Co ter

workflow testing Testing

In
m op
pl er

• Positive, Negative testing, • Client’s Custom Standards

y
ia ab

Boundary value testing

lit Testing
nc il

na
e ity

• Testing of SOAP messages / o • Test for Consistency, and

i
/

responses t Atomicity
nc
Fu SOA
Testing
Approach
Automation possible
across the 4 aspects
using various 3rd
Se party tools
cu
ce i ce

• Test for Identification, Integrity,

ri t • Performance metrics
an rv

Authentication, and
y • Response Time
m Se

Authorization.
• Throughput
or f
rf o

• Security at transport, • Latency

Pe ity

communication protocol, service • Transaction Testing

/ al

description layer, and Firewall

u

• Service Reliability
Q

security
• Probing attacks, Coercive parsing,
malicious contents

CONFIDENTIAL© Copyright 2005 Wipro Technologies 16

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 8
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Automation in Application Security Black Box Testing

Development drawbacks: Attacks possible:
Buffer Overflows
SQL Injection
Broken access control
Cross-Site scripting
Logical Flaws
Parameter
Weak encryption Report Tampering
Session Hijacking Findings Implementation Cookie Poisoning
Privilege Escalation

WIPRO’
WIPRO’S Scan/
Test
escalation APPLICATION identify
visible
&
PENETRATION systems
TESTING
Rollback
METHODOLOGY

Vulnerability Vulnerability
exploitation identification

Time based Testing - Quarterly or on Application Rollouts

CONFIDENTIAL© Copyright 2005 Wipro Technologies 17

In a nutshell…

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 9
STEP-AUTO Conference, Bangalore
Test Automation 20th Sep 2007

Critical factors for Automation success

Treated as a critical process and supported by senior

management

Robust and extendable automation framework

Clear ownership and integration of test technology and

process

Rigorous measurement program to capture progress

Skilled test engineering professionals deployed with the right

tools

CONFIDENTIAL© Copyright 2005 Wipro Technologies 19

Emerging Trends in Test Automation

Gunasekaran Veerapillai - Wipro
Technologies 10