Test Automation: An Architected Approach

Dan Young March 17th, 2005 dyoung@bbandt.com

What goes wrong?

Vendors push record/playback as a viable solution Managers want to believe the hype and believe that they can see results immediately Users under pressure rush for any implementation that works

Planning an Automation Project

Requirements Balancing Personnel An Automation Plan Return on Investment

An Architected Solution
Encapsulated functions (functions within functions) Encapsulated test cases Managing Results Pre-scripting

An Architected Solution: Encapsulated Non-Scripted Functions

Low-level nonscripted functions handle common processes that a business analyst isnt necessarily concerned with. May accept parameters May return a value Recursion may be used at this level

An Architected Solution: Encapsulated Scripted Functions

Handle the entire process of what a test case does Accept parameters, and parse variables as appropriate Most likely do not return a value Do write a result

An Architected Solution: Sample Application

Contact Management software Sample Ability: Add New Contact

An Architected Solution: Encapsulated Scripted Functions II

The first step is to parse the parameters passed from the test case Individual values are assigned to variables for later use

An Architected Solution: Encapsulated Scripted Functions III

Function then performs the entire test case, including launching the app, selecting the database, populating the fields, verification and cleanup Errors (expected or unexpected) are handled within the function

An Architected Solution: Encapsulated Test cases

Test case focuses on a particular area of functionality Test case consists of initializing variables and making a single function call. Anyone can add test cases with very little training, or even knowledge of the AUT

An Architected Solution: Managing Results: Flat File Model

All results written to an external txt or csv file Results written to a particular directory structure for ease of comparison by a third party comparison utility Only files that fail comparison need to be examined Root directory of results directory structure is unique so that results of multiple test runs can be captured

An Architected Solution: Managing Results: Database Model

Automation tool can write results directly to a database. Queries and Reports can be designed to quickly show test status Could be used to populate a web page with up-to-theminute test results.

An Architected Solution: Pre-Scripting

With simplified test case entry, all fields and scripted functions can be agreed upon in advance Test cases can be set up before functions are written

Selling the Architected Solution

Management is under pressure to get results quickly, but good automation architecture requires planning and time for infrastructure New projects: Balance the need for quick results with the need for a maintainable architecture Existing projects: Architecture can be built in as automation for particular areas of the product needs revised

Selling the Architected Solution: Benefits

Long-term cost and risk reduction since a modular approach minimizes maintenance, and a lot of the infrastructure is reusable between projects and products Consistent levels of automated testing across projects Provides more defined career paths for QA staff Increased employee satisfaction and productivity since an employee can focus on areas of interest Anyone with product knowledge can begin entering test cases quickly Automation comes on line more quickly since manual testing is done simultaneously Readily available results save time for development as well as QA

Selling the Architected Solution: Pitfalls to Avoid

Include all necessary people in planning (e.g. project manager, product manager, testing lead, system architect). Each role may have information that can be designed into the initial architecture instead of added later Avoid the temptation to shortcut the process Try to have some idea of what comes after the current project

Selling the Architected Solution: Pitfalls to Avoid II

Do not over-encapsulate

Conclusion
Properly planned and implemented automated testing will reduce project cost and risk in the long term Good planning and management support are imperative for maintainable, lasting automation Questions??