USER ACCEPTANCE TEST STANDARDS This document contains the User Acceptance Testing standards for Application Development

used within the Ministry. It includes a template and instructions for the creation of a User Acceptance Test Plan. Table of Contents VERSION CONTROL 1. INTRODUCTION 1.1 Audience 1.2 Purpose 1.3 Assumptions 1.4 Other Standards 2. USER ACCEPTANCE TEST STANDARDS 2.1 Overview 2.2 New System Test 2.3 Regression Test 2.4 Limited Testing 2.4.1 Form-Based Testing 2.4.2 Business Process Testing 2.4.3 Report Testing 3. ASSESSING THE TEST TYPES FOR A TEST PLAN 3.1 Overview 3.2 Assessing the Test Type Required 4. CREATING A USER ACCEPTANCE TEST PLAN 4.1 Test Instructions 4.2 Form-Based Test Scripts 4.3 User Security Matrix 4.4 Business Process Test Scripts 4.5 Report Test Scripts 4.6 Defect Tracking Form 4.7 Defect Tracking Log 5. CONCLUSION APPENDICES Appendix A: USER ACCEPTANCE TEST PLAN TEMPLATE VERSION CONTROL This section of the document records the various versions or releases of this document. Version 0.1 1.0 Description First Draft QA/Edits Distribution Whole Document Whole Document Date January 17, 2000 February 3, 2000 Author Diane Barrand Diana Von Ratenberg Organization Pangaea MSRM IMB

1. Introduction This section outlines the Audience and Purpose of this standards document. 1.1 Audience This document is to be used by the Application Managers or anyone tasked with preparing a User Acceptance Test Plan for the Ministry. 1.2 Purpose The purpose of this document is to define the standards and deliverables for User Acceptance Testing for the Ministry. This document also provides a template and instructions for developing a User Acceptance Test Plan for a specific application. 1.3 Assumptions In order to make effective use of this standard; the audience must be familiar with the Ministry 's System Development Life Cycle Overview, of which this standard is part. 1.4 Other Standards Refer to System Development Life Cycle Overview for more information.

User Acceptance Testing is part of the Implementation Phase in the MSRM Systems Development Life Cycle (the 6th Phase in the Systems Development Life Cycle). This is the phase where the application is delivered to the Ministry for user acceptance. 2. User Acceptance Test Standards The purpose of this section is to define the levels of testing required for User Acceptance Testing, depending on the extent of change being applied to the application. 2.1 Overview This section is a descriptive overview, which will assist in developing a User Acceptance Test Plan as described in Section 3 of this document. The possible levels of testing required in a User Acceptance Test Plan are: New System Test: where the application to be tested is entirely new (is not an enhancement or system upgrade). Regression Test: where the amount of change in an existing application requires a full system retest. Limited Test: where the amount of change in an existing application requires only change specific testing. 2.2 New System Test Purpose: To ensure that the system meets all specified objectives. To ensure that all requirements are included in the new application. When to begin planning: Plan the User Acceptance Test as early in the life cycle as possible. It is important that the original project scope is verified in the User Acceptance Test Plan by including the key points of the original project documentation in the test scripts. The Following Material can Contribute to the Test Plan (if available): Software Requirements Specification documentation Software Design Description Document Project Statement Technical documentation Online Help/Documentation User Training material User Manual Application Change Requests Contractor Test plans Contractor Test Results Similar User Acceptance Test Plans (from other applications) Include Test Script Types : Form-Based test scripts Business Process test scripts Report test scripts 2.3 Regression Test Purpose: To ensure the "entire" system is working correctly. To ensure that application changes have not changed existing functionality. To ensure that the new development work meets all requirements. When to begin planning: Plan the User Acceptance Test as early in the life cycle as possible. It is important that the original project scope is verified in the User Acceptance Test Plan by including the key points of the original project documentation in the test scripts. Material to Review: Software Requirements Specification documentation Software Design Description documentation Project Statement Technical documentation specific to the changes Online Help/Documentation User Training material User Manual Contractor Test plans Contractor Test Results Previous User Acceptance Test Plans

Include Test Script Types : Form-Based test scripts Business Process test scripts Report test scripts 2.4 Limited Testing The section below outlines the different types of testing in cases where a full system test is not required. There are three types of limited testing: : Form-Based test scripts Business Process test scripts Report test scripts 2.4.1 Form-Based Testing Purpose: To verify that individual application forms are performing correctly. To ensure that all required fields and buttons exist on the application forms. To ensure that the flow of information and data entry is logical and correct based on the application 's business requirements. To ensure that a new form added to an existing application is functioning according to specifications. To verify that new functionality added to an existing form has not adversely affected the existing functionality on that form. To ensure that the flow of fields on a new form is sensible. To check common relationships of fields between forms. To verify navigation between forms. Material to Review: Requirement documentation Technical documentation Design documentation Contractor Test plans Contractor Test Results 2.4.2 Business Process Testing Purpose: To ensure that all business related functions and processes are supported by the application. To ensure that the flow of data and screens is logical for each business process. To ensure that security and access requirements are met. To test the application with "real" scenarios. To test all batch processes. To test for performance related problems. To test the application 's interfaces. Material to Review: Requirement documentation Technical documentation Design documentation Contractor Test plans Contractor Test Results Previous User Acceptance Test Plans Security Information Business Process Documentation 2.4.3 Report Testing Purpose: To ensure that the new report meets its requirements. To ensure that the data extracted for the new report is correct. To ensure that the report format is correct and logical. To ensure that the print process is working correctly. Material to Review: Software Requirement Specification documentation Software Design Description Documentation Technical Specifications

 Contractor Test Plans Contractor Test Results 3. Assessing the Test Types for a Test Plan The purpose of this section is to describe the process of planning and assessing the level of testing required for a specific application. 3.1 Overview To complete a User Acceptance Test Plan for a specific application, the Application Manager must first plan the tests based on the initial application project documentation. By researching the purpose of the development work, and the degree to which this development work will affect the rest of the application, the test planner can then begin to create the scripts for testing. (A test script ' includes the individual test steps to be executed in order to verify an application is working as expected). The development of a User Acceptance Test Plan involves a number of iterative steps: 1. Assess the type of testing required 2. Develop the procedures and instructions for testing 3. Develop the necessary test scripts 4. Execute the test scripts 5. Report any defects 6. Retest any fixes This section covers the first of these steps. The remaining steps are covered in Section 4 of this document. 3.2 Assessing the Test Type Required The criterion for determining the degree and type of testing that may be required is listed in the chart below. Use this chart as a guide for determining what test scripting will be required for your particular User Acceptance Test Plan. If the application changes to be tested fall in more than one criterion multiple test script types may be required. Once you have determined the appropriate test types to follow, complete the User Acceptance Test Template as described in the section below. Application Criteria New Application (not replacing an existing application). Test Criteria The Application Manager developing the User Acceptance Test Plan should be as involved as possible in the design and reviews of the new application with the contractor. The User Acceptance Test Plan should be developed with communication from the contractor and with as much information gathered through the system documentation as possible. The Test Plan should be developed using the required aspects of the system that is being replaced. Test scripts for any enhancements to the new (replacement system) should be developed as if the application is a new application basing information of requirement and design documents. Running a parallel test with the old system, and comparing critical report results would be the optimal test scenario for any functionality that is to be duplicated in the new system. .

New Application (replacing an existing application).

In this case running a parallel test with the system using the old database and the system using the new database is advised. Database Change (no other Comparing critical report results would be the optimal test scenario to change to the application e.g. ensure that the new application database and drivers are producing upgrade from Oracle 7.3 to identical results to the old system. Performance testing should be Oracle 8.05) included in these test criteria. Create a User Acceptance Test Plan to manage the parallel test. Application Enhancements (the existing application is enhanced with new functionality). Form-based test scripts should be developed for any forms that are being added to the existing application to ensure that each of the new forms is behaving correctly and is meeting the requirements. Business process test scripts should be developed and tested to ensure that the new functionality is integrated with the existing application correctly, and to ensure that no existing functionality has been lost in the enhancement process. Test scripts should be developed based on the specific requirements for any new reports. In this case, unchanged forms may not require testing. E.g. if they are not involved in any

business processes which have been changed. New form-based test scripts should be developed based on the requirements documents to ensure that the application meets its enhanced requirements. Existing test scripts can be amended in this case. Business process test scripts should be developed and tested to Application Enhancement ensure that the enhanced functionality is integrated with the existing (the existing application is application correctly, and to ensure that no existing functionality has being enhanced to change the been lost in the enhancement process. Test scripts should be existing functionality). developed based on the specific requirements for any enhanced reports. In this scenario, any form that has been changed, or is affected by a change, new data or new functionality should be tested to assure that existing functionality is not lost. This will ensure that the new requirements are being met. Infrastructure Change (the application is not changing but a test is required to port it to In this case, a full regression test of the form-based test scripts and all a new environment, new business processes should be repeated as new environments can server, or the application be create unexpected problems for an existing application. being utilized for a new division or purpose). 4. Creating a User Acceptance Test Plan The User Acceptance Test Plan Template is an independent document, which is intended to serve as a tool for the creation of an application specific User Acceptance Test Plan. This template can be downloaded and edited in MSWord. 4.1 Test Instructions How to Customize the Instruction Set: Include instructions for components assessed as necessary. Delete the unnecessary instructions from the template. Modify the appropriate application specific information and resources. 4.2 Form-Based Test Scripts How to Customize the Form-Based Test Module List and Scripts: List all modules to be included in the User Acceptance Test Add any application specific steps to the test script Remove any steps from the sample test script that are not relevant to your application Examples: Module ID Module name CRS0000 Main Menu CRS1020 Enter a License Scenario CRS1080 Enter an Authorization Scenario CRS1050 Enter an Exam Mark Scenario CRS1030 View all Licenses Scenario CRS1020 Update a License Scenario Figure 1: List of Modules Example Form Based Testing Component Date Initials 1. Are all fonts, colors, shading and toolbars consistent with standards and project guidelines? 2. Is the online help module available? 3. Are all date formats correct (DD-MON-YYYY) Are the date edits being correctly applied? Are dates greater than 2000 accepted? 4. Does all text wrap when displayed in the text editor? 5. Is there a scroll bar on every applicable block? Figure 2: Form-Based Test Script Example

4.3 User Security Matrix How to Customize the User Security Matrix: Fill in the blanks for the table for each appropriate User Security Level to be tested. If your application does not have multiple security levels or multiple modules, this matrix will not be necessary for your User Acceptance Test Plan. User Role : List the Security Level to be tested Module Label: List the technical id for the Module (eg. CRS1010) Module description: describe the Module name and purpose (eg.Update License information) Module Type: Select from types: UPDATE, VIEW, CREATE NEW or REPORT Examples: User Security/Access Level Matrix USER ROLE Module Label Module Details Module Type Description 1 SYS_ADMIN CRS0000 Main Menu Access to all buttons Menu 2 SYS_ADMIN CRS1020 Enter License Access to all functionality Update 3 CLERK_1 CRS0000 Main Menu Access to View Buttons Menu only other buttons "grayed" out Figure 3: User Security Matrix Example 4.4 Business Process Test Scripts How to Customize the Business Process Test Scripts: Delete the sample business process test scripts List all business functions per user Security level as test steps Examples: Acceptance Testing Action Date Initials 1. System Administrator logs on 2. Is the correct module name/date/version # displayed at the top of the window? 3. Are the appropriate buttons available (system admin has access to all)? 4. Are the appropriate menu options available (system admin has access to all) Figure 4: Business Process Test Script Example (Main Menu) 4.5 Report Test Scripts How to Customize the Report Test Scripts: Delete the sample business process test scripts List all report functions as test steps Examples: Acceptance Testing Action Date Initials 1. Can you access the Report module from the main menu? 2. Are all appropriate reports listed based on the Security of the user being tested? 3. Can you access the Detailed Information Report from the list of available reports? 4. Is the correct report name/date/version # displayed at the top of the window? 5. Are the appropriate buttons available (based on the security matrix)? Figure 5: Report Test Script Example 4.6 Defect Tracking Form How to Customize the Defect Tracking Form: Modify the User Security levels listed in the form to match your application Add any key information you wish to have reported with defects 4.7 Defect Tracking Log How to Customize the Defect Tracking Log: Add new columns for application specific information added to the Defect Tracking Form Remove any columns containing information not required in your summary information Examples: Defect Log Form # Tester Description Date User Severity Repeated Date Status # # Reported Level (1,2,3,4,5) (E,S,O,1x) Reported

Tested 1 CRS001 DB

View 99/10/5 CLRK1 1 E License button not available for Clerk 1 2 CRS020 LM Error on 99/10/08 All 1 S 99/10/06 Open printing 3 CRS030 DB Field width 99/10/09 All 2 E 99/10/10 Fixed cuts off description 4 CRS030 KA Sort order 99/10/10 All 2 1x 99/10/10 Open incorrect Figure 6: Example Defect Tracking Log 5. Conclusion The test scripts created for a new system, for regression testing or for limited testing should be recycled throughout the applications life. By editing existing test scripts the User Acceptance Test Planning process can save time and money, as well as maintain the test quality, as key requirement information will be re-used.

to Contractor 99/10/06 Fixed