Software Quality Assurance Software QA involves the entire software development PROCESS monitoring and improving the process,

making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'. Software Testing Testing involves operation of a system or application under controlled conditions and evaluating the results (eg, 'if the user is in interface A of the application while using hardware B, and does C, then D should happen'). The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should. It is oriented to 'detection'. Organizations vary considerably in how they assign responsibility for QA and testing. Sometimes they're the combined responsibility of one group or individual. Also common are project teams that include a mix of testers and developers who work closely together, with overall QA processes monitored by project managers. It will depend on what best fits an organization's size and business structure. Software Quality Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end -users, customer acceptance testers, customer contract officers, customer management, the development organization's management/accountants/testers/salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as userfriendly and bug-free. Some recent major computer system failures caused by software bugs A May 2005 newspaper article reported that a major hybrid car manufacturer had to install a software fix on 20,000 vehicles due to problems with invalid engine warning lights and occasional stalling. In the article, an automotive software specialist indicated that the automobile industry spends $2 billion to $3 billion per year fixing software problems. Media reports in January of 2005 detailed severe problems with a $170 million high-profile U.S. government IT systems project. Software testing was one of the five major problem areas according to a report of the commission reviewing the project. In March of 2005 it was decided to scrap the entire project.

In July 2004 newspapers reported that a new government we lfare management system in Canada costing several hundred million dollars was unable to handle a simple benefits rate increase after being put into live operation. Reportedly the original contract allowed for only 6 weeks of acceptance testing and the system was never tested for its ability to handle a rate increase. Millions of bank accounts were impacted by errors due to installation of inadequately tested software code in the transaction processing system of a major North American bank, according to mid-2004 news reports. Articles about the incident stated that it took two weeks to fix all the resulting errors, that additional problems resulted when the incident drew a large number of e-mail phishing attacks against the bank's customers, and that the total cost of the incident could exceed $100 million. Every software project need testers? While all projects will benefit from testing, some projects may not require independent test staff to succeed. Which projects may not need independent test staff? The answer depends on the size and context of the project, the risks, the development methodology, the skill and experience of the developers, and other factors. For instance, if the project is a short-term, small, low risk project, with highly experienced programmers utilizing thorough unit testing or test-first development, then test engineers may not be required for the project to succeed. In some cases an IT organization may be too small or new to have a testing staff even if the situation calls for it. In these circumstances it may be appropriate to instead use contractors or outsourcing, or adjust the project management and development approach (by switching to more senior developers and agile testfirst development, for example). Inexperienced managers sometimes gamble on the success of a project by skipping thorough testing or having programmers do post-development functional testing of their own work, a decidedly high risk gamble. For non-trivial-size projects or projects with non-trivial risks, a testing staff is usually necessary. As in any business, the use of personnel with specialized skills enhances an organization's ability to be successful in large, complex, or difficult tasks. It allows for both a) deeper and stronger skills and b) the contribution of differing perspectives. For example, programmers typically have the perspective of 'what are the technical issues in making this functionality work?'. A test engineer typically has the perspective of 'what might go wrong with this functionality, and how can we ensure it meets expectations?'. Technical people who can be highly effective in approaching tasks from both of those perspectives are rare, which is why, sooner or later, organizations bring in test specialists.

What is Operating System? The most important program that runs on a computer. Every general-purpose computer must have an operating system to run other programs. Operating systems perform basic tasks, such as recognizing input from the keyboard, sending output to the display screen, keeping track of files and directories on the disk, and controlling peripheral devices such as disk drives and printers. Commonly used Operating systems are Windows XP Linux Solaris What is Software? A program or set of instructions that controls the operation of a computer. Distinguished from the actual hardware of the computer Internet Explorer Microsoft word Notepad What is Hardware? The physical equipment of computing and computer-directed activities. The physical components of a computer system. Mouse Modem Hard drive CD Rom What is SDLC? SDLC: System Development Life Cycle A methodology used to develop, maintain, and replace information systems. Typical phases in the SDLC are: Analysis Design Development Integration and Testing Implementation, etc

What is an application? A software program designed to perform a specific task or group of tasks, such as word processing, communications, or database management 2 Tier Application 3 Tier Application N Tier Application What is a Scripting Language? A programming language in which programs are the series of commands that are interpreted and then executed one by one. Doesn't require the compilation phase, for the price of lower performance VB script Java Script Perl Etc.. What is Data? Information stored on the computer system, used by applications to accomplish tasks. Types of Data: Text Numbers Alpha Numeric Pictures Music file Movie file Animation What is a Database? A database is a collection of information/data stored in a computer in a systematic way, such that a computer program can consult it to answer questions. The software used to manage and query a database is known as a database management system (DBMS). The properties of database systems are studied in information science. Type of data storage methods/formats File System in different formats DBMS RDBMS (Relational Database Management system) Etc ..

What is a Client? A client is a system that accesses a (remote) service on another computer by some kind of network. The term was first applied to devices that were not capable of running their own stand-alone programs, but could interact with remote computers via a network. These dumb terminals were clients of the time-sharing mainframe computer. Some examples are Web Client Java Client Etc.. What is a Browser? Software program used to view and interact with various types of Internet resources available on the World Wide Web. Netscape and Internet Explorer are two common examples. What is a Server? A computer that delivers information and software to other computers linked by a network. What is a Web Server? A computer that is connected to the Internet and stores files written in HTML (hypertext markup language) that is publicly available through an Internet connection. Apache Web logic Web Sphere IIS Etc.. What is application Server? An application server is a software platform that delivers content to the Web. This means that an application server interprets site traffic and constructs pages based on a dynamic content repository. This content is typically personalized based on site visitor information, such as the content he/she has viewed up to that point, his/her past buying history, or preferences he/she has set during previous visits.

Commonly used servers are Tomcat Web Logic Web Sphere Load Balancing Distributing processing and communications activity evenly across a computer network so that no single device is overwhelmed. Load balancing is especially important for networks where it's difficult to predict the number of requests that will be issued to a server. Busy Web sites typically employ two or more Web servers in a load balancing scheme. If one server starts to get swamped, requests are forwarded to another server with more capacity. Load balancing can also refer to the communications channels themselves. What is Proxy Server? A server that receives requests intended for another server and that acts on the behalf of the client behalf (as the client proxy) to obtain the requested service. A proxy server is often used when the client and the server are incompatible for direct connection. For example, the client is unable to meet the security authentication requirements of the server but should be permitted some services. What is a Protocol? On the Internet "protocol" usually refers to a set of rules that define an exact format for communication between systems. HTTP protocol defines the format for communication between web browsers and web servers. IMAP protocol defines the format for communication between IMAP email servers and clients SSL protocol defines a format for encrypted communications over the Internet. What are Cookies? Are small files that can be created and written to by a programming/scripting language. Client-side cookies are JavaScript cookies that are read/written to a user's hard drive by a JavaScript program that runs in the web browser when a user visits a web site. Server-side cookies may be created by languages such as PHP. ...

What is XML? Extensible Markup Language. A flexible way to create common information formats and share both the format and the data on the World Wide Web, intranets, and elsewhere. XML is a formal recommendation from the World Wide Web Consortium (W3C) similar to the language of today's Web pages, the Hypertext Markup Language (HTML). What is Networking? A system of connecting computer systems or peripheral devices, each one remote from the other. What is Bandwidth? Bandwidth is the amount of data that can be transferred over the network in a fixed amount of time. On the Net, it is usually expressed in bits per second (bps) or in higher units like Mbps (millions of bits per second). 28.8 modem can deliver 28,800 bps, a T1 line is about 1.5 Mbps What is Firewall? A firewall is either the program or the computer it runs on, usually an Internet gateway server, that protects the resources of one network from users from other networks. Typically, an enterprise with an intranet that allows its workers access to the wider Internet will wa nt a firewall to prevent outsiders from accessing its own private data resource What is middleware? Software that connects two otherwise separate applications. Middleware is sometimes called "plumbing" because it connects two sides of an application and passes data between them. Oracle's SQL*Net connect clients to the database server. Oracle's Gateways connect different types of databases (for example Oracle to SQL Server or DB2) JMS Etc.. What is Environment? A collection of hardware, software, network communications and procedures that work together to provide a discrete type of computer service. There may be one or more environments on a physical platform, eg test, production. An environment has unique features and characteristics that dictate

how the y are administered in similar, yet diverse manners. Development Testing or QC Staging or Pre-Production Production Etc.. What is Network Drive? A connection to the handle drive of a remote computer, allowing you to access shared files and directories. You can establish a network drive connection to a directory in the file space. What is Version Control? Each time content is changed and checked back into a content management system, a copy of the content is saved and its identifier (version number) is incremented to indicate its difference from the previous copy. Commonly used applications are: Visual Source Safe Perforce Etc.. What is Net Work Printer? A printer available for use by workstations on a network. A network printer either has its own built-in network interface card, or it's connected to a printer on the network. This is commonly shared between a group, people around the printer. Some printers are privileged such as color, laser and some other specialty printers depending upon the organization types.

What is an IP Address? This is a unique string of numbers that identifies a computer or server on the Internet. These numbers are normally shown in groups separated by periods. Example: 216.239.51. Hosting accounts for websites can have either a shared or unique IP address What is host name? In the Internet suite of protocols, the name that is given to a machine. Sometimes, host name is used to mean fully qualified domain name (FQDN).

Other times, it is used to mean the most specific sub name of a fully qualified domain name. For example, if rchland.vnet.ibm.com is the fully qualified domain name, either of the following can be considered the host name: (a) rchland.vnet.ibm.com, or (b) rchland What is Configuration? This is a general-purpose computer term that can refer to the way you have your computer set up. It is also used to describe the total combination of hardware components that make up a computer system and the software settings that allow various hardware components of a computer system to communicate with one another.

Why does software have bugs? Miscommunication or no communication - as to specifics of what an application should or shouldn't do (the application's requirements). Software complexity - the complexity of current software applications can be difficult to comprehend for anyone without experience in modern-day software development. Multi-tiered applications, client-server and distributed applications, Data communications, enormous relational databases, and sheer size of applications have all contributed to the exponential growth in software/system complexity. Programming errors - programmers, like anyone else, can make mistakes. Changing requirements (whether documented or undocumented) - the end-user may not understand the effects of changes, or may understand and request them anyway - redesign, rescheduling of engineers, effects on other projects, work already completed that may have to be redone or thrown out, hardware requirements that may be affected, etc. If there are many minor changes or any major changes, known and unknown dependencies among parts of the project are likely to interact and cause problems, and the complexity of coordinating changes may result in errors. Enthusiasm of engineering staff may be affected. In some fast-changing business environments, continuously modified requirements may be a fact of life. In this case, management must understand the resulting risks, and QA and test engineers must adapt and plan for continuous extensive testing to keep the inevitable bugs from running out of control

Time pressures - scheduling of software projects is difficult at best, often requiring a lot of guesswork. When deadlines loom and the crunch comes, mistakes will be made. Egos - people prefer to say things like: 'no problem' 'piece of cake' 'I can whip that out in a few hours' 'it should be easy to update that old code' Poorly documented code - it's tough to maintain and modify code that is badly written or poorly documented; the result is bugs. In many organizations management provides no incentive for programmers to document their code or write clear, understandable, maintainable code. In fact, it's usually the opposite: they get points mostly for quickly turning out code, and there's job security if nobody else can understand it ('if it was hard to write, it should be hard to read'). Software development tools - visual tools, class libraries, compilers, scripting tools, etc. often introduce their own bugs or are poorly documented, resulting in added bugs. What kinds of testing should be considered? Black box testing - not based on any knowledge of internal design or code. Tests are based on requirements and functionality. White box testing - based on knowledge of the internal logic of an application's code. Tests are based on coverage of code statements, branches, paths, conditions. unit testing - the most 'micro' scale of testing; to test particular functions or code modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code; may require developing test driver modules or test harnesses. incremental integration testing - continuous testing of an application as new functionality is added; requires that various aspects of an application's functionality be independent enough to work separately before all parts of the program are completed, or that test d rivers be developed as needed; done by programmers or by testers. integration testing - testing of combined parts of an application to determine if they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems. functional testing - black-box type testing geared to functional requirements of an application; this type of testing should be done b y testers. This doesn't mean that the programmers shouldn't check that their code works before releasing it (which of course applies to any stage of testing.)

system testing - black-box type testing that is based on overall requirements specifications; covers all combined parts of a system. end-to-end testing - similar to system testing; the 'macro' end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate. sanity testing or smoke testing - typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing systems every 5 minutes, bogging down systems to a crawl, or corrupting databases, the software may not be in a 'sane' enough condition to warrant further testing in its current state. regression testing - re-testing after fixes or modifications of the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing. acceptance testing - final testing based on specifications of the end -user or customer, or based on use by end-users/customers over some limited period of time. load testing - testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system's response time degrades or fails. stress testing - term often used interchangeably with 'load' and 'performance' testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc. performance testing - term often used interchangeably with 'stress' and 'load' testing. Ideally 'performance' testing (and any other 'type' of testing) is defined in requirements documentation or QA or Test Plans. usability testing - testing for 'user-friendliness'. Clearly this is subjective, and will depend on the targeted e nd-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers. install/uninstall testing - testing of full, partial, or upgrade i nstall/uninstall processes. recovery testing - testing how well a system recovers from crashes, hardware failures, or other catastrophic problems. failover testing - typically used interchangeably with 'recovery testing' security testing - testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques. compatability testing - testing how well software performs in a particular hardware/software/operating system/network/etc. environment. exploratory testing - often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the software as they test it.

ad-hoc testing - similar to exploratory testing , but often taken to mean that the testers have significant understanding of the software before testing it. context-driven testing - testing driven by an understanding of the environment, culture, and intended use of software. For example, the testing approach for life critical medical equipment software would be completely different than that for a low-cost computer game. user acceptance testing - determining if software is satisfactory to an end-user or customer. comparison testing - comparing software weaknesses and strengths to competing products. alpha testing - testing of an application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers. beta testing - testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers. mutation testing - a method for determining if a set of test data or test cases is useful, by deliberately introducing various code changes ('bugs') and retesting with the original test data/cases to determine if the 'bugs' are detected. Proper implementation requires large computational resources. What is SEI? CMM? CMMI? ISO? IEEE? ANSI? SEI = 'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes. CMM = 'Capability Maturity Model', now called the CMMI ('Capability Maturity Model Integration'), developed by the SEI. It's a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors. Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable. Level 2 - software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated. Level 3 - standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to

ensure understanding and compliance. Level 4 - metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high. Level 5 - the focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median size of organizations was 100 software engineering/maintenance personnel; 32% of organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most problematical key process area was in Software Quality Assurance. ISO = 'International Organisation for Standardization' - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2000 Quality Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed. IEEE = 'Institute of Electrical and Electronics Engineers' - among other things, creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730), and others. ANSI = 'American National Standards Institute', the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality). Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.

What steps are needed to develop and run software tests? The following are some of the steps to consider: Obtain requirements, functional design, and internal design specifications and other necessary documents Obtain budget and schedule requirements Determine project-related personnel and their responsibilities, reporting requirements, required standards and p rocesses (such as release processes, change processes, etc.) Determine project context, relative to the existing quality culture of the organization and business, and how it might impact testing scope, aproaches, and methods. Identify application's highe r-risk aspects, set priorities, and determine scope and limitations of tests Determine test approaches and methods - unit, integration, functional, system, load, usability tests, etc. Determine test environment requirements (hardware, software, communications, etc.) Determine testware requirements (record/playback tools, coverage analyzers, test tracking, problem/bug tracking, etc.) Determine test input data requirements Identify tasks, those responsible for tasks, and labor requirements Set schedule estimates, timelines, milestones Determine input equivalence classes, boundary value analyses, error classes Prepare test plan document and have needed reviews/approvals Write test cases Have needed reviews/inspections/approvals of test cases Prepare test environment and testware, obtain needed user manuals/reference documents/configuration guides/installation guides, set up test tracking processes, set up logging and archiving processes, set up or obtain test input data Obtain and install software releases Perform tests Evaluate and report results Track problems/bugs and fixes Retest as needed Maintain and update test plans, test cases, test environment, and testware through life cycle Test plan A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It

should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project: Title Identification of software including version/release numbers Revision history of document including authors, dates, approvals Table of Contents Purpose of document, intended audience Objective of testing effort Software product overview Relevant related document list, such as requirements, design documents, other test plans, etc. Relevant standards or legal requirements Traceability requirements Relevant naming conventions and identifier conventions Overall software project organization and personnel/contact-info/responsibilties Test organization and personnel/contact-info/responsibilities Assumptions and dependencies Project risk analysis Testing priorities and focus Scope and limitations of testing Test outline - a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable Outline of data input equivalence classes, boundary value analysis, error classes Test environment - hardware, operating systems, other required software, data configurations, interfaces to other systems Test environment validity analysis - differences between the test and production systems and their impact on test validity. Test environment setup and configuration issues Software migration processes Software CM processes Test data setup requirements Database setup requirements Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs Test automation - justification and overview Test tools to be used, including versions, patches, etc. Test script/test code maintenance processes and version control Problem tracking and resolution - tools and processes Project test metrics to be used Reporting requirements and testing deliverables Software entrance and exit criteria Initial sanity testing period and criteria Test suspension and restart criteria

Personnel allocation Personnel pre-training needs Test site/location Outside test organizations to be utilized and their purpose, responsibilties, deliverables, contact persons, and coordination issues Relevant proprietary, classified, security, and licensing issues. Test case A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, and expected results. Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle if possible. 3 Tier Application. Introduction Why 3-tier What is 3-tier-architecture Advantages Introduction Through the appearance of Local-Area-Networks, PCs came out of their isolation, and were soon not only being connected mutually but also to servers. Client/Server-computing was born. Servers today are mainly file and database servers; application servers are the exception. However, database-servers only offer data on the server; consequently the application intelligence must be implemented on the PC (client). Since there are only the architecturally tiered data server and client, this is called 2 -tier architecture. This model is still predominant today, and is actually the opposite of its popular terminal based predecessor that had its entire intelligence on the host system. One reason why the 2-tier model is so widespread, is because of the quality of the tools and middleware that have been most commonly used since the 90s: Remote-SQL, ODBC, relatively inexpensive and well integrated PC-tools (like Visual Basic, Power-Builder, MS Access, 4-GL-Tools by the DBMS manufactures). In comparison the server side uses relatively expensive tools. In addition the PC-based tools show good Rapid-Application-Development (RAD) qualities i.e. that simpler applications can be produced in a comparatively short

time. The 2-tier model is the logical consequence of the RAD-tools popularity : for many managers it was and is simpler to attempt to achieve efficiency in software development using tools, than to choose the steep and stony path of "brainware". Why 3-tier? Unfortunately the 2 -tier model shows striking weaknesses, that make the development and maintenance of such applications much more expensive. The complete development accumulates on the PC. The PC processes and presents information which leads to monolithic applications that are expensive to maintain. Thats why its called a "fat client". In a 2-tier architecture, business-logic is implemented on the PC. Even the businesslogic never makes direct use of the windowing-system, programmers have to be trained for the complex API under Windows. Windows 3.X and Mac-systems have tough resource restrictions. For this reason applications programmers also have to be well trained in systems technology, so that they can optimize scarce resources. Increased network load: since the actual processing of the data takes place on the remote client, the data has to be transported over the network. As a rule this leads to increased network stress. How to conduct transactions is controlled by the client. Advanced techniques like two-phase-committing cant be run. PCs are considered to be "untrusted" in terms of security, i.e. they are relatively easy to crack. Nevertheless, sensitive data is transferred to the PC, for lack of an alternative. Data is only "offered" on the server, not processed. Stored-procedures are a form of assistance given by the database provider. But they have a limited application field and a proprietary nature. Application logic cant be reused because it is bound to an individual PCprogram. The influences on change-management are drastic: due to changes in business politics or law (e.g. changes in VAT computation) processes have to be changed. Thus possibly dozens of PC-programs have to be adapted because the same logic has been implemented numerous times. It is then obvious that in turn each of these programs have to undergo quality control, because all programs are expected to generate the same results again. The 2-tier-model implies a complicated software-distribution-procedure: as all of the application logic is executed on the PC, all those machines (maybe thousands) have to be updated in case of a new release. This can be very expensive, complicated, prone to error and time consuming. Distribution procedures include the distribution over networks (perhaps of large files) or the production of an adequate media like floppies or CDs. Once it arrives at the users desk, the software first has to be installed and tested for correct execution. Due to the distributed character of such an update procedure, system

management cannot guarantee that all clients work on the correct copy of the program. 3- and n-tier architectures endeavour to solve these problems. This goal is achieved primarily by moving the application logic from the client back to the server. What is 3 - and n-tier architecture? From here on we will only refer to 3-tier architecture, that is to say, at least 3-tier architecture. The following diagram shows a simplified form of reference-architecture, though in principal, all possibilities are illustrated.

Client-tier Is responsible for the presentation of data, receiving user events and controlling the user interface. The actual business logic (e.g. calculating added value tax) has been moved to an application-server. Today, Java -applets offer an alternative to traditionally written PC-applications. See our Internet-page for further information. Application-server-tier This tier is new, i.e. it isnt present in 2-tier architecture in this explicit form. Business-objects that implement the business rules "live" here, and are available to the client-tier. This level now forms the central key to solving 2 -tier problems. This tier protects the data from direct access by the clients. The object oriented analysis "OOA", on which many books have been written, aims in this tier: to record and abstract business processes in business-objects. This way it is possible to map out the applications-server-tier directly from the CASE-tools that support OOA. Furthermore, the term "component" is also to be found here. Today the term predominantly describes visual components on the client-side. In the non-visual area of the system, components on the server-side can be defined as configurable objects, which can be put together to form new application processes. Data-server-tier This tier is responsible for data storage. Besides the widespread relational database systems, existing legacy systems databases are often reused here. It is important to note that boundaries between tiers are logical. It is quite easily possible to run all three tiers on one and the same (physical) machine. The main importance is that the system is neatly structured, and that there is a well planned definition of the software boundaries between the different tiers.

The advantages of 3-tier architecture As previously mentioned 3-tier architecture solves a number of problems that are inherent to 2-tier architectures. Naturally it also causes new problems, but these are outweighed by the advantages. Clear separation of user-interface-control and data presentation from applicationlogic. Through this separation more clients are able to have access to a wide variety of server applications. The two main advantages for client-applications are clear: quicker development through the reuse of pre-built business-logic components and a shorter test phase, because the server-components have already been tested. Re-definition of the storage strategy wont influence the clients. RDBMS offer a certain independence from storage details for the clients. However, cases like changing table attributes make it necessary to adapt the clients application. In the future, even radical changes, like lets say switching form an RDBMS to an OODBS, wont influence the client. In well designed systems, the client still accesses data over a stable and well designed interface which encapsulates all the storage details. Business-objects and data storage should be brought as close together as possible, ideally they should be together physically on the same server. This way - especially with complex accesses - network load is eliminated. The client only receives the results of a calculation - through the business-object, of course. In contrast to the 2-tier model, where only data is accessible to the public, business-objects can place applications -logic or "services" on the net. As an example, an inventory number has a "test-digit", and the calculation of that digit can be made available on the server. As a rule servers are "trusted" systems. Their authorization is simpler than that of thousands of "untrusted" client-PCs. Data protection and security is simpler to obtain. Therefore it makes sense to run critical business processes, that work with security sensitive data, on the server. Dynamic load balancing: if bottlenecks in terms of performance occur, the server process can be moved to other servers at runtime. Change management: of course its easy - and faster - to exchange a component on the server than to furnish numerous PCs with new program versions. To come back to our VAT example: it is quite easy to run the new version of a tax-object in such a way that the clients automatically work with the version from the exact date that it has to be run. It is, however, compulsory that interfaces remain stable and that old client versions are still compatible. In addition such components require a high standard of quality control. This is because low quality components can, at worst, endanger the functions of a whole set of client applications. At best, they will still irritate the systems operator. As shown on the diagram, it is relatively simple to use wrapping techniques in 3tier architecture. As implementation changes are transparent from the viewpoint of the object's client, a forward strategy can be developed to replace legacy system smoothly. First, define the object's interface. However, the functionality is

not newly implemented but reused from an existing host application. That is, a request from a client is forwarded to a legacy syste m and processed and answered there. In a later phase, the old application can be replaced by a modern solution. If it is possible to leave the business objects interfaces unchanged, the client application remains unaffected. A requirement for wrapping is, however, that a procedure interface in the old application remains existent. It isnt possible for a business object to emulate a terminal. It is also important for the project planner to be aware that the implementation of wrapping objects can be very complex. What should be done after a bug is found? The bug needs to be communicated and assigned to developers that can fix it. After the problem is resolved, fixes should be re-tested, and determinations made regarding requirements for regression testing to check that fixes didn't create problems elsewhere. If a problem-tracking system is in place, it should encapsulate these processes. A variety of commercial problemtracking/management software tools are available The following are items to consider in the tracking process: Complete information such that developers can understand the bug, get an idea of it's severity, and reproduce it if necessary. Bug identifier (number, ID, etc.) Current bug status (e.g., 'Released for Retest', 'New', etc.) The application name or identifier and version The function, module, feature, object, screen, etc. where the bug occurred Environment specifics, system, platform, relevant hardware specifics Test case name/number/identifier One-line bug description Full b ug description Description of steps needed to reproduce the bug if not covered by a test case or if the developer doesn't have easy access to the test case/test script/test tool Names and/or descriptions of file/data/messages/etc. used in test File excerpts/error messages/log file excerpts/screen shots/test tool logs that would be helpful in finding the cause of the problem Severity estimate (a 5-level range such as 1-5 or 'critical'-to-'low' is common) Was the bug reproducible? Tester name Test date Bug reporting date Name of developer/group/organization the problem is assigned to Description of problem cause Description of fix Code section/file/module/class/method that was fixed Date of fix Application version that contains the fix

Tester responsible for retest Retest date Retest results Regression testing requirements Tester responsible for regression tests Regression testing results A reporting or tracking process should enable notification of appropriate personnel at various stages. For instance, testers need to know when retesting is needed, developers need to know when bugs are found and how to get the needed information, and reporting/summary capabilities are needed for managers. What is 'configuration management'? Configuration management covers the processes used to control, coordinate, and track: code, requirements, documentation, problems, change requests, designs, tools/compilers/libraries/patches, changes made to them, and who makes the changes What if the software is so buggy it can't really be tested at all? The best bet in this situation is for the testers to go through the process of reporting whatever bugs or blocking-type problems initially show up, with the focus being on critical bugs. Since this type of problem can severely affect schedules, and indicates deeper problems in the software development process (such as insufficient unit testing or insufficient integration testing, poor design, improper build or release procedures, etc.) managers should be notified, a nd provided with some documentation as evidence of the problem. How can it be known when to stop testing? This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, that complete testing can never be done. Common factors in deciding when to stop are: Deadlines (release deadlines, testing deadlines, etc.) Test cases completed with certain percentage passed Test budget depleted Coverage of code/functionality/requirements reaches a specified point Bug rate falls below a certain level Beta or alpha testing period ends What if there isn't enough time for thorough testing? Use risk analysis to determine where testing should be focused. Since it's rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go

wrong, risk analysis is appropriate to most software development projects. This requires judgement skills, common sense, and experience. (If warranted, formal methods are also available.) Considerations can include: Which functionality is most important to the project's intended purpose? Which functionality is most visible to the user? Which functionality has the largest safety impact? Which functionality has the largest financial impact on users? Which aspects of the application are most important to the customer? Which aspects of the application can be tested early in the development cycle? Which parts of the code are most complex, a nd thus most subject to errors? Which parts of the application were developed in rush or panic mode? Which aspects of similar/related previous projects caused problems? Which aspects of similar/related previous projects had large maintenance expenses? Which parts of the requirements and design are unclear or poorly thought out? What do the developers think are the highest-risk aspects of the application? What kinds of problems would cause the worst publicity? What kinds of problems would cause the most customer service complaints? What kinds of tests could easily cover multiple functionalities? Which tests will have the best high-risk-coverage to time-required ratio? What if the project isn't big enough to justify extensive testing? Consider the impact of project errors, not the size of the project. However, if extensive testing is still not justified, risk analysis is again needed and the same considerations as described previously in 'What if there isn't enough time for thorough testing?' apply. The tester might then do ad hoc testing, or write up a limited test plan based on the risk analysis. How does a client/server environment affect testing? Client/server applications can be quite complex due to the multiple dependencies among clients, data communications, hardware, and servers, especially in multitier systems. Thus testing requirements can be extensive. When time is limited (as it usually is) the focus should be on integration and system testing. Additionally, load/stress/performance testing may be useful in determining client/server application limitations and capabilities. There are commercial tools to assist with such testing. How can World Wide Web sites be tested? Web sites are essentially client/server applications - with web servers and 'browser' clients. Consideration should be given to the interactions between html pages, TCP/IP communications, Internet connections, firewalls, applications that run in web pages (such as applets, javascript, plug-in applications), and applications that run on the server side (such as cgi scripts, database interfaces, logging applications, dynamic page generators, asp, etc.). Additionally, there are a wide variety of servers and browsers, various versions of each, small but

sometimes significant differences between them, variations in connection speeds, rapidly changing technologies, and multiple standards and protocols. The end result is that testing for web sites can become a major ongoing effort. Other considerations might include: What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times). What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)? Who is the target audience? What kind of browsers will they be using? What kind of connection speeds will they by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)? What kind of performance is expected on the client side (e.g., how fast should pages appear, how fast should animations, applets, etc. load and run)? Will down time for server and content maintenance/upgrades be allowed? how much? What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested? How reliable are the site's Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing? What processes will be required to manage updates to the web site's content, and what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.? Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers? Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site?? How will internal and external links be validated and updated? how often? Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variabilities, and real-world internet 'traffic congestion' problems to be accounted for in testing? How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing? How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested? Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger, provide internal links within the page. The page layouts and design elements should be consistent throughout a site, so that it's clear to the user that they're still within a site. Pages should be as browser-independent as possible, or pages should be provided or generated based on the browser-type. All pages should have links external to the page; there should be no dead-end pages.

The page owner, revision date, and a link to a contact person or organization should be included on each page. What makes a good Software Test engineer? A good test engineer has a 'test to break' attitude, an ability to take the point of view of the customer, a strong desire for quality, and an attention to detail. Tact and diplomacy are useful in maintaining a cooperative relationship with developers, and an ability to communicate with both technical (developers) and non-technical (customers, management) people is useful. Previous software development experience can be helpful as it provides a deeper understanding of the software development process, gives the tester an appreciation for the developers' point of view, and reduce the learning curve in automated test tool programming. Judgement skills are needed to assess high-risk areas of an application on which to focus testing efforts when time is limited. What makes a good Software QA engineer? The same qualities a good tester has are useful for a QA engineer. Additionally, they must be able to understand the entire software development process and how it can fit into the business approach and goals of the organization. Communication skills and the ability to understand various sides of issues are important. In organizations in the early stages of implementing QA processes, patience and diplomacy are especially needed. An ability to find problems as well as to see 'what's missing' is important for inspections and reviews. SOFTWARE LIFE CYCLE: The life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects. SDLC (SOFTWARE DEVELOPMENT LIFE CYCLE) SDLC has total seven phases: Initiate the Project (Initial Phase) Define the System (Analysis Phase) Design the system (after client approval) Build the System System Testing (Based on SRS Software Requirement Planning) Deploy the system (Production) Support the system (Maintenance)

TESTING LIFE CYCLE: 1. 2. 3. 4. 5. 6. Test Planning Test Development Test Execution Test Results Defects Generation Reporting

Methodologies: What and Why? Software engineering is the practice of using selected process techniques to improve the quality of a software development effort. This is based on the assumption, subject to endless debate and supported by patient experience, that a methodical approach to software development results in fewer defects and, therefore, ultimately provides shorter delivery times and better value. The documented collection of policies, processes and procedures used by a development team or organization to practice software engineering is called its software development methodology (SDM) or system development life cycle (SDLC).

Methodology as Risk Management The challenge in selecting and following a methodology is to do it wisely -- to provide sufficient process disciplines to deliver the quality required for business success, while avoiding steps that waste time, squander productivity, demoralize developers, and create useless administrivia. The best approach for applying a methodology is to consider it as a means to manage risk. You can identify risks by looking at past projects. If your organization has been plagued by problems resulting from poor requirements management, then a robust requirements management methodology would be well advised. Once this problem has been solved, through a repeatable process, the organization might then streamline its process, while ensuring that quality is maintained. Every step along the system development life cycle has its own risks and a number of available techniques to improve process discipline and resulting output quality. Moving through the development life cycle, you might encounter the following major steps:

Project charter and business case

Definition of the business process and business requirements Documentation of user, functional and system requirements Top level architecture, technical approach, and system design System decomposition into component and unit specifications and design Coding, unit test planning, and unit test Generation of test data for unit testing and system testing System integration and testing Implementation, delivery and cut-over Training and user support System upgrades and routine software maintenance

In addition, you might have support activities throughout the development effort such as:

Configuration management (version identification, baseline management and change control) Requirements management and tracability Quality management (quality assurance, quality reviews, defect tracking) System engineering reviews (requirements review, prelim. and critical design reviews, etc.) Support environment (development tools, libraries, files management, data management)

Written guidance for all these steps would constitute the core of your methodology. You can see how it wouldn't take long to fill a number o f big binders with development processes and procedures. Hence, the importance of selecting processes wisely - to address known risks - keeping the methodology streamlined, and allowing for some discretion on the part of the project team.
Waterfall Methodologies Summarized

Rather than try to give an all-encompassing definition for methodologies that should be classified as waterfall approaches, it easier to describe some common characteristics. Primarily, a waterfall methodology structures a project into distinct phases with defined deliverables from each phase. The phases are always named something different, depending on which company is trying to differentiate its own particular flavor, but the basic idea is that the first phase tries to capture What the system will do (its requirements), the second determines How it will be designed, in the middle is the actual programming, the fourth phase is the full system Testing, and the final phase is focused on Implementation tasks such as go-live, training, and documentation.

Define 15%

Design 15%

Code 35%

Test 30%

Waterfall Sequence
Waterfall Deliverables
Define Require ments. Design Screens Database Objects Test Plan UI Logic Reports Code Test Test Scripts Defect Report User Feedback Imp Training

Imp 5%

Documentat ion

Project Management Project Charter, Status Reports, Change Requests

Typically waterfall methodologies result in a project schedule with 20-40% of the time budgeted for the first two phases, 30-40% of the time to the programming, and the rest allocated to testing and implementation time. The actual project organization tends to be highly structured. Most medium to large size projects will include a rigidly detailed set of procedures and controls to cover everything from the types of communications to use in various situations, to authorizing and tracking change orders, to the specific ways that defects are logged, communicated, resolved, and re-tested. Perhaps most importantly, waterfall methodologies also call for an evolution of project staffing throughout the various phases. While typical cons ulting companies will refer to the differences in staffing as simply roles, which imply that the same people could remain on the project and simply switch roles, the reality is that the project staff constantly changes as it progresses. Reasons for the change include economics, mentoring, and expertise - economics in the sense that the project budget encourages the replacement of a relatively highly paid architect with a lower paid staff programmer as soon as possible. On the other hand, an architect with a particular skill set or an analyst with valuable subject area knowledge may be demanded on another project. A fundamental assumption is that the extensive project documentation and control procedures enable relatively easy knowledge transfer to new project staff.

Waterfall Resources
Define SMEs Analysts Proj Mgr.
Waterfall Strengths

Design SMEs Analysts Architects Proj . Mgr.

Code Architects Coders Proj. Mgr.

Test Testers Coders Proj. Mgr.

Imp Analysts Coders Proj. Mgr.

Most of the benefits from using a waterfall methodology are directly related to its underlying principles of structure. These strengths include:

Ease in analyzing potential changes Ability to coordinate larger teams, even if geographically distributed Can enable precise dollar budget Less total time required from Subject Matter Experts

Because the requirements and design documents contain an abstract of the complete system, the project manager can relatively quickly analyze what impact a change will have on the entire system. An example might be if one developer wanted to modify the fields in a database table or a class. The project manager could look up what other components of the system rely on that particular table or class and determine what side effects the change may have. The same documents that take so much time to assemble at the front-end also make dividing up and subsequently coordinating the work easier. Because the design produced in a waterfall approach is so detailed, it is easier to ensure that the pieces will be easier to integrate when the project nears the end of the programming phase. Even with a waterfall approach, the only way to ensure a precise up-front budget cost is to have an external vendor submit a fixed bid for the remaining phases of the project after one or two of the initial phases have been completed. In this way, financial risk is contained even if the project takes longer than expected. Perhaps one of the most compelling reasons for using the waterfall approach is simply the relatively small amount of time required of the subject matter experts. Because the SMEs in a project typically have other primary responsibilities, their time is limited. The waterfall approach ensures tha t a significant involvement from them is only required during the initial requirements phase as well as part of

the design, testing, and implementation phases. They have very little involvement during the entire programming phase.
Waterfall Weaknesses

While waterfall has advantages, its highly structured approach also leads to disadvantages such as the following:

Lack of flexibility Hard to predict all needs in advance Intangible knowledge lost between hand-offs Lack of team cohesion Design flaws not discovered until the Testing phase

Even though the impact of changes can be analyzed more effectively using a waterfall approach, the time required to analyze and implement each change can be significant. This is simply due to the structured nature of the project, and this is particularly acute when the changes are frequent or large. Even with improvements in providing wire frame screen mockups and more detailed flowcharts, the front-end planning process has a hard time effectively predicting the most effective system design on the front-end. One of the most challenging factors that causes this difficulty is the unfamiliarity most Subject Matter Experts are with formal system design techniques. One complaint that I have heard a lot is the document looks impressive, but I dont know if the system will meet my needs until I see the actual screens. Even more disturbing is the inevitable loss of knowledge between the planning and programming phases. Even with the most detailed documents, the analysts and architects always have an implicit understanding of the project needs that are very hard to transfer via paper documents. The information loss is particularly harmful to the project when it is developing a relatively new system as opposed to modifying an existing system. Closely linked to the knowledge loss effect, is the fact that the waterfall methodology discourages team cohesion. Many studies have found that truly effective teams begin a project with a common goal and stay together to the end. The tendency to switch out project staff from phase to phase weakens this overall team cohesion. In fact, it is common for the project manager be the only person that sees a project from beginning to end. The effect on team productivity is very hard to quantify, but may be illustrated with the following question: Would you have a passion for quality if you knew that someone else would be responsible for fixing your document or code in the next phase? The most significant weakness is the possibilities that a poor design choice will not be discovered until the final phases of testing or implementation. The risk of this occurring increase as project size and duration goes up. Even dedicated

and competent people make simple mistakes. In the context of the rigid wate rfall timetable, mistakes made in the master design may not be discovered until six or nine months of programming have been completed and the entire system is being tested.
Iterative Methodologies Summarized

The iterative family of methodologies shares a common emphasis on highly concentrated teams with minimal structure and access to constant feedback from the Subject Matter Experts. While it may appear that they lack design or testing, these types of methodologies actually place a great deal of emphasis on them. They just do it in a different way. Typically, a project will begin with the integrated project team being assembled and briefed on the project objectives. The team will consist of all of the essential roles from the very beginning and each member may actually play multiple roles. Rather than a distinct progression through phases, the iterative methodology emphasizes creating a series of working prototypes for evaluation by the SMEs until the objectives are accomplished and the system is ready for final release. During the process, it is critical for the actual project lead as well as the senior members of the team to balance the SME requests against the overall system constraints and platform limitations to ensure that quality and performance objectives can be met. It is the development teams responsibility to offer constructive feedback to the SMEs in order to suggest alternatives and work together to the best mutual solution. Where possible, individual team members will be given complete ownership of a particular component and charged with ensuring its usability, quality, and performance. The senior team members are responsible for enforcing quality and consistency standards. Even on large projects, the initial planning will consist of only a broad outline of the business objectives and creating a framework for the overall project components. In some cases, a set of core features for the entire system will be built initially and subsequent features added as the project progresses. In other cases, just certain modules will be built entirely during the early part of the project and other components added over time.

Iterative Sequence
Iteration 1 Iteration 2 20% 20% Iteration 3 30% Iteration 4 25% Release 5%

Iteration 1 Iteration 2 Iteration 3 25% 20% 25%

Iteration 4 Release 25% 5%

Iterative Resources
Iteration 1 SMEs Analysts Architect Coders Proj Mgr. Iteration 2 SMEs Analysts Architect Coders Proj Mgr. Iteration 3 SMEs Analysts Architect Coders Proj Mgr. Iteration 4 SMEs Analysts Architect Coders Proj Mgr. Release SMEs Analysts Architect Coders Proj Mgr.

Iterative Deliverables
Iteration 1 Iteration 2 Iteration 3 Tangible Software Iteration 4 Tangible Software Release Improved Software Training Project Management Document ation Tangible Tangible Software Software

Project Charter, Status Reports, Budget Control, Timeline Control

Iterative Strengths

Many of the strengths of the iterative system are listed below:

Rapid feedback from actual users Flexibility to address evolving requirements Design flaws discovered quickly Easy to roll-out new functionality in stages Higher motivation and great productivity Very little knowledge loss between phases

Feedback from the Subject Matter Experts or users can be based on an actual working prototype of the system relatively early in the project life cycle. This enables the SME to base his or her feedback on actually working with a limited version of final product. Much like it is easier to decide if a product meets your needs if you can examine it in the store than if someone were to just describe it to you, the SME is instantly able to identify potential problems with the

application as the developer is interpreting his requirements before too much time has passed. Since the development team receives feedback at early stages in the overall development process, changes in requirements can be more easily incorporated into the finished product. More importantly, if the SME determines that a feature would no t be as valuable, it can be omitted before too much development time has been spent or integrating the particular component into the overall system. In a similar way, since the team is deploying actual working prototype versions of the application along the way, a flaw in the design should become more apparent earlier in the project schedule. Instead of discovering a potential problem only after the system goes to full-scale testing, more design flaws can be addressed before they impact other features and require significant effort to correct. Because each iteration actually functions (sometimes to a limited degree), deploying parts of the system in a staged roll-out becomes much easier. Using an iterative methodology, the team simply stabilizes an earlier iteration of the component, collaborates with the SME to ensure it is stable and rolls it out. Another advantage of doing a staged roll-out in this way is that actual production use will generate more improvement suggestions to be incorporated in subsequent iterations of the same component and/or other components. The team approach stressed in the iterative methodology increases overall motivation and productivity. Because the same people are involved from beginning to end, they know that the design choices made will ultimately affect their ability to successfully complete the project. Productivity will be enhanced because of the sense of ownership the project team has in the eventual result. While it may seem like the empowerment fad, many studies ha ve found that a team charged with a common goal tends to be much more productive than groups of people with individual incentives and shifting assignments. One example of such a study is Groups that Work by Gerard Blair. The fact that an integrated team maintains a thorough understanding of the project is a more tangible benefit. This effect arises simply by having the same individuals involved from the very beginning and listening first hand to the Subject Matter Experts describe their needs and objectives. The subsequent feedback during each iteration of the project builds upon the initial understanding. Since the same person is listening to the needs and writing the code, less time needs to be spent authoring documents to describe those requirements for eventual hand-off. This translates into more time spent writing and testing the actual software.
Iterative Weaknesses

The drawbacks to using an iterative approach are worth considering and should be weighed carefully when deciding on a methodology for a new project. Some of the more serious weaknesses include:

Difficulty coordinating larger teams

Can result in a never-ending project if not managed properly Tendency to not document thoroughly Predicting the precise features to be accomplished in a fixed time/budget

Iterative projects tend to be most effective with small, highly skilled teams. It is much more difficult to ensure that the components mesh together smoothly across larger, geographically distributed projects. While steps can be taken to minimize the chances of failure, coordinating large iterative development efforts is typically very hard to accomplish effectively because of the lack of detailed planning documents. Because there are no specific cut-off milestones for new features, an iterative project runs the risk of continuing into perpetuity. Even though one of the strengths is the ability to react to changing business needs, the project leader must determine when the major business needs have been met. Otherwise, the project will continue to adapt to ever changing business needs and the software will never end. This will result in never really deploying a finished product to full production use. This is a risk even in a staged roll-out situation because there are always improvements possible to any software. In any software project, there is always the tendency to borrow time from the final system documentation tasks to resolve more defects or polish certain features more. This risk increases on iterative projects because there is usually no scheduled documentation period. The result is a system that is very hard to maintain or enhance. In a similar way, in an iterative project it is much easier to fix a definite project schedule or dollar budget than determine exactly what features will be able to be built within that timeline. This is simply due to the fact that the features change based on user feedback and the evolution of design.
Conclusion

I have clearly presented the tradeoffs between two basic approaches to software development in order to show that no methodology is universally superior. Instead, the approach that you should take on your next project should depend on its particular needs and the constraints that you have to work with. While certainly not an exhaustive reference on the subject of how a particular methodology is structured, my purpose was to help you become more familiar with the strengths and weaknesses inherent in each of the large schools of thought prevalent in the software development community. Based on the issues discussed, a few basic guidelines that may help point you in the right decision are listed below. Keep in mind that no methodology should ever be considered a

substitute for ensuring project members have the proper experience and skillset for the task to be accomplished.

The iterative methodology is usually better for new concepts Waterfall is usually better for modifications to existing systems or building large scale systems after proof-of concept prototypes have been established

However, some situations will require a hybrid approach

Spiral Methodology: While the waterfall methodology offers an orderly structure for software development, demands for reduced time-to-market make its series steps inappropriate. The next evolutionary step from the waterfall is where the various steps are staged for multiple deliveries or handoffs. The ultimate evolution from the water fall is the spiral, taking advantage of the fact that development projects work best when they are both incremental and iterative, where the team is able to start small and benefit from enlightened trial and error along the way. The spiral methodology reflects the relationship of tasks with rapid prototyping, increased parallelism, and concurrency in design and build activities. The spiral method should still be planned methodically, with tasks and deliverables identified for each step in the spiral.

Documentation: The reality is that increased processes usually result in increased documentation. An improved process produces intermediate work products that represent the elaboration of the product design at each step in the development life cycle. Where possible, documentation should be generated using automated tools, so outputs can contribute to generation of code structures or help generate the code itself. The difference between hacking and software engineering is professional discipline applied with common sense. Software quality, reliability, and maintainability are enhanced by having good documentation for requirements, architecture, interfaces, detailed design, well-commented code, and good test procedures. Requirements documentation practices should facilitate your customer's understa nding and review of the real requirements. Software project planning should include estimating the time and resources to produce, review, approve, and manage such documentation products.

Sample Software Documentation Work Products

Sample Test Plan

TABLE OF CONTENTS

1. Introduction 1.1 Document Purpose 1.2 Objectives 2. Project Scope 2.1 In Scope 2.2 Out Of Scope but Critical to Project Success 2.3 Out of Scope 3. Project Resources 4. Test Strategies/Techniques 4.1 Test Design 4.2 Test Data 5. Automation Coding Strategy 6. Test Suite Backup Strategy 7. Test Suite Version Control Strategy 8. Metrics Table 9. Project Tasks/Schedule 10. Tool Inventory 11. Hardware/Software Configuration 12. Naming Conventions 13. Defect Responsibility/Resolution 14. Exit Criteria 15. Goals and Deliverables 16. Glossary of Standard Terms

Introduction Document Purpose Document overview; high-level summary of major issues addressed. This Test Plan reviews: Existing project information. Business Requirements and critical transactions to be tested. Testing types and strategies to be implemented A proposed testing schedule Objectives State the objective of the testing project, its duration and justification. General comments concerning the objective of testing are appropriate (e.g. make the QA function more efficient; lower testing cycle time; improve software quality; enhance testing process. . Project Scope In Scope State scope in detail and duration of process. Out Of Scope but Critical to Project Success State any out-of-scope critical project dependency. E.g.: Database snapshots for test system that accurately reflect current user population. Out of Scope State in detail any out-of-scope activities. (Ex., Performance, stress, and volume testing (beyond the gathering of timing information from automated script executions) are out of project scope.) Project Resources Table 3.1. Project Roles and Responsibilities Role Responsibilities Resource Name(s) Testers Plan testing activities Execute Test Cases Automate Test Cases Find, report and track defects Measure test effort Analyze results Developers Deliver complete builds of the application Provide Testers with feedback regarding changes, new functionality Provide expertise and knowledge of the application-under-test Eliminate agreed upon defects Interview Users Business Create Business Requirements Analysts Create Test Scenarios, Test Cases Users Describe and review Business Requirements

Describe and review user profiles Perform User Acceptance Testing (UAT) DBA Provide access rights to database Assist with extraction of data for testing purposes Provide a stable testing environment Assist with returning database instance to a known preferred state Provide trouble-shooting and knowledge Network Provide network access privileges Administrator General troubleshooting and knowledge Desktop Installation of software Administrators Troubleshooting of hardware/software Information regarding standard desktop Management High-level problem solving Mediation of issues Interface of activities with different business units . Test Strategies/Techniques Test Design Describe test types that are relevant to the project. Provide justification for their relevance. Table 4.1. Summary of Test Types Test Type Definition Unit Test Test verifies the program (or module) logic and is based on the knowledge of the program structure. Programmers using the White Box technique perform this test type. Integration Test Test verifies the entire systems functionality (including feeds to and from the system) according to the business and design specifications. Business Requirements Verifies the specific requirements of the user are met. Also known as Business Rules Acceptance Testing Verifies that the system needs to meet the initial objectives and users exceptions. Used to prove that the system works. Known as positive testing. Regression Testing Verify that t he fixes/modifications are correct and no other parts of the system have been affected. System Test Testing the Volume testing - to determine whether the application architecture in a program can handle the required volume of data, production-simulated requests, etc. environment for normal and worst-case situations.

Graphical User Interface (GUI) Test

Load Testing Identify peak load conditions at which the program will fail to handle required processing loads within required time span. Performance testing - determine whether the program meets its performance requirements. Resource Usage testing - determine whether the program uses resources (memory, disk space, etc.) at levels which exceed expectations Interoperability Testing - Assure the application can co-exist on the desktop with other applications running. Also known as Compatibility Testing. Security testing - Show that the programs security requirements have been met Concurrency testing - Tests to verify that the system does not corrupt data when 2 or more users attempt to update the same record at the same time or when 2 or more users update different records at the same time and set a unique field to the same value Verify GUI features and elements and compare them to GUI standards and test design

How will test types proposed above be tested?

Table 4.2. Use Case listing with brief description and Test Case mapping. Use Case ID Description Test Case UC-1 Use Case 1 description TC-1a TC-1b Table 4.3. Test Case listing with mapping to generative Use Case, description and Requirement reference. Test Case Use Case ID Description Requirement ID TC-1a UC-1 Test Case 1a description R1.1-R5.3

TC-1b TC-1c

UC-1 UC-1

R6.1-R10.3 R10.3-R11

Test Data Description of data sets to be used in testing, origin of data sets, purpose for using each set (e.g. different user data for different user permissions), where data sets obtained, whose expertise guided data set selection, etc. Automation Coding Strategy This section describes the automation coding strategy that will be used for every test script: Generic examples follow. Automation of the test suite for the XX application will be performed using XX Softwares XX suite (automation tool: XX; scripting language: XX). The automation coding strategy that will be used in test suite building will include the following rules: Start and Stop Point: All Test Script navigation will start and finish on the XX window/page of the XX application. Browser Caption Verifications: Browser Captions will be verified on every window that is encountered in the application. The execution of these verifications will occur immediately after each window is loaded. Object Properties: Properties of objects that must be verified will be retrieved from application objects using the test tools data capture functionality. The retrieved data will then be compared against validated data in test suite files. Results will be output to the test log. Maintainability: Scripting will adhere to modular coding practices and will test following the strategy described above. Test suite builds will employ RTTS proprietary language extension (rttsutil.dll). Test Suite Backup Strategy List all paths to test artifacts here. How will test suite data (code, external files, tool data, etc.) be backed up for storage? How often? Where will backup location be? How accessible will backups be? How many backups will be kept at any given time? How long will backup data be kept? Will it be archived? Test Suite Version Control Strategy As test suites are modified for each build, how will test suite version control/change management be addressed? Will an external software tool be used for version control? Will there be a need to run archived test suites against old builds of the application? If so, how will this be facilitated? Metrics Table

A central part of test planning is the gathering of metrics. An accurate collection of metrics for all key project activities provides an assessment of the total effort required for the project. Table 8.1. Project Metrics Activity Interview with a knowledgeable User to characterize one user transaction Walkthrough of the valid test case Creation of a written test case by a business analyst/SME Automation and debugging of the script reflecting the test case Extraction of one requirement from requirements documentation Extraction of one requirement from user guide documentation Extraction of one requirement from release notes Project Tasks/Schedule Table 9.1. Project Schedule Task Resources Test Plan Completed Test Environment Prepared Requirements Processed by Tool Test Cases Planned Test Cases Created Test Cases Recorded and Executed Defects submitted and tracked Test Cycle Evaluation Test Suite Backup Strategy

Metric

Comments None Installation of Automated Tool Sectioned by user paths through the application One per requirement One per requirement Recorded in Tool, executed against each build and release of PROJECT. Submitted and tracked in Defect Tracking Tool Continuous effort Continuous effort

Projected Completion

Task Test Suite Version Control

Resources

Comments Continuous effort

Projected Completion

Tool Inventory Table 10.1. Software tools to be used in the Automated Testing of Project Function Tool Name Vendor Version Project Administration

Test Management

Capture/Playback

Defect/Issue Tracking Requirements Management

Team Communications (email, WebEx)

Utilities (RTTS Utilities)

. Hardware/Software Configuration Table 11.1. Hardware/Software System Resources Resource Details Test PC(s) Network OS Communication Protocol

Server Database Server - Web Applications Server Database Automation Software Other Software Front-End Development Tools Naming Conventions All Test Cases and Test Scripts created for this project will adhere to the following naming convention. Each Test Script will have the same name as its respective Test Case. We will use the following scheme, based upon : B C B I 2__ Numeric counter char. char. 2 char. 3 char. 4 char. 5 char. 6 1 through 9
(only used in the case of more than one) FIELD 3: FIELD 2: Type of Transaction: Area of application where I=Issue the transaction is performed: R=Release NC=Name Clearance Q=Quote

Underscore FIELD 1: The initial characters describe the stream by name: QBC=Quote, Binder, Cert stream

Field 1 represents the defined user stream through PROJECT by name. This section varies in length from one to three characters. Separation Character is an underscore. Field 2 represents the section of PROJECT being tested. Field 3 represents the type of transaction being tested. Additional Character (if needed) represents a numeric counter for multiple scripts of the same type and name. Defect Responsibility/Resolution Possible defects identified through automated or manual testing will be discussed with development team members and/or the Project Manager to verify that the observed behavior constitutes a defect. Identified defects will be logged in defect tracking software. Defects found manually will be coded into relevant automated test scripts for inclusion in future regression testing. Once the development team has corrected a defect, the defect will be retested using the same Test Script that detected the defect. Validated fixes will be entered into the defect-tracking tool. Accurate defect status data will be maintained in the defect-tracking tool by. In order to preserve data quality in the defect tracking process, will serve as

gatekeeper for the defect database. Responsibilities include: evaluation of all reported defects to verify the conditions under which they occur; reproducibility of reported defects; accuracy of defect descriptions; uniqueness of logged defects. Exit Criteria The following exit criteria will be used for each stage of the testing process. Testing can proceed to the next stage of the process when a sufficient proportion of the current stage has been completed (for example, test case preparation need not be completed before automated coding begins). The end of the project should satisfy all exit criteria. Stage 1: Test Process Assessment Delivery of a written assessment of the current test process with recommendations for improvement Stage 2: Test Planning Stage Test Plan delivery. Stage 3: Test Design Stage The application hierarchy, requirements hierarchy, defined transactions, and detailed, written test cases approved. Stage 4: Test Automation, Execution, and Defect Tracking Stage 100% of test cases are scripted and executed, 100% of produced documents are verified, and 100% of defects are retested and removed. Stage 5: Evaluation and Improvement Automated suite evaluation and improvement. Goals and Deliverables Sample generic goals and deliverables follow. Goals The following list describes the defined goals for the test process: To accomplish all tasks described in this test plan. To install a measurable, improvable, repeatable, and manageable test process at Client Company. To decrease the time necessary to test new builds of Client Companys PROJECT. To verify the functionality and content of the current version of the PROJECT application. To reduce the frequency of error associated with manual testing. To find and successfully track 100% of defects present along the user path defined in this plan. Deliverables The following list describes the defined deliverables for each stage of the testing process: Test Process Assessment - An assessment of the current test process with recommendations for improvement.

Test Planning Stage - A complete Test Plan, including preliminary Test Requirements. Test Design Stage - Test Cases describing input, conditions, and expected output for each requirement, verified Test Requirements. Test Automation, Execution, and Defect Tracking stage Test Scripts, logged test results, defect/issue reports. Evaluation and Improvement Metrics proving the efficiency and benefit of automated testing, Test Cycle Evaluation, Project summary/evaluation.

Glossary of Standard Terms Table 16.1. Glossary Term Definition Test Scenario A path through an application to elicit the normal functioning of the application. The path may be a user path, a path defined by specific requirements or a path examining back-end functionality. Examples: make a deposit (path=common user path); send request to server for cost of mailing a package from point A to point B (path=back-end path). Test Case A text document that states the objectives and details of a test scenario: the steps taken, specific test data used, test conditions, and expected results. Test Script A script containing the Automation Tool code that executes the Test Scenario described in the corresponding Test Case.

TESTING TYPE XML Testing

Java Testing (EJB, J2EE)

DESCRIPTION Validation of XML data content on a transaction-bytransaction basis. Where desirable, validation of formal XML structure (metadata structure) may also be included. Direct exercise of class methods to validate that both object properties and methods properly reflect and handle data according to business and functional requirements of the layer. Exercise of transactions at

Data Integrity Testing

GUI Testing Issue/Defect Tracking

Requirements Management

Interoperability Testing

Functional Testing Security Testing

Business Rules Testing

COM+ Testing

Integration Testing

this layer may be performed to measure both functional and performance characteristics Validation of system data at all data capture points in a system, including front-end, middle- or content-tier, and back-end database. Data integrity testing includes strategies to examine and validate data at all critical component boundaries. Validation of GUI characteristics against GUI requirements. Tracking software issues and defects is at the core of the software quality management process. Software quality can be assessed at any point in the development process by tracking numbers of defects and defect criticality. Software readiness-fordeployment can be analyzed by following defect trends for the duration of the project. Requirements both define the shape of software (lookand-feel, functionality, business rules) and set a baseline for testing. As such, requirements management, or the orderly process of gathering requirements and keeping requirements documentation updated on a release-by- release basis, is critical to the deployment of quality software. Validation that applications in a given platform configuration do not conflict, causing loss of functionality. Validation of business requirements, GUI requirements and data handling in an application. Validation that security requirements of a system have been correctly implemented, including: resistance to password cracking, Denial of Service (DOS) attacks, and that known security flaws have been properly patched. Validation that business rules have been properly implemented in a system, enforcing correct business practices on the user. Direct exercise of COM methods to validate that both object properties and methods properly reflect and handle data according to business and functional requirements of the COM layer. Exercise of transactions at this layer may be performed to measure both functional and performance characteristics. Testing in which software components, hardware components, or both are combined and tested to evaluate the interaction between them.

Network Latency Modeling

Transaction Characterization

Load/Scalability Testing

Performance Testing

Stress Testing

Configuration Testing Volume Testing

Resource Usage Testing Concurrency Testing

Infrastructure Testing Component Testing

Fail over Testing

Analysis of the fundamental amount of time it takes a given message to traverse a given distance across a specific network. This factor influences all messages that traverse a network, and is key in modeling network behavior. Determining the footprint of business transactions. This includes bandwidth on the network, CPU and memory utilization on back-end systems. Additionally used in Network Latency Modeling and Resource Usage Testing. Increase load on the target environment until requirements are exceeded or saturation of a resource. This is usually combined with other test types to optimize performance. Determining if the test environment meets requirements at set loads and mixes of transactions by testing specific business scenarios. Exercising the target system or environment at the point of saturation (depletion of a resource: CPU, memory, etc.) to determine if the behavior changes and possibly becomes detrimental to the system, application or data. Encompasses testing various system configurations to assess the requirements and resources needed. Determining the volume of transactions that a complete system can process. Volume Testing is conducted in conjunction with Component, Configuration and/or Stress Testing. Multi-user testing conducted beyond Transaction Characterization to determine the total resource usage of applications and subsystems or modules. Multi-user testing geared towards determining the effects of accessing the same application code, module or database records. Identifies and measures the level of locking, deadlocking and use of singlethreaded code and locking semaphores. Verifying and quantifying the flow of data through the environment infrastructure. The appropriate tests are conducted against the components individually to verify that each individual component can support without failure. This testing is typically conducted while the environment is being assembled to identify any weak links. In environments that employ redundancy and load balancing, Fail over Testing analyzes the theoretical fail over procedure, tests and measures the overall fail

Reliability Testing

SLA Testing

Web Site Monitoring

over process and its effects on the end-user. Once the environment or application is working and optimized for performance, a longer period (24 to 48 hour) Reliability Test will determine if there are any long term detrimental issues that may affect performance in production. Specialized business transaction testing to measure Service Level Agreements with third party vendors. The typical agreement guarantees a specified volume of activity over a predetermined time period with a specified maximum response time. Monitoring business transaction response times after production deployment to ensure end -user satisfaction.

Test Case

How to Write Better Test Cases Poor test cases do indeed expose you to considerable risk. They may cover the requirements in theory, but are hard to test and have ambiguous results. Better tests have more reliable results as well as lowering costs in three categories: 1. Productivity - less time to write and maintain cases 2. Testability - less time to execute them 3. Scheduling reliability- better reliability in estimates Elements of test cases A test case is a set of actions with expected results based on requirements for the system. The case includes these elements: The purpose of the test or description of what requirement is being tested The method of how it will be tested The setup to test: version of application under test, hardware, software, operating system, data files, security access, time of day, logical or physical date, prerequisites such as other tests, and any another other setup information pertinent to the requirement(s) being tested Actions and expected results, or inputs and outputs Any proofs or attachments (optional) These same elements need to be in test cases for eve ry level of testing

 Unit Integration System, or acceptance testing. They are valid for functional, performance, and usability testing. An alternate description of test cases is that the description, purpose, and setup are the case or Specification. The steps to accomplish it are called a script. Quality of test cases Quality of writing a test case is objective and measurable. It is simple to set up an objective checklist of the structural elements of test cases -- purpose, method, setup, inputs and outputs. Then walk through each case. Is the element there or not? In addition to their structure, the cases must also meet these standards of quality: Accurate: They test what their descriptions say they will test. Economical: They have only the steps or fields needed for their purpose. They don't give a guided tour of the software. Repeatable, self-standing: -. A test case is a controlled experiment. It should get the same results every time no matter who tests it. If only the writer can test it and get the result, or if the test gets different results for different testers, it needs more work in the setup or actions. Appropriate: A test case has to be appropriate for the testers and environment. If it is theoretically sound but requires skills that none of the testers have, it will sit on the shelf. Even if you know who is testing the first time, you need to consider down the road -- maintenance and regression. Traceable: You have to know what requirement the case is testing. It may meet all the other standards, but if its result, pass or fail, doesn't matter, why bother? Self-cleaning: Picks up after itself. It returns the test environment to the pre-test state. Tests should be destructive, including trying to break a simulated production environment in controlled, repeatable ways.

Format of test cases What does a test case look like? They seem to fall into three major groups: Step-by-step Matrix Automated script. While the automated script will run as an online document, there is no assumption that the other two must be paper-based. They, too, might be online. Best uses for each type of case The most productive uses for step-by-step cases are: One-off test cases, each one different Business scenario goes from screen to screen Many processing rules GUI interfaces Input and output hard to represent in a matrix The most productive uses for matrix cases are: Many variations of filling out a form, same fields, different values, input files Same inputs, different platforms, browsers, configurations Character based screens Input and outputs best represented in a matrix Nearly any test can be represented in a matrix, but the question to decide is whether a matrix is the best way to test. It is most important that the matrix be supported by a description, setup, and how to record results for the test. A variation of the matrix is a list of inputs. It can be inserted in a step-bystep test or stand as a matrix with the explanatory elements of the test. Automated scripts: A decision to use automated testing software is more related to the project and organization doing the testing than to what is being tested. There are some technical issues that must be met, varying from tool to tool, but most applications can find a technical fit. The project management must understand that writing automated cases takes longer than manual tests because the manual tests must be still are written first. When the interface is stable, then the tests can be recorded.

The real payback of automated testing comes in the maintenance phase of the software lifecycle. Then the scripts can be executed repeatedly, even unattended, for great savings in testing time.

Besides record/playback scripts, automated tools are used for performance and load testing. They may u manual step-by-step cases or matrixes which detail how se automated tools will be used to create virtual users, launch transaction scripts, monitor performance, and other activities. Choosing a test type The preference for one type of test case over another is driven as much by the culture and perceptions of the organization as by what is the best fit for the software and test plan. Myth: Step-by-step test cases take too long to write. We can't afford them. Reality: They may or may not take longer to write, but it is easy to maintain. They are the only way to test some functions adequately. Myth: A matrix is always the best choice. Make it work. Reality: A persistent problem is putting together a matrix with proper set-up information. Too often this information is omitted, or worse yet, if different setups or classes of input can't be forced into a matrix with a like group, they are not tested at all Myth: High tech is best. If you can automate test cases, do it. Reality: A decision to use automated testing should be based on many factors. Myth: We don't have time to write manual test cases. Let's automate them. Reality: Automated test cases take longer to create than the other two types. Step-by-step cases tend to be more verbal, and matrixes more numeric. Good training should build understanding and confidence to use all types of cases, each where it is most productive. Often the most productive route is to use all three types of cases.

The first two for unit, integration, and system testing; and automated scripts for regression testing. Improving test cases

Improving testability of test cases The definition of testability is easy to test -- accurately. Easy can be measured by how long it takes to execute the test, and whether the tester has to get clarification during the testing process. Accurately means that if the tester follows the directions, the result of pass or fail will be correct. Improving productivity with templates A test case template is a form with labeled fields. This is a great way to start improving test cases. It jump starts the writing process and supports each of the elements of a good case. Here are some other benefits of using templates: Prevents blank page panic Assists the disorganized Builds in standards Prints spiffy looking tests Assists testers to find information Can include other fields relating to testing process

Improving productivity with clones: Cloning test cases means to model one test case on another one. A case is a good candidate for cloning if it fits the need for a step-by-step case but has variables that can easily be substituted. For example, you may have tests for maintaining a supplier database. Many, but not all, the steps would also apply to a shipper database. As you get to know the software throug h requirements or prototypes, strategize which functions work in such a way that you can clone the test cases. Writing them as clones does not mean they need to be tested together. You can clone steps as well as test cases. Word processing and test authoring software support cloning with features such as "Save As," "Copy," and "Replace." It's very important to proofread these cases to make sure all references to the original are replaced in the clone. Matrixes can also be cloned, especially if the setup section is the same. The variables would be changes in the field names and values. Again, make sure to proofread the new version.

Improving productivity with test management software Software designed to support test authoring is the single greatest productivity booster for writing test cases. It has these advantages over word processing, database, or spreadsheet software: Makes writing and outlining easier Facilitates cloning of cases and steps Easy to add, move, delete cases and steps Automatically numbers and renumbers Prints tests in easy-to-follow templates Test authoring is usually included in off-the-shelf test management software, or it could be custom written. Additional functions: Exports tests to common formats Multi-user Tracks test writing progress, testing progress Tracks test results, or ports to database or defect tracker Links to requirements and/or creates coverage matrixes Builds test sets from cases Allows flexible security The seven most common test case mistakes 1. Making cases too long 2. Incomplete, incorrect, or incoherent setup 3. Leaving out a step 4. Naming fields that changed or no longer exist 5. Unclear whether tester or system does action 6. Unclear what is a pass or fail result 7. Failure to clean up Handling challenges to good test cases

Before writing cases, and at every status meeting, find out where the greatest risk of requirement changes are. Strategize what cases will and won't be affected by the change. Write the ones that won't first. Build in variables or "to be decided" that you will come back and fill in later. Make sure the budget owner knows the cost of revising test cases that are already written. Quantify what it costs per case.

Let project management set priorities for which cases should be written or revised. Let them see you can't do it all and ask them to decide where they have greatest risk. Release the not-quite-right test cases unrevised. Ask the testers to mark up what has to be changed. Schedule more time to test each case, plus time for maintaining the tests. If a testing date is moved up, get management to participate in the options of how test cases will be affected. As in the changing requirements challenge, let them choose what they want to risk. Add staff only if time permits one to two weeks of training before they have to be productive, and only if you have someone to mentor and review their work. Shift the order of writing cases so you write those first that will be tested first. Try to stay one case ahead of the testers. You can skinny down the test cases to just a purpose, what requirement is being tested, and a setup. Offer to have writers do the testing and write as they go. Schedule more time for testing and finishing the writing after testing.

Protecting test case assets: The most important activity to protect the value of test cases is to maintain them so they are testable. They should be maintained after each testing cycle, since testers will find defects in the cases as well as in the software. When testing schedules are created, time should be allotted for the test analyst or writer to fix the cases while programmers fix bugs in the application. Configuration management (CM) of cases should be handled by the organization or project, rather than the test management. If the organization does not have this level of process maturity, the test manager or test writer needs to supply it. Either the project or the test manager should protect valuable test case assets with the following configuration management standards: Naming and numbering conventions Formats, file types Versioning Test objects needed by the case, such as databases Read only storage Controlled access Off-site backup Test management needs to have an index of all test cases. If one is not supplied by CM, create your own. A database should be searchable on keys of project, software, test name, number, and requirement. A full-text search capability would be even better.

Leveraging test cases Test cases as development assets have a life beyond testing. They represent a complete picture of how the software works written in plain English. Even if the focus is destructive, they must also prove that all business scenarios work as required. Often the cases are written for testers who are the business users so they use real world language and terms. A set of use cases has tremendous value to others who are working to learn or sell the software: Business users Technical writers Help desk technicians Trainers Sales and marketing staff Web administrators All of these people have a stake in seeing the software succeed, and are also potential testers. Depending on the organization, good will and open communication between test writers and these groups can greatly speed up the time to production or release. TEST CASE CHECKLIST Quality Attributes

Accurate - tests what the description says it will test. Economical - has only the steps needed for its purpose Repeatable, self-standing - it results no matter who tests it. Appropriate - for both immediate and future testers Traceable - to a requirement Self cleaning - returns the test environment to clean state Structure and testability Has a name and number Has a stated purpose that includes what requirement is being tested Has a description of the method of testing Specifies setup information - environment, data, prerequisite tests, security access Has actions and expected results States if any proofs, such as reports or screen grabs, need to be saved Leaves the testing environment clean Uses active case language Does not exceed 15 steps Matrix does not take longer than 20 minutes to test Automated script is commented with purpose, inputs, expected results

Setup offers alternative to prerequisite tests, if possible Is in correct business scenario order with other tests