BUG TAXONOMY AND STATISTICS

by Boris Beizer, as amended1 by Otto Vinter

Copyright Notice: This document is based1 on the Bug Taxonomy and Statistics Appendix, SOFTWARE TESTING TECHNIQUES, second edition, Van Nostrand Reinhold, New York, 1990, by Boris Beizer. The author retains all beneficial rights thereto. The document appears by written permission of Boris Beizer. All copies of the document must include this copyright notice. EXPLANATION AND NOTATION -- This document is a taxonomy for bugs. Bugs are categorized by a 4-digit number, perhaps with subnumbers using the point system; e.g., "1234.1.6." The "x" which appears is a place holder for possible future filling in of numbers as the taxonomy is (inevitably) expanded. For example: 3xxx -- structural bugs in the implemented software 32xx -- processing bugs 322x -- expression evaluation bugs 3222 -- arithmetic expressions bugs 3222.1 -- wrong operator bugs The last digit in a set is always a 9: e.g., 9xxx, 39xx, 329x, 3229, 3226.9. This is the category to use when a finer breakdown is not available, e.g. for bugs that do not fit in the other (sub)categories: 9xxx -- other or unspecified bugs 39xx -- other structural bugs 329x -- other processing bugs 3229 -- other expression evaluation bugs 3222.9 -- other arithmetic bugs

This document is an amended version of the appendix from the referenced book by Boris Beizer. Otto Vinter has performed the following changes to the original appendix: The 114x category listed in the statistics section has been included in the taxonomy text also. Inconsistencies between category names in the statistics section and the taxonomy text have been corrected. A superfluous SUM-column in the statistics section has been removed. And a few small text changes have been made to fit the present document text. Otto Vinter has made the following additions: Categories 78xx, 8111, and 8112 introduced during the PRIDE project (PRIDE Final Report, Appendix A: http://ottovinter.dk/Finalrp3.doc) have been added to the taxonomy text. And a statistics section covering bug data from a number of projects at Brel & Kjaer has been added. Please address questions about this document to vinter@ottovinter.dk, bbeizer@localnet.com or bsquare@sprintmail.com

BUG TAXONOMY
1xxx: FUNCTIONAL BUGS: REQUIREMENTS AND FEATURES: bugs having to do with requirements as specified or as implemented. 11xx: REQUIREMENTS INCORRECT: the requirement or a part of it is incorrect. 111x: Incorrect: requirement is wrong. 112x: Undesirable: requirement is correct as stated but it is not desirable. 113x: Not needed: requirement is not needed. 114x: Ambiguous: requirement is ambiguous. 12xx: LOGIC: the requirement is illogical or unreasonable. 121x: Illogical: illogical, usually because of a self-contradiction which can be exposed by a logical analysis of cases. 122x: Unreasonable: logical and consistent but unreasonable with respect to the environment and/or budgetary and time constraints. 123x: Unachievable: requirement fundamentally impossible or cannot be achieved under existing constraints. 124x: Inconsistent, incompatible: requirement is inconsistent with other requirements or with environment. 1242: Internal: the inconsistency is evident within the specified component. 1244: External: the inconsistency is with external (to the component) components or the environment. 1248: Configuration sensitivity: the incompatibility is with one or more configurations (hardware, software, operating system) in which the component is expected to work. 13xx: COMPLETENESS: the requirement as specified is either ambiguous, incomplete, or overly specified. 131x: Incomplete: the specification is incomplete; cases, features, variations or attributes are not specified and therefore not implemented. 132x: Missing, unspecified: the entire requirement is missing. 133x: Duplicated, overlapped: specified requirement totally or partially overlaps another requirement either already implemented or specified elsewhere. 134x: Overly generalized: requirement as specified is correct and consistent but is overly generalized (e.g., too powerful) for the application. 137x: Not downward compatible: requirement as specified will mean that objects created or manipulated by prior versions can either not be processed by this version or will be incorrectly processed. 138x: Insufficiently extendable: requirement as specified cannot be expanded in ways which are likely to be needed -- important hooks left out of specification. 14xx: VERIFIABILITY: specification bugs having to do with verifying that the requirement was correctly or incorrectly implemented. 141x: Unverifiable: the requirement, if implemented, cannot be verified by any means or within available time and budget. For example, it is possible to design a test but the outcome of the test cannot be verified as correct or incorrect. 142x: Untestable: it is not possible to design and/or execute tests which will verify the requirement. Untestable is stronger than unverifiable.

15xx: PRESENTATION: bugs in the presentation or documentation of requirements. The requirements are presumed to be correct, but the form in which they are presented is not. This can be important for test design automation systems which demand specific formats. 152x: Presentation, documentation: general presentation, documentation, format, media, etc. 153x: Standards: presentation violates standards for requirements. 16xx: REQUIREMENT CHANGES: requirements, whether or not correct, have been changed between the time programming started and testing ended. 162x: Features: requirement changes concerned with features. 1621: Feature added: a new feature has been added. 1622: Feature deleted: previously required feature deleted. 1623: Feature changed: significant changes to feature, other than changes in cases. 163x: Cases: cases within a feature have been changed. Feature itself is not significantly modified except for cases. 1631: Cases added: 1632: Cases deleted: 1633: Cases changed: processing or treatment of specific case(s) changed. 164x: Domain changes: input data domain modified: e.g., boundary changes, closure, treatment. 165x: User messages and diagnostics: changes in text, content, or conditions under which user prompts, warning, error messages, etc. are produced. 166x: Internal interfaces: direct internal interfaces such as call sequences, or indirect interfaces (e.g., via data structures) have been changed. 167x: External interfaces: external interfaces, such as device drivers, protocols, etc. have been changed. 168x: Performance and timing: changes to performance requirements (e.g., throughput) and/or timings. 2xxx: FUNCTIONALITY AS IMPLEMENTED: requirement known or assumed to be correct, implementable, and testable, but implementation is wrong. 21xx: CORRECTNESS: having to do with the correctness of the implementation. 211x: Feature misunderstood, wrong: feature as implemented is not correct -- not as specified. 218x: Feature interactions: feature is correctly implemented by itself, but has incorrect interactions with other features, or specified or implied interaction is incorrectly handled. 22xx: COMPLETENESS, FEATURES: having to do with the completeness with which features are implemented. 221x: Missing feature: an entire feature is missing. 222x: Unspecified feature: a feature not specified has been implemented. 223x: Duplicated, overlapped feature: feature as implemented duplicates or overlaps features implemented by other parts of the software. 23xx: COMPLETENESS, CASES: having to do with the completeness of cases within features. 231x: Missing case: 232x: Extra case: cases which should not have been handled are.

233x: Duplicated, overlapped case: duplicated handling of cases or partial overlap with other cases. 234x: Extraneous output data: data not required is output. 24xx: DOMAINS: processing case or feature depends on a combination of input values. A domain bug exists if the wrong processing is executed for the selected input-value combination. 241x: Domain misunderstood, wrong: misunderstanding of the size, shape, boundaries, or other characteristics of the specified input domain for the feature or case. Most bugs related to handling extreme cases are domain bugs. 242x: Boundary locations: the values or expressions which define a domain boundary are wrong: e.g., "X>=6" instead of "X>=3." 243x: Boundary closures: end points and boundaries of the domain are incorrectly associated with an adjacent domain: e.g., "X>=0" instead of "X>0." 244x: Boundary intersections: domain boundaries are defined by a relation between domain control variables. That relation, as implemented, is incorrect: e.g., "IF X>0 AND Y>0 .." instead of "IF X>0 OR Y>0..." 25xx: USER MESSAGES AND DIAGNOSTICS: user prompt or printout or other form of communication is incorrect. Processing is assumed to be correct: e.g., a false warning, failure to warn, wrong message, spelling, formats. 26xx: EXCEPTION CONDITIONS MISHANDLED: exception conditions such as illogicals, resource problems, failure modes, which require special handling, are not correctly handled or the wrong exception-handling mechanisms are used. 3xxx: STRUCTURAL BUGS: bugs related to the component's structure: i.e., the code. 31xx: CONTROL FLOW AND SEQUENCING: bugs specifically related to the control flow of the program or the order and extent to which things are done, as distinct from what is done. 311x: General structure: general bugs related to component structure. 3112: Unachievable path: a functionally meaningful processing path in the code for which there is no combination of input values which will force that path to be executed. Do not confuse with unreachable code. The code in question might be reached by some other path. 3114: Unreachable code: code for which there is no combination of input values which will cause that code to be executed. 3116: Dead-end code: code segments which once entered cannot be exited, even though it was intended that an exit be possible. 312x: Control logic and predicates: the path taken through a program is directed by control flow predicates (e.g., boolean expressions). This category addresses the implementation of such predicates. 3122: Duplicated logic: control logic which should appear only once is inadvertently duplicated in whole or in part. 3124: Don't care: improper handling of cases for which what is to be done does not matter either because the case is impossible or because it really doesn't matter: e.g., incorrectly assuming that the case is a don't-care case, failure to do case validation, not invoking the correct exception handler, improper logic simplification to take advantage of such cases.

3126: Illogicals: improper identification of, or processing of, illogical or impossible conditions. An illogical is stronger than a don't care. Illogicals usually mean that something bad has happened and that recovery is needed. Examples of bugs include: illogical not really so, failure to recognize illogical, invoking wrong handler, improper simplification of control logic to take advantage of the case. 3128: Other control flow predicate bugs: control-flow problems which can be directly attributed to the incorrect formulation of a control flow predicate: e.g., "IF A>B THEN..." instead of "IF A<B THEN..." 313x: Case selection bug: simple bugs in case selections, such as improperly formulated case selection expression, GOTO list, or bug in assigned GOTO. 314x: Loops and iteration: bugs having to do with the control of loops. 3141: Initial value: initial iteration value wrong: e.g., "FOR I = 3 TO 17..." instead of "FOR I = 8 TO 17." 3142: Terminal value or condition: value, variable, or expression used to control loop termination is incorrect: e.g., "FOR I = 1 TO 7..." instead of "FOR I = 1 TO 8." 3143: Increment value: value, variable, or expression used to control loop increment value is incorrect: e.g., "FOR I = 1 TO 7 STEP 2..." instead of "FOR I = 1 TO 7 STEP 5.." 3144: Iteration variable processing: where end points and/or increments are controlled by values calculated within the loop's scope, a bug in such calculations. 3148: Exception exit condition: where specified values or conditions or relations between variables force an abnormal exit to the loop, either incorrect processing of such conditions or incorrect exit mechanism invoked. 315x: Control initialization and/or state: bugs having to do with how the program's control flow is initialized and changes of state which affect the control flow: e.g., switches. 3152: Control initialization: initializing to the wrong state or failing to initialize. 3154: Control state: for state-determined control flows, incorrect transition to a new state from the current state: e.g., input condition X requires a transition to state B, given that the program is in state A, instead, the transition is to state C. Most incorrect GOTOs are included in this category. 316x: Incorrect exception handling: any incorrect invocation of a control-flow exception handler not previously categorized. 32xx: PROCESSING: bugs related to processing under the assumption that the control flow is correct. 321x: Algorithmic, fundamental: inappropriate or incorrect algorithm selected, but implemented correctly: e.g., using an incorrect approximation, using a shortcut string search algorithm that assumes string characteristics which may not apply. 322x: Expression evaluation: bugs having to do with the way arithmetic, boolean, string, and other expressions are evaluated. 3222: Arithmetic: bugs related to evaluation of arithmetic expressions. 3222.1: Operator: wrong arithmetic operator or function used. 3222.2: Parentheses: syntactically correct bug in placement of parentheses or other arithmetic delimiters. 3222.3: Sign: bug in use of sign.

3224: Logical or boolean, not control: bug in the manipulation or evaluation of boolean expressions which are not (directly) part of control-flow predicates: e.g., using wrong mask, AND instead of OR, incorrect simplification of boolean function. 3226: String manipulation: bug in string manipulation. 3226.1: Beheading: the beginning of a string is cut off when it should not have been or not cut off when it should be. 3226.2: Curtailing: as for beheading but for string end. 3226.3: Concatenation order: strings are concatenated in wrong order or concatenated when they should not be. 3226.3.1: Append instead of precede: 3226.3.2: Precede instead of append: 3226.4: Inserting: having to do with the insertion of one string into another. 3226.5: Converting case: case conversion (upper to lower, say) is incorrect. 3226.6: Code conversion: string is converted to another code incorrectly or not converted when it should be. 3226.7: Packing, unpacking: strings are incorrectly packed or unpacked 3228: Symbolic, algebraic: bugs in symbolic processing of algebraic expressions. 323x: Initialization: bugs in initialization of variables, expressions, functions, etc. used in processing, excluding initialization bugs associated with declarations and data statements and loop initialization. 324x: Cleanup: incorrect handling of cleanup of temporary data areas, registers, states, etc. associated with processing. 325x: Precision, accuracy: insufficient or excessive precision, insufficient accuracy and other bugs related to number representation system used. 326x: Execution time: excessive (usually) execution time for processing component. 4xxx: DATA: bugs in the definition, structure, or use of data. 41xx: DATA DEFINITION, STRUCTURE, DECLARATION: bugs in the definition, structure and initialization of data: e.g., in DATA statements. This category applies whether the object is declared statically in source code or created dynamically. 411x: Type: the data object type, as declared, is incorrect: e.g., integer instead of floating, short instead of long, pointer instead of integer, array instead of scalar, incorrect userdefined type. 412x: Dimension: for arrays and other objects which have a dimension (e.g., arrays, records, files) by which component objects can be indexed, a bug in the dimension or in the minimum or maximum dimensions, or in redimensioning statements. 413x: Initial, default values: bugs in the assigned initial values of the object (e.g., in DATA statements), selection of incorrect default values, or failure to supply a default value if needed. 414x: Duplication and aliases: bugs related to the incorrect duplication or failure to create a duplicated object. 4142: Duplicated: duplicated definition of an object where allowed by the syntax. 4144: Aliases: object is known by one or more aliases but specified alias is incorrect; object not aliased when it should have been. 415x: Scope: the scope, partition, or components to which the object applies is incorrectly specified.

4152: Local should be global: a locally defined object (e.g., within the scope of a specific component) should have been specified more globally (e.g., in COMMON). 4154: Global should be local: the scope of an object is too global, it should have been declared more locally. 4156: Global/local inconsistency or conflict: a syntactically acceptable conflict between a local and/or global declaration of an object (e.g., incorrect COMMON). 416x: Static/dynamic resources: related to the declaration of static and dynamically allocated resources. 4162: Should be static resource: resource is defined as a dynamically allocated object but should have been static (e.g., permanent). 4164: Should be dynamic resource: resource is defined as static but should have declared as dynamic. 4166: Insufficient resources, space: number of specified resources is insufficient or there is insufficient space (e.g., main memory, cache, registers, disc, etc.) to hold the declared resources. 4168: Data overlay bug: data objects are to be overlayed but there is a bug in the specification of the overlay areas. 42xx: DATA ACCESS AND HANDLING: having to do with access and manipulation of data objects that are presumed to be correctly defined. 421x: Type: bugs having to do with the object type. 4212: Wrong type: object type is incorrect for required processing: e.g., multiplying two strings. 4214: Type transformation: object undergoes incorrect type transformation: e.g., integer to floating, pointer to integer, specified type transformation is not allowed, required type transformation not done. Note, type transformation bugs can exist in any language, whether or not it is strongly typed, whether or not there are user-defined types. 4216: Scaling, units: scaling or units (semantic) associated with object is incorrect, incorrectly transformed, or not transformed: e.g., FOOT-POUNDS to STONE-FURLONGS. 422x: Dimension: for dynamically variable dimensions of a dimensioned object, a bug in the dimension: e.g., dynamic redimension of arrays, exceeding maximum file length, removing one more than the minimum number of records. 423x: Value: having to do with the value of data objects or parts thereof. 4232: Initialization: initialization or default value of object is incorrect. Not to be confused with initialization and default bugs in declarations. This is a dynamic initialization bug. 4234: Constant value: incorrect constant value for an object: e.g., a constant in an expression. 424x: Duplication and aliases: bugs in dynamic (run time) duplication and aliasing of objects. 4242: Object already exists: attempt to create an object which already exists. 4244: No such object: attempted reference to an object which does not exist. 426x: Resources: having to do with dynamically allocated resources and resource pools, in whatever memory media they exist: main, cache, disc, bulk RAM. Included are: queue blocks, control blocks, buffer blocks, heaps, files, etc.

4262: No such resource: referenced resource does not exist. 4264: Wrong resource type: wrong resource type referenced. 428x: Access: having to do with the access of objects as distinct from the manipulation of objects. In this context, accesses include read, write, modify, and (in some instances) create and destroy. 4281: Wrong object accessed: incorrect object accessed: e.g., "X := ABC33" instead of "X := ABD33." 4282: Access rights violation: access rights are controlled by attributes associated with the caller and the object. For example, some callers can only read the object, others can read and modify, etc. Violations of object access rights are included in this category whether or not a formal access rights mechanism exists: that is, access rights could be specified by programming conventions rather than by software. 4283: Data-flow anomaly: data-flow anomalies involve the sequence of accesses to an object: e.g., reading or initializing an object before it has been created, or creating and then not using. 4284: Interlock bug: where objects are in simultaneous use by more than one caller, interlocks and synchronization mechanisms may be used to assure that all data are current and changed by only one caller at a time. These are not bugs in the interlock or synchronization mechanism but in the use of that mechanism. 4285: Saving or protecting bug: application requires that the object be saved or otherwise protected at different program states, or alternatively, not protected. These are bugs related to the incorrect usage of such protection mechanisms or procedures. 4286: Restoration bug: application requires that a previously saved object be restored prior to processing: e.g., POP the stack, restore registers after interrupt. This category includes bugs in the incorrect restoration of data objects and not bugs in the implementation of the restoration mechanism. 4287: Access mode, direct/indirect: object is accessed by wrong means: e.g., direct access of an object for which indirect access is required, call by value instead of name or vice-versa, indexed instead of sequential or vice versa. 4288: Object boundary or structure: access to object is partly correct, but the object structure and its boundaries are handled incorrectly: e.g., fetching 8 characters of a string instead of 7, mishandling word boundaries, getting too much or too little of an object. 5xxx: IMPLEMENTATION: bugs having to do with the implementation of the software. Some of these, such as standards and documentation, may not affect the actual workings of the software. They are included in the bug taxonomy because of their impact on maintenance. 51xx: CODING AND TYPOGRAPHICAL: bugs which can be clearly attributed to simple coding and typographical bugs. Classification of a bug into this category is subjective. If a programmer believed that the correct variable, say, was "ABCD" instead of "ABCE," then it would be classified as a 4281 bug (wrong object accessed). Conversely, if E was changed to D because of a typewriting bug, then it belongs here. 511x: Coding wild card, typographical: all bugs which can be reasonably attributed to typing and other typographical bugs. 512x: Instruction, construct misunderstood: all bugs which can be reasonably attributed to a misunderstanding of an instruction's operation or HOL statement's action.

52xx: STANDARDS VIOLATION: bugs having to do with violating or misunderstanding the applicable programming standards and conventions. The software is assumed to work properly. 521x: Structure violations: violations concerning control-flow structure, organization of the software, etc. 5212: Control flow: violations of control-flow structure conventions: e.g., excessive IF-THEN-ELSE nesting, not using CASE statements where required, not following dictated processing order, jumping into or out of loops, jumping into or out of decisions. 5214: Complexity: violation of maximum (usually) or minimum (rare) complexity guidelines as measured by some specified complexity metric: e.g., too many lines of code in module, cyclomatic complexity greater than 200, excessive Halstead length, too many tokens. 5215: Loop nesting depth: excessive loop nesting depth. 5216: Modularity and partition: modularity and partition rules not followed: e.g., minimum and maximum size, object scope, functionally dictated partitions. 5217: Call nesting depth: violations of component (e.g., subroutine, subprogram, function) maximum nesting depth, or insufficient depth where dictated. 522x: Data definition, declarations: the form and/or location of data object declaration is not according to standards. 523x: Data access: violations of conventions governing how data objects of different kinds are to be accessed, wrong kind of object used: e.g., not using field-access macros, direct access instead of indirect, absolute reference instead of symbolic, access via register, etc. 524x: Calling and invoking: bug in the manner in which other processing components are called, invoked, or communicated with: e.g., direct subroutine call that should be indirect, violation of call and return sequence conventions. 526x: Mnemonics, label conventions: violations of the rules by which names are assigned to objects: e.g., program labels, subroutine and program names, data object names, file names. 527x: Format: violations of conventions governing the overall format and appearance of the source code: indentation rules, pagination, headers, ID block, special markers. 528x: Comments: violations of conventions governing the use, placement, density, and format of comments. The content of comments is covered by 53xx, documentation. 53xx: DOCUMENTATION: bugs in the documentation associated with the code or the content of comments contained in the code. 531x: Incorrect: documentation statement is wrong. 532x: Inconsistent: documentation statement is inconsistent with itself or with other statements. 533x: Incomprehensible: documentation cannot be understood by a qualified reader. 534x: Incomplete: documentation correct but important facts are missing. 535x: Missing: major parts of documentation are missing. 6xxx: INTEGRATION: bugs having to do with the integration of, and interfaces between, components. The components themselves are assumed to be correct. 61xx: INTERNAL INTERFACES: bugs related to the interfaces between communicating components with the program under test. The components are assumed to have passed their component level tests. In this context, direct or indirect transfer of data or control

information via a memory object such as tables, dynamically allocated resources, or files, constitute an internal interface. 611x: Component invocation: bugs having to do with how software components are invoked. In this sense, a "component" can be a subroutine, function, macro, program, program segment, or any other sensible processing component. Note the use of "invoke" rather than "call," because there may be no actual call as such: e.g., a task order placed on a processing queue is an invocation in our sense, though (typically) not a call. 6111: No such component: invoked component does not exist. 6112: Wrong component: incorrect component invoked. 612x: Interface parameter, invocation: having to do with the parameters of the invocation, their number, order, type, location, values, etc. 6121: Wrong parameter: parameters of the invocation are incorrectly specified. 6122: Parameter type: incorrect invocation parameter type used. 6124: Parameter structure: structural details of parameter used are incorrect: e.g., size, number of fields, subtypes. 6125: Parameter value: value (numerical, boolean, string) of the parameter is wrong. 6126: Parameter sequence: parameters of the invocation sequence in the wrong order, too many parameters, too few parameters. 613x: Component invocation return: having to do with the interpretation of parameters provided by the invoked component on return to the invoking component or on release of control to some other component. In this context, a record, a subroutine return sequence, or a file can qualify for this category of bug. Note that the bugs included here are not bugs in the component that created the return data but in the receiving component's subsequent manipulation and interpretation of that data. 6131: Parameter identity: wrong return parameter accessed. 6132: Parameter type: wrong return parameter type used. That is, the component using the return data interprets a return parameter incorrectly as to type. 6134: Parameter structure: return parameter structure misinterpreted. 6136: Return sequence: sequence assumed for return parameters is incorrect. 614x: Initialization, state: invoked component not initialized or initialized to the wrong state or with incorrect data. 615x: Invocation in wrong place: the place or state in the invoking component at which the invoked component was invoked is wrong. 616x: Duplicate or spurious invocation: component should not have been invoked or has been invoked more often than necessary. 62xx: EXTERNAL INTERFACES AND TIMING: having to do with external interfaces, such as I/O devices and/or drivers, or other software not operating under the same control structure. Data passage by files or messages qualify for this bug category. 621x: Interrupts: bugs related to incorrect interrupt handling or setting up for interrupts: e.g., wrong handler invoked, failure to block or unblock interrupts. 622x: Devices and drivers: incorrect interface with devices or device drivers or incorrect interpretation of return status data. 6222: Device, driver, initialization or state: incorrect initialization of device or driver, failure to initialize, setting device to the wrong state. 6224: Device, driver, command bug: bug in the command issued to a device or driver.

6226: Device, driver, return/status misinterpretation: return status data from device or driver misinterpreted or ignored. 623x: I/O timing or throughput: bugs having to do with timings and data rates for external devices such as: not meeting specified timing requirements (too long or too short), forcing too much throughput, not accepting incoming data rates. 7xxx: SYSTEM AND SOFTWARE ARCHITECTURE: bugs that are not attributable to a component or to the interface between components but affect the entire software system or stem from architectural errors in the system. 71xx: OS bug: bugs related to the use of operating system facilities. Not to be confused with bugs in the operating system itself. 711x: Invocation, command: erroneous command given to operating system or OS facility incorrectly invoked. 712x: Return data, status misinterpretation: data returned from operating system or status information ignored or misinterpreted. 714x: Space: required memory (cache, disc, RAM) resource not available or requested in wrong way. 72xx: Software architecture: architectural problems not elsewhere defined. 721x: Interlocks and semaphores: bugs in the use of interlock mechanisms and interprocess communications facilities. Not to be confused with bugs in these mechanisms themselves: e.g., failure to lock, failure to unlock, failure to set or reset semaphore, duplicate locking. 722x: Priority: bugs related to task priority: e.g., priority too low or too high, priority selected not allowed, priority conflicts. 723x: Transaction-flow control: where the path taken by a transaction through the system is controlled by an implicit or explicit transaction flow-control mechanism, these are bugs related to the definition of such flows. Note that all components and their interfaces could be correct but this class of bug could still exist. 724x: Resource management and control: bugs related to the management of dynamically allocated shared resource objects: e.g., not returning a buffer block after use, not getting an object, failure to clean up an object after use, getting wrong kind of object, returning object to wrong pool. 725x: Recursive calls: bugs in the use of recursive invocation of software components or incorrect recursive invocation. 726x: Reentrance: bugs related to reentrance of program components: e.g., a reentrant component which should not be reentrant, a nonreentrant component which should be, a reentrant call which should be nonreentrant. 73xx: RECOVERY, ACCOUNTABILITY: bugs related to the recovery of objects after failure and to the accountability for objects despite failures. 74xx: PERFORMANCE: bugs related to the throughput-delay behavior of software under the assumption that all other aspects are correct. 741x: Throughput inadequate: 742x: Response time, delay: response time to incoming events too long at specified load or too short (rare), delay between outgoing events too long or too short. 743x: Insufficient users: maximum specified number of simultaneous users or task cannot be accommodated at specified transaction delays.

748x: Performance parasites: any bug whose primary or only symptom is a performance degradation: e.g., the harmless but needless repetition of operations, fetching and returning more dynamic resources than needed. 75xx: INCORRECT DIAGNOSTIC, EXCEPTION: diagnostic or error message incorrect or misleading. Exception handler invoked is wrong. 76xx: PARTITIONS AND OVERLAYS: memory or virtual memory is incorrectly partitioned, overlay to wrong area, overlay or partition conflicts. 77xx: SYSGEN OR ENVIRONMENT: wrong operating system version, incorrect system generation, or other host environment problem. 78xx: EXTERNAL AND OTHER THIRD-PARTY SOFTWARE: bugs in the interface to third-party software or other software developed externally. Due to a misunderstanding or wrong interpretation of the features and operation of the third-party software; or due to problems in the third-party software which the vendor does not correct. 8xxx: TEST DEFINITION OR EXECUTION BUGS: bugs in the definition, design, execution of tests or the data used in tests. These are as important as "real" bugs. 81xx: DESIGN BUGS: bugs in the design of tests. 811x: Requirements misunderstood: 8111: Genuine misunderstanding: test and component are mismatched because test designer did not understand requirements. 8112: Suggestion for improvement: ideas or suggestions for another specification than the one correctly implemented. Proposals for improvements which are not accepted on the current version of the product, but which may be included in a later version. 812x: Incorrect outcome predicted: predicted outcome of test does not match required or actual outcome. 813x: Incorrect path predicted: outcome is correct but achieved by wrong predicted path. The test is only coincidentally correct. 814x: Test initialization: specified initial conditions for test are wrong. 815x: Test data structure or value: data objects used in tests or their values are wrong. 816x: Sequencing bug: the sequence in which tests are to be executed, relative to other tests or to test initialization, is wrong. 817x: Configuration: the hardware and/or software configuration and/or environment specified for the test is wrong. 818x: Verification method, criteria: the method by which the outcome will be verified is incorrect or impossible. 82xx: EXECUTION BUGS: bugs in the execution of tests as contrasted with bugs in their design. 821x: Initialization: tested component not initialized to the right state or values. 822x: Keystroke or command: simple keystroke or button hit error. 823x: Data base: data base used to support the test wrong. 824x: Configuration: configuration and/or environment specified for the test was not used during the run. 828x: Verification act: the act of verifying the outcome was incorrectly executed. 83xx: TEST DOCUMENTATION: documentation of test case or verification criteria is incorrect or misleading. 84xx: TEST CASE COMPLETENESS: cases required to achieve specified coverage criteria missing.

BUG STATISTICS by Boris Beizer

This is a spreadsheet dump for combined bug statistics gathered from many different sources by Boris Beizer. The sources are credited in the book: SOFTWARE TESTING TECHNIQUES, second edition, Van Nostrand Reinhold, New York, 1990. The data were collected before 1989. Primarily US defense, aerospace, and telecommunications companies provided the data. The software is typically Systems Software, e.g. operating systems, compilers, and utility software etc. The programming languages comprise: Assembler, FORTRAN, COBOL, ADA, and C. The columns denote: 1. 2. 3. 4-6. 5-7. 8. 9. Category Number -- e.g., 1xxx, 11xx, 119x.... Bug Category Name -- e.g., requirements incorrect. Bug Category Count -- e.g., 111x has a count of 222. Subgroup Count -- e.g., 11xx = 649, 13xx = 224, 15xx = 13. Group Count -- e.g., 1xxx=1317, 2xxx=2624. Note that category, subgroup, and group counts progress from left to right. How many columns are used depends how fine the group breakdown is. Total Percentage. The column shows the percentage of this bug category compared to the total number of bugs: e.g., 1xxx = 8.12% of the whole, 11xx = 4.00% and 111x = 1.37%. Group Percentages. The column percentages relative to the group. You can determine the group by going up to the nearest 100% entry: e.g., 11xx (incorrect requirements) is 49.28% of all requirements bugs, while ambiguous requirements are 0.15% of all requirements bugs. Subgroup Percentages. As for group percentages, except for subgroups: e.g., case completeness (23xx) can be subdivided into missing (75.13%), duplicated or overlapped (5.18%), extraneous output (18.65%), and miscellaneous (1.04%).

10.

____________________________________________________________________________________________________ TOTAL SUM1 SUM2 SUM3 SUM4 Percent Percent Percent KLOC(WITH COMMENTS) 6877.26 of total Group Subgroup TOTAL NUMBER OF BUGS 16209 16209 16209 16209 16209 BUGS/KLOC 2.36 ____________________________________________________________________________________________________ 1xxx FUNCTIONAL BUGS: REQUIREMENTS 1317 1317 8.12% 100.00% AND FEATURES 11xx REQUIREMENTS INCORRECT 649 649 4.00% 49.28% 100.00% 111x Incorrect 222 1.37% 16.86% 34.21% 114x Ambiguous 2 0.01% 0.15% 0.31% 119x Other incorrect requirements 425 2.62% 32.27% 65.49% 12xx REQUIREMENTS LOGIC 153 0.94% 11.62% 100.00% 124x Inconsistent, incompatible 62 0.38% 4.71% 40.52% 1249 Other inconsistencies 62 0.38% 4.71% 40.52% 129x Other requir. logic problems 91 91 0.56% 6.91% 59.48% 13xx 131x 132x 139x 15xx 152x 159x 16xx 162x 1621 1622 1623 1629 164x 165x 167x 169x 2xxx 21xx 211x 218x 219x 22xx 221x 223x 229x 23xx 231x 233x 234x 239x 24xx 241x 242x 243x 249x 25xx 26xx 29xx REQUIREMENTS COMPLETENESS Incomplete Missing, unspecified Other completeness problems REQUIREMENTS PRESENTATION Presentation, documentation Other presentation problems REQUIREMENT CHANGES Features Feature added Feature deleted Feature changed Other feature changes Domain changes User messages and diagnostics External interfaces Other requirements changes FUNCTIONALITY AS IMPLEMENTED CORRECTNESS Feature misunderstood, wrong Feature interactions Other feature bugs COMPLETENESS, FEATURES Missing feature Duplicated, overlapped feat. Other feature completeness COMPLETENESS, CASES Missing case Duplicated, overlapped case Extraneous Output data Other case completeness bugs DOMAIN BUGS Domain misunderstood, wrong Boundary locations Boundary closure Other domain bugs USER MESSAGES AND DIAGNOSTICS EXCEPTION CONDITION MISHANDLED OTHER FUNCTIONAL BUGS 138 84 2 1 12 278 37 3 110 25 5 8 22 68 175 224 224 1.38% 0.85% 0.52% 0.01% 0.08% 0.01% 0.07% 1.72% 1.08% 0.23% 0.02% 0.68% 0.15% 0.03% 0.05% 0.14% 0.42% 2624 2624 16.19% 2.81% 0.43% 0.20% 2.18% 1.43% 0.35% 0.96% 0.12% 1.19% 0.89% 0.06% 0.22% 0.01% 4.80% 1.89% 2.82% 0.07% 0.02% 5.29% 0.49% 0.19% 17.01% 100.00% 10.48% 61.61% 6.38% 37.50% 0.15% 0.89% 0.99% 100.00% 0.08% 7.69% 0.91% 92.31% 21.11% 100.00% 13.29% 62.95% 2.81% 13.31% 0.23% 1.08% 8.35% 39.57% 1.90% 8.99% 0.38% 1.80% 0.61% 2.88% 1.67% 7.91% 5.16% 24.46% 100.00% 17.38% 100.00% 2.67% 15.35% 1.22% 7.02% 13.49% 77.63% 8.80% 100.00% 2.13% 24.24% 5.91% 67.10% 0.76% 8.66% 7.36% 100.00% 5.53% 75.13% 0.38% 5.18% 1.37% 18.65% 0.08% 1.04% 29.65% 100.00% 11.66% 39.33% 17.42% 58.74% 0.42% 1.41% 0.15% 0.51% 32.66% 100.00% 3.01% 100.00% 1.14% 100.00%

13

13

5 8 22 68 456 2624

70 32 354 56 155 20 145 10 36 2 778 306 457 11 4 857 79 30 857 79 30 231

193

____________________________________________________________________________________________________ SUM4 Percent Percent Percent of total Group Subgroup ____________________________________________________________________________________________________ 3xxx 31xx 311x 3119 312x 3128 3129 314x 3142 3144 3149 315x 3159 319x 32xx 321x 322x 3222 3222.3 3222.9 3224 323x 324x 325x 326x 329x 4xxx 41xx 413x 414x 4149 419x 42xx 421x 4212 4214 4216 4219 423x 4232 424x 4249 426x 4269 428x 4281 4282 4283 4285 4287 4288 4289 429x 49xx STRUCTURAL BUGS CONTROL FLOW AND SEQUENCING General structure Other general structure Control logic and predicates Other control-flow predicate bugs Other control logic and predicate bugs Loops and iterations Terminal value or condition Iteration variable processing Other loop and iteration Control initialization and/or state Other control state bugs Other control flow and sequencing PROCESSING Algorithmic, fundamental Expression evaluation Arithmetic expressions Sign Other arithmetic Logic or boolean, not control Initialization Cleanup Precision, accuracy Execution time Other processing DATA DATA DEFINITION, STRUCTURE, DECLARATION Initial, default value Duplication and aliases Other duplication and aliases Other data definition, structure, declaration bugs DATA ACCESS AND HANDLING Type Wrong type Type transformation Scaling, units Other type bugs Value Initialization Duplication and aliases Other duplication and aliases Resources Other dynamic resource Access Wrong object accessed Access rights violation Data-flow anomaly Saving or protecting bug Access mode, direct/indirect Object boundary or structure Other access bug Other access and handling OTHER DATA BUGS 2078 155 268 293 54 1 65 10 1232 155 561 2078 4082 25.18% 12.82% 0.96% 0.96% 3.46% 1.65% 1.81% 0.74% 0.33% 0.01% 0.40% 0.06% 0.06% 7.60% 2004 12.36% 0.75% 2.75% 1.72% 0.19% 1.53% 1.03% 1.87% 0.06% 0.54% 0.29% 6.11% 3638 22.44% 11.14% 0.97% 0.07% 0.07% 10.10% 1831 11.30% 2.21% 0.06% 0.52% 1.46% 0.17% 1.46% 1.46% 0.06% 0.06% 0.07% 0.07% 4.18% 1.51% 0.05% 0.71% 0.06% 0.70% 0.71% 0.44% 3.32% 0.01% 100.00% 50.91% 100.00% 3.80% 7.47% 3.80% 7.47% 13.74% 26.99% 6.56% 12.90% 7.18% 14.10% 2.94% 1.32% 0.02% 1.59% 0.24% 0.24% 30.18% 5.77% 2.60% 0.05% 3.13% 0.48% 0.48% 59.28% TOTAL SUM1 SUM2 SUM3

120

10 1232

121 30 248 167 303 10 88 47 990

121 278 167 303 10 88 47 990

121 445

303 10 88 47 990 3638 1805

49.09% 100.00% 2.96% 6.04% 10.90% 22.21% 6.81% 13.87% 0.73% 1.50% 6.08% 12.38% 4.09% 8.33% 7.42% 15.12% 0.24% 0.50% 2.16% 4.39% 1.15% 2.35% 24.25% 49.40% 100.00% 49.62% 100.00% 4.32% 0.30% 0.30% 45.00% 8.70% 0.61% 0.61% 90.69%

157 11 1637

157 11 1637

359 10 84 237 28 236 10 11 244 8 115 10 113 115 72 538 2 236 10 11 677

538 2 2

50.33% 100.00% 9.87% 19.61% 0.27% 0.55% 2.31% 4.59% 6.51% 12.94% 0.77% 1.53% 6.49% 12.89% 6.49% 12.89% 0.27% 0.55% 0.27% 0.55% 0.30% 0.60% 0.30% 0.60% 18.61% 36.97% 6.71% 13.33% 0.22% 0.44% 3.16% 6.28% 0.27% 0.55% 3.11% 6.17% 3.16% 6.28% 1.98% 3.93% 14.79% 29.38% 0.05% 100.00%

____________________________________________________________________________________________________ SUM4 Percent Percent Percent of total Group Subgroup ____________________________________________________________________________________________________ 5xxx 51xx 511x 519x 52xx 527x 528x 529x 53xx 531x 532x 534x 539x 59xx 6xxx 61xx 611x 6119 612x 6121 6126 6129 613x 6131 6139 614x 619x 62xx 621x 622x 6222 6224 6226 6229 623x 629x 69xx 7xxx 71xx 711x 714x 719x 72xx 721x 724x 729x 73xx 74xx 742x 749x 75xx 76xx 77xx IMPLEMENTATION CODING AND TYPOGRAPHICAL Coding wild card, typographical Other, general coding bugs STANDARDS VIOLATION Format Comments Other standards and style violation bugs DOCUMENTATION Incorrect Inconsistent Incomplete Other documentation, general OTHER IMPLEMENTATION INTEGRATION INTERNAL INTERFACES Component invocation Other component invocation Interface parameter, invocation Wrong parameter Parameter sequence Other invocation param. bugs Component return Parameter identity wrong Other parameter bugs on return Initialization, state Other, internal interfaces EXTERNAL INTERFACES AND TIMING Interrupts Devices and drivers Device, driver, initialization or state Device, driver, command bug Device, driver, return/status misinterpretation Other, devices and drivers I/O timing or throughput Other, external interfaces and timing OTHER INTEGRATION SYSTEM, SOFTWARE ARCHITECTURE O/S CALL, USE BUG Invocation, command Space Other OS call, use bugs SOFTWARE ARCHITECTURE Interlocks and semaphores Resource Management and Control General software architecture RECOVERY AND ACCOUNTABILITY PERFORMANCE Response time, delay Other performance, unspecified INCORRECT DIAGNOSTIC, EXCEPTION PARTITIONS AND OVERLAYS SYSGEN OR ENVIRONMENT 322 26 296 318 15 68 235 15 68 235 960 550 9 146 255 1 1 1 1455 27 27 75 2 51 37 33 221 413 94 3 28 24 82 23 264 78 23 264 78 47 5 3 39 56 8 75 4 44 20 16 3 9 16 3 9 139 78 282 282 282 128 859 1455 960 322 1601 1601 9.88% 1.99% 0.16% 1.83% 1.96% 0.09% 0.42% 1.45% 5.92% 3.39% 0.06% 0.90% 1.57% 0.01% 8.98% 5.30% 0.17% 0.17% 0.79% 0.46% 0.01% 0.31% 0.43% 0.23% 0.20% 1.36% 2.55% 3.20% 0.58% 0.85% 0.02% 0.17% 0.15% 0.51% 0.14% 1.63% 0.48% 1.74% 0.29% 0.03% 0.02% 0.24% 0.86% 0.35% 0.05% 0.46% 0.02% 0.39% 0.27% 0.12% 0.10% 0.02% 0.06% 100.00% 20.11% 100.00% 1.62% 8.07% 18.49% 91.93% 19.86% 100.00% 0.94% 4.72% 4.25% 21.38% 14.68% 73.90% 59.96% 100.00% 34.35% 57.29% 0.56% 0.94% 9.12% 15.21% 15.93% 26.56% 0.06% 100.00% 100.00% 59.04% 100.00% 1.86% 3.14% 1.86% 3.14% 8.80% 14.90% 5.15% 8.73% 0.14% 0.23% 3.51% 5.94% 4.81% 8.15% 2.54% 4.31% 2.27% 3.84% 15.19% 25.73% 28.38% 48.08% 35.60% 100.00% 6.46% 18.15% 9.42% 26.45% 0.21% 0.58% 1.92% 1.65% 5.64% 1.58% 18.14% 5.41% 4.63% 15.83% 4.44% 50.97% TOTAL SUM1 SUM2 SUM3

70 221 413 518 94 137

5.36% 100.00% 100.00% 16.67% 100.00% 1.77% 10.64% 1.06% 6.38% 13.83% 82.98% 49.29% 100.00% 19.86% 40.29% 2.84% 5.76% 26.60% 53.96% 1.42% 100.00% 22.70% 100.00% 15.60% 68.75% 7.09% 31.25% 5.67% 100.00% 1.06% 100.00% 3.19% 100.00%

4 64

____________________________________________________________________________________________________ SUM4 Percent Percent Percent of total Group Subgroup ____________________________________________________________________________________________________ 8xxx 81xx 811x 819x 82xx 823x 824x 828x 829x 83xx 84xx 89xx 9xxx TEST DEFINITION OR EXECUTION TEST DESIGN BUGS Requirements misunderstood Other test design bugs TEST EXECUTION BUGS Data base, wrong Configuration Verification act incorrect Other test execution bugs TEST DOCUMENTATION TEST CASE COMPLETENESS OTHER TEST DEFINITION OR EXECUTION BUGS OTHER BUGS, UNSPECIFIED 11 3 8 355 6 66 28 255 11 64 6 763 11 64 6 763 763 763 763 447 447 447 2.76% 0.07% 0.02% 0.05% 2.19% 0.04% 0.41% 0.17% 1.57% 0.07% 0.39% 0.04% 4.71% 100.00% 2.46% 100.00% 0.67% 27.27% 1.79% 72.73% 79.42% 100.00% 1.34% 1.69% 14.77% 18.59% 6.26% 7.89% 57.05% 71.83% 2.46% 100.00% 14.32% 100.00% 1.34% 100.00% 100.00% 100.00% TOTAL SUM1 SUM2 SUM3

BUG STATISTICS by Otto Vinter

This is a spreadsheet dump for the bug statistics gathered by Otto Vinter from different projects at Brel & Kjaer in Denmark, a leading manufacturer of high-precision measurement instruments for sound and vibration measurement applications (www.bk.dk). KLOCs and BUGS/KLOC cannot be published in this statistic. The data were collected from 1992 1998. The type of software ranges from embedded real-time systems to MS Windows NT applications. The programming languages comprise: Pascal, C, and C++. The project sizes range from 50 100 KLOCs. They were developed by 5-7 persons in 12 18 months, with an average of seven person years. The columns denote: 1. 2. 3. 4-6. 5-7. 8. 9. Category Number -- e.g., 1xxx, 11xx, 119x.... Bug Category Name -- e.g., requirements incorrect. Bug Category Count -- e.g., 111x has a count of 222. Subgroup Count -- e.g., 11xx = 649, 13xx = 224, 15xx = 13. Group Count -- e.g., 1xxx=1317, 2xxx=2624. Note that category, subgroup, and group counts progress from left to right. How many columns are used depends how fine the group breakdown is. Total Percentage. The column shows the percentage of this bug category compared to the total number of bugs: e.g., 1xxx = 8.12% of the whole, 11xx = 4.00% and 111x = 1.37%. Group Percentages. The column percentages relative to the group. You can determine the group by going up to the nearest 100% entry: e.g., 11xx (incorrect requirements) is 49.28% of all requirements bugs, while ambiguous requirements are 0.15% of all requirements bugs. Subgroup Percentages. As for group percentages, except for subgroups: e.g., case completeness (23xx) can be subdivided into missing (75.13%), duplicated or overlapped (5.18%), extraneous output (18.65%), and miscellaneous (1.04%).

10.

____________________________________________________________________________________________________ TOTAL SUM1 SUM2 SUM3 SUM4 Percent Percent Percent KLOC(WITH COMMENTS) of total Group Subgroup TOTAL NUMBER OF BUGS 982 982 982 982 982 BUGS/KLOC ____________________________________________________________________________________________________ 1xxx 11xx 112x 12xx 122x 123x 124x 13xx 131x 132x 133x 134x 139x 16xx 162x 1621 1622 1623 163x 1631 1633 1639 164x 165x 167x 168x 169x 2xxx 21xx 211x 218x 22xx 221x 222x 23xx 231x 232x 234x 24xx 241x 242x 243x 244x 25xx 26xx REQUIREMENTS AND FEATURES REQUIREMENTS INCORRECT Undesirable LOGIC Unreasonable Unachievable Inconsistent COMPLETENESS Incomplete Missing, Unspecified Duplicated,overlapped Overly Generalized Completeness, other REQUIREMENT CHANGES Features Feature Added Feature Deleted Feature Changed Cases Case Added Case Changed Other cases Domain Changes User Messages and Diagnostics External Interfaces Performance and Timing Requirements changes, other FUNCTIONALITY AS IMPLEMENTED CORRECTNESS Feature misunderstood, Wrong Feature interactions COMPLETENESS, FEATURES Missing Unspecified Completeness, Cases Missing Extra Extraneous output data DOMAINS Domain misunderstood, Wrong Boundary locations Boundary closures Boundary intersections USER MESSAGES AND DIAGNOSTICS EXCEPTION CONDITIONS MISHANDLED 5 5 3 6 3 31 31 1 1 1 50 29 2 19 5 4 1 10 13 2 6 4 95 12 12 5 177 177 18,0% 0,5% 0,5% 1,2% 0,3% 0,6% 0,3% 6,6% 3,2% 3,2% 0,1% 0,1% 0,1% 9,7% 5,1% 3,0% 0,2% 1,9% 1,0% 0,5% 0,4% 0,1% 1,0% 1,3% 0,2% 0,6% 0,4% 294 294 29,9% 13,1% 8,4% 4,8% 2,9% 2,4% 0,4% 7,8% 7,3% 0,4% 0,1% 3,9% 2,7% 0,3% 0,7% 0,1% 1,7% 0,5% 100,0% 2,8% 2,8% 6,8% 1,7% 3,4% 1,7% 36,7% 17,5% 17,5% 0,6% 0,6% 0,6% 53,7% 28,2% 16,4% 1,1% 10,7% 5,6% 2,8% 2,3% 0,6% 5,6% 7,3% 1,1% 3,4% 2,3% 100,0% 43,9% 27,9% 16,0% 9,5% 8,2% 1,4% 26,2% 24,5% 1,4% 0,3% 12,9% 9,2% 1,0% 2,4% 0,3% 5,8% 1,7% 100,0% 100,0% 100,0% 25,0% 50,0% 25,0% 100,0% 47,7% 47,7% 1,5% 1,5% 1,5% 100,0% 52,6% 30,5% 2,1% 20,0% 10,5% 5,3% 4,2% 1,1% 10,5% 13,7% 2,1% 6,3% 4,2% 100,0% 63,6% 36,4% 100,0% 85,7% 14,3% 100,0% 93,5% 5,2% 1,3% 100,0% 71,1% 7,9% 18,4% 2,6% 100,0% 100,0%

65

65

10

10 13 2 6 4 294 129

82 47 24 4 72 4 1 27 3 7 1 17 5

28

77

38

17 5

____________________________________________________________________________________________________ SUM4 Percent Percent Percent of total Group Subgroup ____________________________________________________________________________________________________ 3xxx 31xx 311x 3112 3114 3119 312x 3128 3129 313x 314x 3142 3144 315x 3152 3159 316x STRUCTURAL BUGS CONTROL FLOW AND SEQUENCING General structure Unachievable path Unreachable code Other general structure bugs Control Logic and Predicates Other control flow predicate bugs Other logic and predicate bugs Case Selection Bug Loops and Iteration Terminal value or condition Iteration variable processing Control Initialization Control initialization Other control initialization Incorrect Exception Handling 54 3 3 11 7 10 8 4 1 3 1 3 13 1 6 1 1 3 1 32 43 3 7 17 54 165 16,8% 5,5% 1,7% 0,3% 0,3% 1,1% 1,7% 0,7% 1,0% 0,8% 0,5% 0,4% 0,1% 0,4% 0,3% 0,1% 0,3% 11,3% 1,3% 1,3% 0,1% 0,6% 0,5% 0,1% 0,1% 0,3% 0,1% 3,3% 4,4% 0,3% 0,7% 81 81 8,2% 3,8% 0,4% 0,2% 2,9% 0,3% 0,1% 0,1% 0,1% 4,4% 0,5% 0,1% 0,4% 0,5% 0,5% 3,4% 2,2% 1,0% 0,1% 0,1% 100,0% 32,7% 10,3% 1,8% 1,8% 6,7% 10,3% 4,2% 6,1% 4,8% 3,0% 2,4% 0,6% 2,4% 1,8% 0,6% 1,8% 67,3% 7,9% 7,9% 0,6% 3,6% 3,0% 0,6% 0,6% 1,8% 0,6% 19,4% 26,1% 1,8% 4,2% 100,0% 45,7% 4,9% 2,5% 34,6% 3,7% 1,2% 1,2% 1,2% 53,1% 6,2% 1,2% 4,9% 6,2% 6,2% 40,7% 27,2% 12,3% 1,2% 1,2% 100,0% 31,5% 5,6% 5,6% 20,4% 31,5% 13,0% 18,5% 14,8% 9,3% 7,4% 1,9% 7,4% 5,6% 1,9% 5,6% 100,0% 11,7% 11,7% 0,9% 5,4% 4,5% 0,9% 0,9% 2,7% 0,9% 28,8% 38,7% 2,7% 6,3% 100,0% 10,8% 5,4% 75,7% 8,1% 2,7% 2,7% 2,7% 100,0% 11,6% 2,3% 9,3% 11,6% 11,6% 76,7% 51,2% 23,3% 2,3% 100,0% TOTAL SUM1 SUM2 SUM3

17 8 5 4 3 13 1 6 5 13 13 111

32xx PROCESSING 321x Algoritmic, fundamental 322x Expression evaluation 3222 Arithmetic 3224 Logic or Boolean, not control 3226 String manipulation 3226.2 Curtailing 3226.6 Code conversion 3226.9 Other string manipulation bugs 3229 Other expression evaluation bugs 323x Initialization 324x Cleanup 325x Precison, accuracy 39xx 4xxx 41xx 411x 412x 413x 415x 4152 4154 4156 42xx 421x 4212 4216 423x 4232 428x 4281 4283 4289 49xx OTHER STRUCTURAL BUGS

1 32 43 3 7

32 43 3 7 37

DATA DATA DEFINITION, STRUCTURE, DECLARATION Type 4 Dimension 2 Initial, default values 28 Scope Local should be global 1 Global should be local 1 Global/local inconsistency or conflict 1 DATA ACCESS AND HANDLING Type Wrong type Scaling, units Value Initialization Access Wrong object accessed Data-flow anomaly Other access bugs OTHER DATA BUGS

4 2 28 3

5 1 4 5 22 10 1 1 5 33

43

____________________________________________________________________________________________________ SUM4 Percent Percent Percent of total Group Subgroup ____________________________________________________________________________________________________ 5xxx 51xx 52xx 522x 53xx 531x 532x 533x 534x 535x 539x 6xxx 61xx 611x 612x 6122 6129 613x 6132 6139 619x 62xx 621x 622x 6222 623x 629x 69xx 7xxx 71xx 711x 719x 72xx 721x 722x 724x 726x 729x 74xx 741x 742x 749x 77xx 78xx 8xxx 81xx 811x 8111 8112 8119 812x 814x 817x 818x 83xx 89xx 9xxx IMPLEMENTATION CODING AND TYPOGRAPHICAL STANDARDS VIOLATION Data definition, declaration DOCUMENTATION Incorrect Inconsistent Incomprehensible Incomplete Missing Other documentation bugs INTEGRATION INTERNAL INTERFACES Component Invocation Interface Parameter, Invocation Parameter type Other parameter invocation Component Invocation Return Parameter type Other component invocation return Other internal interface bugs 1 2 8 2 1 8 4 14 37 1 2 40 40 40 4,1% 0,1% 0,2% 0,2% 3,8% 0,8% 0,2% 0,1% 0,8% 0,4% 1,4% 10 1 1 1 1 2 4 3 4 8 2 1 2 3 1 2 1 1 19 9 5 1 2 2 2 5 4 14 15 11 1 61 61 61 1 2 19 19 1,9% 1,0% 0,1% 0,2% 0,1% 0,1% 0,3% 0,1% 0,2% 0,4% 0,8% 0,2% 0,1% 0,1% 0,2% 0,3% 0,1% 6,2% 0,2% 0,1% 0,1% 1,9% 0,9% 0,5% 0,1% 0,2% 0,2% 1,1% 0,2% 0,5% 0,4% 1,4% 1,5% 71 27 26 13 1 1 2 1 1 14 59 66 86 86 8,8% 7,2% 6,7% 2,7% 2,6% 1,3% 0,1% 0,1% 0,2% 0,1% 0,1% 1,4% 59 59 6,0% 100,0% 2,5% 5,0% 5,0% 92,5% 20,0% 5,0% 2,5% 20,0% 10,0% 35,0% 100,0% 52,6% 5,3% 10,5% 5,3% 5,3% 15,8% 5,3% 10,5% 21,1% 42,1% 10,5% 5,3% 5,3% 10,5% 15,8% 5,3% 100,0% 3,3% 1,6% 1,6% 31,1% 14,8% 8,2% 1,6% 3,3% 3,3% 18,0% 3,3% 8,2% 6,6% 23,0% 24,6% 100,0% 82,6% 76,7% 31,4% 30,2% 15,1% 1,2% 1,2% 2,3% 1,2% 1,2% 16,3% 100,0% 100,0% 100,0% 100,0% 100,0% 21,6% 5,4% 2,7% 21,6% 10,8% 37,8% 100,0% 10,0% 20,0% 10,0% 10,0% 30,0% 10,0% 20,0% 40,0% 100,0% 25,0% 12,5% 12,5% 25,0% 37,5% 100,0% 100,0% 50,0% 50,0% 100,0% 47,4% 26,3% 5,3% 10,5% 10,5% 100,0% 18,2% 45,5% 36,4% 100,0% 100,0% 100,0% 93,0% 38,0% 36,6% 18,3% 1,4% 1,4% 2,8% 1,4% 100,0% 100,0% TOTAL SUM1 SUM2 SUM3

EXTERNAL INTERFACES AND TIMING Interrupts 2 Devices and Drivers Device, driver, initialization or state1 I/O Timing or Throughput 2 Other external interface bugs 3 OTHER INTEGRATION BUGS SYSTEM AND SOFTWARE ARCHITECTURE OS bug Invocation, command Other OS bugs SOFTWARE ARCHITECTURE Interlocks and semaphores Priority Resource management and control Reentrance Other architecture bugs PERFORMANCE Throughput inadequate Response time, delay Other performance bugs SYSGEN OR ENVIRONMENT EXTERNAL SOFTWARE, THIRD PARTY TEST DEFINITION AND EXECUTION BUGS DESIGN BUGS Requirements misunderstood Genuine misunderstanding Proposal for improvement Other requirements misunderstood Incorrect Outcome Predicted Test initialization Configuration Verification method, criteria TEST DOCUMENTATION OTHER TEST BUGS OTHER BUGS, UNSPECIFIED 1

14 15

1 1 2 1 1 14 59 1 14 59