OOS: PART 1 OF 3: Out-ofSpecification Investigations Still Perplex Labs

By Jerry Lanese, PhD

Out of specification. Its a term that brings the fear of the gods to the laboratory. It causes gridlock, finger pointing, and delays in the normal workflow. When the laboratory reports an out-of-specification (OOS) result, the rest of the organization immediately concludes that the laboratory does not know how to run the test. Out-of-specification observations are not confined to the laboratory, as demonstrated by the following inspectional observation, which points to inadequate laboratory controls as the problem, although the real problem is with a production control: Failure to have adequate laboratory controls that includes sampling plans and test procedures designed to assure that your drug products conform to appropriate strength, quality and purity specifications [21 C. F. R. 211.160(b)]. For example, in 2009 portions of batches of **** powder were rejected due to fill weight OOS results.1 While this series of articles focuses on guidance provided by the Food and Drug Administration (FDA) as it applies to an OOS observation in the laboratory, the FDA and industry often use the term OOS and apply regulatory expectations beyond this scope. Because we seem to use the term out-of-specification so often, it is important to understand the term. For this discussion, OOS observation is the initial laboratory finding that is outside of predefined limits. If the laboratory investigation demonstrates that the original value truly reflects a property of the material, it is a confirmed OOS. The need to investigate a result that is outside of defined limits begins with the good manufacturing processes (GMPs) that were put into place in the late 1970s. Any unexplained discrepancy (including a percentage of theoretical yield exceeding the maximum or

minimum percentages established in master production and control records) or the failure of a batch or any of its components to meet any of its specifications shall be thoroughly investigated, whether or not the batch has already been distributed. The investigation shall extend to other batches of the same drug product and other drug products that may have been associated with the specific failure or discrepancy. A written record of the investigation shall be made and shall include the conclusions and follow-up.2

The Barr Case

The pharmaceutical industry did not hear much about the enforcement of this paragraph, or the term out-of-specification, prior to the early 1990s. Then, Barr Laboratories decided to sue the FDA because its founder, Edward Cohen, believed it was not being treated fairly due to its part in what is often called the Generic Scandal. The FDA saw this as an opportunity to air some of its frustrations and bring legal weight to its regulatory actions through a judicial decision and countersued. The two issues were settled in one trial that ended in 1993. One of the issues was how Barr Laboratoriesand the industry as a wholehandled initial test results that were outside of predefined limits (OOS observations). At the beginning of the trial, the FDAs position was that when the laboratory gets a test result that is outside of product specification, the batch fails and should be rejected immediately. Barr Laboratories took the position that the possibility that the observation was the result of a laboratory makes it appropriate to investigate in order to confirm or refute the original value. During the proceedings, Judge Alfred M. Wolin stated, An out-of-specification result does not constitute a failure. His written decision states, An out-of-specification result identified as a laboratory error by a failure investigation or an outlier test, or overcome by retesting is not a failure. Thus, the Court is unwilling to adopt the governments view of failure.3 OOS suddenly became a permanent part of the pharmaceutical industry lexicon. Although Wolins decision supported Barrs view that it is appropriate to challenge an OOS observation with an investigation, he was critical of the firms investigation process. The FDA began to scrutinize how firms handled OOS observations. (For this discussion, an OOS observation is the initial value obtained in the laboratory. A confirmed OOS is the result of the laboratory investigation in which the original observation is confirmed.) The industry searched for an acceptable way to handle OOS observations. There were many conferences on handling OOS observations and investigations.

Since the Ruling

The FDA has refined and tightened the definition of OOS and what should be investigated. Calibration out-of-tolerance observations no longer fall within the definition.
An OOS observation still brings many laboratories to a standstill. Several years after Wolins decision, the FDA attempted to revise the GMPs with a strong focus on investigationsand proposed a definition of out-of-specification: Out-of-Specification means an examination, measurement, or test result that does not comply with pre-established criteria, as required by 211.160 (b) of this chapter.4 Because this definition references the drug GMPs, it applies only to testing in the pharmaceutical industry. However, the FDA wasand isreporting observations of inadequate OOS investigations in many of the other industries under its jurisdiction. A strict interpretation of this definition also includes out-of-tolerance cali- bration findings, because the requirement for calibration of laboratory

instruments is found in 211.160(b)(4). Although calibration out-of-tolerance findings must be investigated and resolved, the laboratory OOS investigation process is not the appropriate way to handle this situation. Two years later, in 1998, the agency published the draft guidance for handling OOS observations, followed by the final version of the OOS guidance in 2006.5-6 The final version included a clarified definition of OOS and the definition of the limited scope of the guidance: For purposes of this document, the term OOS results includes all test results that fall outside the specifications or acceptance criteria established in drug applications, drug master files (DMFs), official compendia, or by the manufacturer. The term also applies to all in-process laboratory tests that are outside of established specifications. This guidance applies to chemistry-based laboratory testing of drugs regulated by CDER. It is directed toward traditional drug testing and release methods. These laboratory tests are performed on active pharmaceutical ingredients, excipients and other components, in-process materials, and finished drug products to the extent that current good manufacturing practice (CGMP) regulations (21 CFR parts 210 and 211) and the Federal Food, Drug, and Cosmetic Act (the Act) (section 501(a)(2)(B)) apply. The principles in this guidance also apply to in-house testing of drug product components that are purchased by a firm. The OOS guidance also suggests that firms investigate test results that are within predefined limits but are out of trend. The FDA has refined and tightened the definition of OOS and what should be investigated. Calibration out-of-tolerance observations no longer fall within the definition. The scope of the guidance for the investigation of OOS results is restricted. It applies only to chemistry-based testing in the pharmaceutical industry and specifically excludes the investigation of OOS results from testing by physical, biological, and microbiological methods. It also excludes in-process testing performed solely for purposes of triggering real time equipment or system adjustments to prevent process drift and testing by process analytical technology.7 There are still many test results that are out of established limits but do not fall within the scope of the OOS guidance. These must be investigated. Some suggestions for conducting such investigations:

If you are in any FDA-regulated industry performing chemistry-based quality control testing for that matter, if you are in any industry performing chemistry-based quality control testinguse the OOS guidance and the interpretation that follows for the investigation of OOS observations. If you are conducting physical, biological, and microbiological-based testing in an FDAregulated environment and encounter an OOS observation, adapt the steps outlined in the OOS guidance to the testing technique and follow the guidance and the suggestions in this series as closely as possible. Trend all testing, develop statistical control charting tools, and investigate test results that fall outside of defined control limits in the same way OOS observations are investigated. Product development testing laboratories should follow the principles defined in the OOS guidance. Where there are no formal specifications, the organization does have expected limits. Identify the cause of any deviation. Retesting without the identification of the cause of the OOS (or unexpected) observation does not support the philosophy of Quality by Design and continuous improvement. Clearly identify, in all documents supporting the procedures, any in-process testing that includes limits designed to signal production personnel to make an addition or alter the process. These are outside the scope of the OOS guidance. Results outside the defined control limits and, further, beyond reasonably expected values should be investigated, however. This investigation would fit better into the deviation system than the OOS process. Use a very conservative approach in your investigation. Follow your OOS procedure closely and make sure your procedure is consistent with the OOS guidance. During an FDA inspection or other audit, the OOS and deviation/discrepancy investigations will be in the spotlight. An investment in thorough, complete investigations will pay off. During the OOS investigation,

there will be pressure from the rest of the organization to release the material quickly, to limit personnel time, materials, and resources used, and to make a decision using limited information. Resist these pressures and complete a thorough, timely, and scientifically sound investigation. Always keep in mind that the fundamental purpose of an OOS (or any) investigation is to identify and eliminate the source of a problem in order to improve the process in a proactive way. This is the continuous improvement element of any contemporary quality system.

OOS Procedure

Figure 1: An out-of-specification process chart.

Any OOS investigation is based on an OOS investigation procedure. Since the Barr decision, there have been many FDA 483 inspectional observations stating that a firms procedure for OOS investigations is inadequate. Apparently FDA investigators carry a checklist with them that identifies the elements that should be in an adequate OOS procedure. One clue as to what should be included in laboratory OOS procedures comes from the proposed amendments of 1996, with some interpretation to reflect current thinking.8 Procedures shall include:

Criteria for determining whether out-of-specification results were caused by sampling or laboratory error. Scientifically sound procedures and criteria for additional sampling and testing, if necessary. A written record of the investigation, to include: The reason for the investigation; A description of the investigation, including all laboratory tests; Results of the investigation; Scientifically sound and appropriate justification for excluding any OOS laboratory result; and Laboratory results to support the final determination of the tested items conformity to all appropriate specifications or acceptance criteria. Review, evaluation, and approval of the investigation, including the test results, by the quality unit, to ensure a thorough investigation and compliance to regulations and internal policy and procedure. Note: The list found in the proposed amendments includes additional items that should be covered after the laboratory has confirmed an OOS.

To include some current terminology, the procedure should require the determination of the root cause of the OOS observation, corrective action(s), and preventive action(s). The procedure should also reflect the steps in the guidance that will be discussed. A recent 483 observation provides a clear indication that investigators are looking for OOS investigation steps identified in the guidance:

Your firms out-of-specification (OOS) procedure *** is inadequate in that the procedure for invalidation of an OOS result is inappropriate. The procedure does not specify the reanalysis of the original sample solution or stock solution before resampling of a new portion of the

original sample. In this observation, the investigators are looking for analytical sample reinjection, which is included in Phase I of the OOS guidance.9 There are only two reasonable conclusions resulting from a thorough investigation of a laboratory OOS observation: One, the initial observation is correct. The result is confirmed. An error occurred in production of the material. The investigation will have to go beyond the laboratory to identify this error. Or two, the initial observation is incorrect. It does not accurately reflect the measured property of the material. An error occurred in the sampling or testing of the material. The error must be identified in the laboratory OOS investigation. Although the responsibility for sampling is often shared with units within quality assurance or production, it is considered a laboratory activity. In both cases there is a deviation that should be entered in the deviation management system. The laboratory error is relatively minor and might not require further investigation. It should at least be tracked and trended. The confirmed OOS result is a critical deviation and will require the identification of the cause, corrective action, and preventive action. In the next article, we will continue with a discussion of Phase 1 of the laboratory investigation found in the OOS guidance.

REFERENCES
1. 2. U.S. Food and Drug Administration. FDA Warning Letter 320-10-10. September 30, 2010. Available at: www.fda.gov/ICECI/EnforcementActions/WarningLetters/ucm229548.htm. Accessed February 23, 2011. U.S. Food and Drug Administration. Title 21 Code of Federal Regulations (21 CFR 211.192). Current good manufacturing practice for finished pharmaceuticals. Records and reports: production record review. FDA. Available at: www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=211.192. Accessed February 23, 2011. United States of America v. Barr Laboratories, Inc., 812 F Supp 458 (DNJ 1993). U.S. Food and Drug Administration. Proposed changes to Title 21 Code of Federal Regulations Part 211: Current good manufacturing practices for finished pharmaceuticals. Washington, DC: FDA; May 1996. [Proposed rule withdrawn December 2007.] U.S. Food and Drug Administration. Draft Guidance for Industry: Investigating out-ofspecification (OOS) test results for pharmaceutical production. Washington, DC: FDA; September 1998. U.S. Food and Drug Administration. Guidance for Industry: Investigating out-of-specification (OOS) test results for pharmaceutical production. FDA. Available at: www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM07 0287.pdf. Accessed February 23, 2011. U.S. Food and Drug Administration. Title 21 Code of Federal Regulations (21 CFR 211.192). Current good manufacturing practice for finished pharmaceuticals. Records and reports: production record review. FDA. Available at: www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=211.192. Accessed February 23, 2011. U.S. Food and Drug Administration. Proposed changes to Title 21 Code of Federal Regulations Part 211: Current good manufacturing practices for finished pharmaceuticals. Washington, DC: FDA; May 1996. [Proposed rule withdrawn December 2007.] U.S. Food and Drug Administration. Guidance for Industry: Investigating out-of-specification (OOS) test results for pharmaceutical production. FDA. Available at: www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM07

3. 4. 5. 6.

7.

8. 9.

0287.pdf. Accessed February 23, 2011. John G. (Jerry) Lanese is an independent consultant who focuses on quality systems and the components of an effective quality system. In 1994, he formed The Lanese Group, and since that time he has been a consultant in the area of quality systems and cGMP compliance and has advised small and large medical device and pharmaceutical companies, including firms under FDA Consent Decree, API and excipient manufacturers, electronic firms, device component manufacturers, and other manufacturing organizations.

QUALITY CONTROL - OOS Part 2 of 3 | Anatomy of an OOS Result

By Jerry Lanese, PhD

In the previous article, we discussed the background for laboratory out-of-specification (OOS) investigations. In this article, we will discuss Phase I of the OOS investigation as outlined in the OOS Guidance. For the following discussion, refer to Figure 1, which shows a flow diagram of the OOS laboratory investigation process based on the phases indicated in the OOS Guidance. Phases IA, IB, IIA, and IIB can be correlated to discussions in the OOS Guidance. Numbered steps are added in the figure and identified in the discussion below. It should be noted that two steps are in dashed boxes. These are activities outside of the laboratory OOS investigation and are shown only to clarify their relationship to the laboratory OOS investigation. As we will see, the repeat test activity occurs only after the laboratory investigation has identified an attributable laboratory error and has closed. Phase IIA, review of production activities, is conducted by the production or quality unit. Ideally, these activities occur simultaneously with the laboratory investigation. In reality, the timing of the review of production depends on organizational politics and dynamics. Laboratory management and personnel should aggressively and proactively take actions that prevent OOS (and out-of-trend) observations. Management is responsible for providing qualified, maintained, and calibrated equipment and instruments; clear, current, and understandable procedures; and

training for staff at all levels. Analysts must always be alert and on the lookout for potential problems. Problems with systems supporting equipment, procedures, and training should be brought to the attention of management. When an analyst is performing a test and observes something unusual that might result in an OOS test result, the analyst should stop and discuss the situation with a laboratory supervisor before going on. Any decision to stop an analysis and start over should be recorded in the analysts notebook and justified. The laboratory OOS procedure should detail the complete laboratory process, starting with an OOS observation. Unfortunately, the OOS laboratory investigation process is complex and includes a number of decision, or branching, points. Effective, timely, and compliant resolution of OOS observations requires a clearly written OOS procedure and a staff thoroughly trained and knowledgeable in that standard operating procedure (SOP). When an analyst completes a test, compares the result with the predefined limits, and realizes that he or she has an OOS observation (step 1), it is immediately reported to the laboratory manager/supervisor and entered into the OOS logbook (step 2). An OOS logbook and the OOS observation logging procedure should be defined in, and required by, the OOS procedure. Although the OOS logbook and processes related to the logging of OOS observations are not discussed in the OOS Guidance, they are a regulatory expectation. This logbook is often one of the first things requested by an inspection team that includes the laboratory control system in the inspection. Because it is one place in which problems are recorded and will therefore be inspected closely, the logbook should be maintained meticulously. Entry of the OOS observation into the log signals the beginning of the OOS investigation. The OOS procedure should define a maximum time for the completion of an OOS investigation. Many laboratory OOS procedures include a requirement that the laboratory investigation be completed within 20 working days. This expectation appears to be acceptable to the regulatory investigators and was originally derived from testimony in Judge Alfred M. Wolins 1993 decision in the Barr Laboratories case and an FDA inspection guidance. The clock starts running with the entry into the OOS log. Immediately, the supervisor and analyst move to Phase IA (step 3). This step involves the supervisors thorough review of the testing process, the materials and equipment used in the testing process, and the analysts actions. Most organizations have a checklist for this review. The checklist may be a freestanding form, or it may be included in the OOS SOP. It is based on best industry practice and internal experience; there is no one-size-fits-all checklist. The supervisor should not create the checklist on the fly. Investigators evaluate checklists, as evidenced by the following observation: The investigational checklist you currently use is insufficient to detect and evaluate instrument problems and standard/sample preparations errors. In some laboratories, this review is treated as a mechanical, superficial operation without attention to detail, executed only for the purpose of completing the checklist. It is important that the review be thorough. This supervisor review includes the following steps:

Discuss the method with the analyst; confirm analyst training and knowledge and performance of the correct procedure. Examine the raw data obtained in the analysis, including chromatograms and spectra, and identify unidentified peaks and anomalous or suspect information. Verify that the transcription of any data between records is accurate and that the calculations used to convert raw data values into a final test result are scientifically sound, appropriate, and correct; also determine if automated calculation methods are appropriately validated and if unauthorized or unvalidated changes have been made to automated calculation methods. Confirm the performance of the instruments, particularly that they have been qualified for use for this test and that they are in a current state of maintenance and calibration. Determine if appropriate reference standard, solvents, reagents, and other solutions were

used, and that they meet quality control specifications and were properly controlled since receipt. Evaluate the performance of the test method to ensure that it is performing according to the standard expected based on method validation data and historical data. This includes comparing the method performance and spectra or chromatograms obtained during the test to examples in the test method or reported in the method validation report and determining that system suitability results meet acceptance criteria.

Throughout this step, the analyst and supervisor are looking for sound evidence that a laboratory error is the attributable cause of the OOS observation. They must record their actions and findings. Examples of attributable causes include the following:

The analyst used the wrong mobile phase in the preparation of the samples; The analyst used the wrong pipette in a dilution step; The analyst set the wrong wavelength on the spectrophotometer; The oven controls for the gas chromatography column chamber malfunctioned; The spectrophotometer light source malfunctioned; Test procedure steps are not correct; There was a spike in the laboratory power at the time of the analysis; The reference material has expired; The wrong phosphate salt was used in the preparation of one of the required solutions; and A value was incorrectly transcribed from the instrument printout to the spreadsheet used for calculation.

Figure 1. An out-of-specification process chart.

The investigation must identify a specific attributable cause and be supported by data. If the analyst and supervisor are able to identify an attributable cause of the OOS observation (step 4), the investigation and conclusion are recorded in a format defined in the OOS SOP (step 8). The report should include what was done, the conclusion, and the corrective action and recommended preventive action(s) (step 7). In general, the corrective action requires the analyst to repeat the test (step A). The preventive action is an action that will prevent the attributable cause from occurring again. One preventive action that is often recommended is analyst retraining; this has been overused over the past decade and should be recommended cautiously. If the attributable cause is the fact that the analyst did not follow a procedure, the root cause is probably deeper; the procedure and the training the analyst received should be evaluated. The root cause may be buried in the knowledge the analyst has been provided. If the laboratory investigation checklist is designed well, the completion of this form may serve as the laboratory investigation report (step 6). When the report has been reviewed and approved by the quality unit (step 7), the laboratory investigation is complete and closed in the OOS log (step 18). When Phase IA (steps 3 and 4) of the laboratory OOS investigation does not identify the attributable laboratory error, the investigation moves to Phase IB (step 5). This part of the investigation is accomplished by the analyst under the close supervision of the supervisor. While the focus in Phase IA

is on the equipment, solutions, reagents, and procedures, the focus of Phase IB is on the samples and analytical preparations that resulted from the process. This part of the investigation depends on having the test sample and standard preparations available. Many laboratories have a policy that the analyst will not discard test sample and standard preparations until the test results have been reviewed and approved by the designated reviewer. This policy supports the possible need for the sample preparations for a laboratory OOS investigation. Even if the test sample and standard preparations are available, the investigation must consider preparation stability. Standard and sample preparation stability should be identified in the test procedure. If it cannot be found there, it should be in the test method validation report. When the laboratory does not have data that supports the stability of the sample and standard preparations, it is difficult to draw any scientifically sound conclusions based on the examination of these preparations. Because test sample and standard preparation stability information usually covers a limited time, it is important that the investigation move through this step as quickly as possible.

In repeat testing, an analyst repeats the test exactly as described in the test method, taking the analytical sample from the original sample. Typically, the repeat test is performed by the original analyst.
The analytical preparation that yielded the OOS is reanalyzed. If the test is based on chromatography, this analytical preparation is reinjected into the chromatographic system under the conditions defined in the test procedure. If the test is based on spectrophotometry, the analytical sample is introduced into the spectrophotometer sample chamber at the test conditions. An instrument readout that differs from the one that produced the OOS result is an indication of an instrument malfunction and warrants further investigation. Other intermediates of, or extraction residues from, the analytical sample preparation may be available. These should be examined or reprocessed for any clue that would point to an attributable cause of the OOS observation. As with Phase IA, the objective of Phase IB is to identify an attributable cause for the OOS observation (step 6). If that attributable laboratory error is identified, the original laboratory results are invalidated, with appropriate notation made in the analysts notebook or worksheet. Results of the investigation and recommended preventive action (step 7) are identified in the investigation report (Step 8), which is reviewed and approved by the quality unit (step 9). The OOS investigation is closed in the OOS log (step 18), and the laboratory moves to repeat testing (step A). When the laboratory identifies an attributable cause, it should be entered into an appropriate system and tracked to determine if there are any trends that justify a preventive action. As stated in the OOS Guidance: As part of an effective quality system, a firms upper management should appropriately monitor these trends to ensure that any problematic areas are addressed. In repeat testing, an analyst repeats the test exactly as described in the test method, taking the analytical sample from the original sample. Typically, the repeat test is performed by the original analyst. Some organizations, in their OOS SOP, require that a different analyst perform the repeat test. Other organizations define an extensive, multiple-analyst scheme for retesting. The OOS SOP should clearly define the repeat testing process for the laboratory, and, once established, the procedure should be faithfully followed. When the laboratory cannot identify an attributable laboratory error to explain the OOS observation through Phase I (A and B, steps 3 and 4, and steps 5 and 6), there is still the regulatory expectation that the attributable cause be determined and a preventive action implemented to support process improvement. There is also still the organizational hope that the laboratory was wrong and the material will eventually be released. The laboratory investigation moves to Phase II, which will be

discussed in part three of this series. John G. (Jerry) Lanese is an independent consultant with a focus on quality systems and the components of an effective quality system. In 1994, he formed The Lanese Group and consults on quality systems and cGMP compliance for small and large medical device and pharmaceutical companies, including companies under FDA Consent Decree, API and excipient manufacturers, electronic firms, device component manufacturers, and other manufacturing organizations.

QUALITY CONTROL - Part 3 of 3 | OOS: The Last Resort

By Jerry Lanese, PhD

A 70-person PerkinElmer OneSource on-site team takes complete responsibility for maintaining and qualifying more than 50,000 Merck Research Laboratories assets in six facilities.

Practical advice for handling the second phase of out-ofspecification investigations

In the previous two articles, we discussed the background of laboratory out-of-specification (OOS) investigations and Phase I of the OOS investigation as outlined in the U.S. Food and Drug Administration (FDA) OOS guidance. If Phase I (see Figure 1) of the laboratory investigation does not identify an attributable laboratory error as the cause of the original OOS observation, the OOS guidance allows the organization to move into an expanded laboratory investigation (Phase IIB) in parallel with a review of the production activities (Phase IIA). How the two parts of the investigation are coordinated depends on the corporate culture. Review of production records should be a priority, because there is a reasonable possibility that it will reveal that the OOS is attributable to an error in the operations area. The Phase IIB (Step 11) laboratory investigation focuses on retesting to demonstrate that the original

value does not represent the material. There are a number of conflicting opinions and expectations that must be considered as the organization develops its OOS retest policy, procedure, and protocols:

The operations unit is generally not willing to consider that the OOS observation may be an accurate measure of the quality of the material. There is often pressure to release the product, as well as a belief that the path to quick release involves demonstrating that the original failing result does not reflect the quality of the batch. There is some debate in the pharmaceutical industry as to whether there should be any retestingany effort to identify the original value as an outlierif an attributable laboratory error has not been identified in Phase I. It is unlikely that the retesting will identify the cause of the original OOS. At best, it will only confirm that the original value is an outlier without identifying the cause of the original value. The investigation will not be complete in the eyes of the FDA until the cause is identified and corrected. The organization should consider the FDA position that for any deviation, the investigation should identify the cause and recommend a corrective and/or preventive action.

Figure 1. An out-of-specification process chart.

Before moving to Phase IIB, (Step 11) retesting, of the investigation, the supervisor and analyst should ask themselves: Does the test under consideration already include the retest? Some tests defined in the United States Pharmacopeia (USP) have the retest steps built into them, specifically, tests for uniformity of dosage and drug release (dissolution). If the analyst has completed the 24 dosage units described in the USP dissolution procedure or the 30 units described in the USP uniformity of dosage procedure, retesting has been completed. If the criteria are not met, batch failure has already been confirmed. Unfortunately, many do not comprehend the impact of completing these tests with failing results. The laboratory might consider a policy or procedure that requires that the analyst inform laboratory management when moving to the second stage of testing so that management is aware of the potential problem. Because the rest of the organization does not understand that these procedures already include the retest, it will expect the laboratory to move to the OOS observation investigation mode. In reality, the organization should move into the investigation of a confirmed material failure, which will be driven by quality assurance and will involve the whole organization. Any laboratory OOS standard operating procedure should clearly identify the companys policy on the use of outlier testing and the expectation that the cause of the original OOS observation (a deviation) will be identified. The following observations in an FDA warning letter demonstrate that the investigators are looking for the identification of the cause of the deviation: Your OOS procedure contains no provision for conducting retest of new samples, yet your firm released these batches for distribution based on passing repeat test results without conducting a thorough investigation as required under 21 C.F.R. 211.192 to support your conclusion and rationale to release the affected lots . Your firms OOS investigation relating to impurity levels for _____

concluded that the root cause was laboratory error, but the investigation did not identify what specific laboratory error occurred. The regulatory expectation requires that retesting be conducted according to an approved protocol. Since the landmark 1993 Barr ruling, industry representatives have asked the FDA and others for a recommendation on what should be in the protocol. This is not addressed in the OOS guidance. The following should be identified in the OOS procedure as required content for the retest protocol, and the points should be covered in each retest protocol: The sample that will be used for the retesting. Retest replicates should come from the same sample that was used for the original test. Under circumstances in which the original sample is unstable, the sampling procedure will identify this and proceduralize resampling in the event of retest. If the original sample is inadequate, the sampling plan does not meet current regulatory expectations that the laboratory sample be adequate for the original test and any necessary retesting in the event of an OOS observation. If the laboratory continues, a resample will have to be justified and a resampling plan prepared and approved by the quality unit before resampling occurs and retesting is begun. The original, inadequate sampling plan should be investigated and corrected through the site deviation system.

Figure 2. Retesting Scenarios

The method that will be used for the testing. The method used to obtain the original result will be used. For each retest sample preparation, the analyst will perform the complete procedure, including the preparation of mobile phase, standards, system suitability solutions, and other required preparations. If any compromises to this are allowed, they are identified in the protocol. For example, the analysts might batch the analytical and standard preparations on a chromatograph and use a common system suitability preparation to evaluate the operation of the chromatographic system. The analysts who will perform the retesting. The draft OOS guidance stated that the original analyst should not be included in the retesting. This requirement could cause a hardship for some laboratories. The final guidance suggests that at least two analysts, not including the original analyst, be involved in the retesting. Some feel that there is value in having the original analyst participate. Any analyst, including the original, could have a procedural bias that impacts his/her results. That potential must be considered in the preparation of the retest protocol and the evaluation of the data. Suggestion: Retesting should be performed by three analysts, including the original. The protocol should have acceptance criteria for intra- and inter-analyst result agreement, as well as the

requirement that any lack of conformance to these criteria be resolved before the data analysis for an outlier begins. The number of replicates that will be tested. The purpose of retesting is to demonstrate that the original value, the OOS observation, does not accurately reflect the material under test. This is done by demonstrating that the original value is a statistical outlier when compared to multiple, independent retest results of the same material. Two retests will not provide any statistical confidence. Five or six retests will provide borderline confidence. Several possible retest scenarios are shown in Figure 2. Scenario 1 has been published and accepted by many. It is based on a chromatographic procedure in which each injection is calculated as a separate result. However, the individual results lack the independence appropriate for the evaluation. In the other scenarios, the complete test procedure is followed for each retest sample preparation. This includes replicate injections and calculations defined by the procedure. Suggestion: Use nine retests, three by each of three analysts. The protocol should require intra- and inter-analyst agreement as part of the overall outlier study. How the results will be evaluated and what constitutes an outlier. The protocol should identify acceptance criteria for the test results before they are evaluated for the outlier. Where possible, the laboratory should use a statistician to lead it through this evaluation. If a statistician is not available, the laboratory can use one of the standard statistical packages. Test method system suitability criteria must be met. There should be acceptance criteria for intra- and inter-analyst agreement. For the determination that the original value is an outlier, a number of statistical routines have been used. The student T is one routine that is used often; however, the use of any routine to identify a value as an outlier must be justified. Any criteria for determining whether or not the original value is an outlier must be established in the protocol before retesting is initiated. Suggestion: One possible criterion to consider is the following: The original is an outlier if it is outside of calculated average 3 for the retest results. When retesting is complete, the laboratory will report its complete investigation, results, and conclusions to quality assurance, which will be responsible for determining the disposition of the material. Often the firm looks for ways to measure the performance of the laboratory OOS investigation process. Performance measures that should be considered include:

Outstanding OOS investigations; OOS investigations going beyond X days; OOS observations per 100 laboratory tests; OOS observations per month; and Average time for completion of an OOS investigation.

John G. (Jerry) Lanese is an independent consultant with a focus on quality systems and the components of an effective quality system. In 1994, he formed The Lanese Group and consults on quality systems and cGMP compliance for small and large medical device and pharmaceutical companies, including companies under FDA Consent Decree, API and excipient manufacturers, electronic firms, device component manufacturers, and other manufacturing organizations.

Editors Choice
1. U.S. Food and Drug Administration. Guidance for Industry: Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production. FDA. Available at: www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm07 0287.pdf. Accessed June 12, 2011. Kuwahara SS. A history of the OOS problem. Biopharm International website. November 1, 2007. Available at: http://biopharminternational.findpharma.com/biopharm/article/articleDetail.jsp?id=470169&s k=&date=&pageID=6. Accessed June 12, 2011.

2.

3.

Kuselman I, Pennecchi F, Burns C, et al. Investigating out-of-specification test results of chemical composition based on metrological concepts. Accred Qual Assur. 2010; 15(5):283288.

http://pharmaquality.com/ME2/Audiences/dirmod.asp?sid=325598564E8C4B3EB736C715924131 2D&nm=Browse+Articles&type=Publishing&mod=Publications%3A%3AArticle&mid=D3E3C719D8 D44216836DCA4F4144BEC4&tier=4&id=B2D6B04DAEBB4943AE689C5D84B11748&AudID=5648A5 C28C97462DBBDB309539B820EF