You are on page 1of 5

20 E P


14 rog
J M ram
The Journal of Molecular Diagnostics, Vol. 16, No. 3, May 2014

Methods-Based Prociency Testing in Molecular Genetic
Iris Schrijver,*y Nazneen Aziz,z Lawrence J. Jennings,x{ Carolyn Sue Richards,k Karl V. Voelkerding,**yy and Karen E. Weckzzxx

From the Departments of Pathology* and Pediatrics,y Stanford University Medical Center, Stanford, California; the Transformation Program Ofce,z College
of American Pathologists, Northeld, Illinois; the Department of Pathology and Laboratory Medicine,x Ann & Robert H. Lurie Childrens Hospital of
Chicago, Chicago, Illinois; the Department of Pathology and Laboratory Medicine,{ Northwestern University Feinberg School of Medicine, Chicago, Illinois;
the Department of Molecular and Medical Genetics,k Oregon Health & Science University, Portland, Oregon; the Department of Pathology,** University
of Utah School of Medicine, Salt Lake City, Utah; the ARUP Laboratories Institute for Clinical and Experimental Pathology,yy Salt Lake City, Utah;
and the Departments of Pathology and Laboratory Medicinezz and Genetics,xx University of North Carolina, Chapel Hill, North Carolina

CME Accreditation Statement: This activity (JMD 2014 CME Program in Molecular Diagnostics) has been planned and implemented in accordance with the Essential Areas and
policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Society for Clinical Pathology (ASCP) and the
American Society for Investigative Pathology (ASIP). ASCP is accredited by the ACCME to provide continuing medical education for physicians.

The ASCP designates this journal-based CME activity (JMD 2014 CME Program in Molecular Diagnostics) for a maximum of 48 AMA PRA Category 1 Credit(s). Physicians
should only claim credit commensurate with the extent of their participation in the activity.

CME Disclosures: The authors of this article and the planning committee members and staff have no relevant nancial relationships with commercial interests to disclose.

What Is Methods-Based Prociency Testing? analyte-specic testing in a practice environment of large-scale

sequencing of patients DNA.
Prociency testing (PT) is intended to be an external measure Methods-based prociency testing (MBPT) is a subset of
of clinical laboratory quality.1 In the United States, it is a overall PT and refers to an EQA approach that is based on
requirement of accreditation by the Centers for Medicare and method, rather than based on each individual analyte tested.
Medicaid Services. It is part of a quality assurance program to MBPT is already well established for several pathology sub-
verify the accuracy and reliability of laboratory testing. Lab- specialty areas, and the CAP offers a variety of PT products that
oratories in the United States are certied under the Clinical are wholly methods based, for example, in cytogenetics, ow
Laboratory Improvement Amendments and accredited by cytometry, and immunohistochemistry. Until recently, how-
professional organizations with deemed status, such as the ever, PT in the area of molecular testing has been entirely an-
College of American Pathologists (CAP). Participation in alyte specic. The CAP supports an MBPT approach for
external quality assessment (EQA) may be through CAP PT clinical molecular genetic testing, and the Centers for
programs or through another prociency testing provider Medicare and Medicaid Services agrees with the concept of
accepted by the Clinical Laboratory Improvement Amend- MBPT in molecular diagnostics, taking into account the cost
ments. For those analytes that do not have EQA surveys and difculty in obtaining materials for the many different
available, laboratories must implement an alternative EQA possible mutations in any given gene. Thus, the concept
(PT) assessment procedure. Appropriate alternative perfor- of MBPT complies with federal laboratory regulations.
mance assessment procedures may include split sample anal- Although MBPT in molecular diagnostics is currently
ysis with other laboratories, or, if that is not available,
assessment of split samples with an established in-house
method and previously assayed material, which are run and Accepted for publication February 11, 2014.
interpreted by laboratory personnel who do not have access to Disclosures: None declared.
Current address of N.A., Phoenix Childrens Hospital, Phoenix, AZ.
the prior results.2 With the increase in diagnostic sequencing, Address correspondence to Iris Schrijver, M.D., Department of Pathol-
however, the list of genes for which CAP PT is not available is ogy, L235, Stanford University Medical Center, 300 Pasteur Dr., Stanford,
rapidly growing, and it is virtually impossible to continue CA 94305. E-mail:

Copyright 2014 American Society for Investigative Pathology

and the Association for Molecular Pathology.
Published by Elsevier Inc. All rights reserved.
Schrijver et al

limited to inherited genetic conditions, in the future, mo- effectively for thousands of individual genes than a disease-
lecular cancer testing will probably include MBPT as well. specic PT approach, in which the PT for each gene would
Methods-based approaches are not limited to PT, but they incorporate numerous challenge events. In addition, MBPT is
also affect other aspects of developing and implementing not analyte specic; therefore laboratories performing rare tests
laboratory testing. Several published guidelines and exam- can participate in an EQA program, provided they are using
ples describe the steps involved in validating a molecular test, comparable methodologies. By participating, the performance
but they focus on analyte-specic tests.3e11 It is important to of laboratories performing rare genetic tests can be evaluated
note that these same validation principles can also apply to and even graded. MBPT for ultra-rare disorders (<2000 cases
methods-based testing. In fact, several molecular tests may be prevalence) has already been recommended by the American
better approached through method-based validation rather College of Medical Genetics and Genomics,13 but, in light of
than through analyte-specic validation, particularly when it the paradigm shift that results from widespread NGS-based
may be impractical or even impossible to obtain positive genomic testing, the CAP extends this option to less rare con-
samples of all targeted mutations. All validations should ditions, as well. Of note, MBPT is only acceptable if provided
address the same critical question: given the intended use, is by an external PT provider, as opposed to an internal scheme
the test t for its purpose? To answer this question, labora- designed by individual laboratories.
tories must assess the potential sources of error and must Although there are clear advantages to the integration of
provide documented evidence that the test will consistently MBPT, laboratories are also required to participate in analyte-
meet the predetermined analytical requirements. For example, specic PT, if available, because specic aspects of test per-
validation of a new DNA extraction method may only require formance and interpretation can be optimally addressed by PT
documentation that the method yields DNA of sufcient designed for the gene(s) or disease process being analyzed.
quantity, purity, and integrity for anticipated downstream ap- Such aspects include the assessment of competency about the
plications. More complex methods, however, may also be identication and interpretation of specic mutations in a given
amenable to method-based validation, at least for those gene, as opposed to a more general nomenclature-specic
processes that have been standardized. Multiplex ligation- competency. Thus, laboratories should subscribe to all formal
dependent probe amplication is an example of a method that assay/analyte-specic PT available for the assays they perform,
involves several steps, but these steps have been standardized because this evaluates gene- and mutation-specic competency.
across probe sets. After validating the analytical performance of MBPT also has limitations compared with analyte-based PT.
the test system and verifying equivalent performance across Traditional PT challenges have attempted to replicate clinical
multiple probe sets, the validation of a new probe set may samples to assess performance of the entire test system. MBPT,
be limited to actual probe validation.12 Similarly, aspects of by contrast, is focused on a step or process within the test
chromosomal microarray testing, Sanger sequencing, and next- system and may fail to assess other error-prone steps. For
generation sequencing (NGS) are amenable to method-based example, MBPT may be focused on the analytical step rather
validation. The initial validation of the test system may be than preanalytical steps (eg, specimen processing and nucleic
quite involved, but the addition of another targeted genomic acid extraction) or the postanalytical steps (eg, interpretation
region or gene sequence may only require probe or primer and reporting). However, these challenges are not exclusive to
validation and a determination of the normal reference range. MBPT. Even traditional PT samples sacrice authenticity to
provide samples that are stable, traceable, and homogenous, or
which target rare, well-characterized mutations. Therefore,
What Are the Advantages and Limitations of most PT samples, whether traditional or method-based, are
MBPT? variably articial and cannot assess every single individual step
in a process. Nevertheless, PT samples provide an external
The advantages of MBPT are in part practical, because it is measure of quality and therefore are a critically important part
logistically challenging to develop separate prociency chal- of a laboratorys quality assurance program. The mentioned
lenges for each of the approximately 20,000 human genes that limitations are considered acceptable, provided the laboratory
may be analyzed. More importantly, a widespread MBPT monitors error-prone steps in the test system in other ways
program in which all laboratories conducting nucleic acid (eg, quality controls, monitoring prevalence of mutations in a
sequencing may participate provides better comparisons of population, report review, etc), and these processes are
performance among laboratories to efciently and thoroughly incorporated in CAP molecular pathology checklists and
test the important aspects of analysis and interpretation that are reviewed during on-site peer-reviewed inspections.
central to laboratory performance. For example, the ability of
laboratories to detect different types of mutations, including
single nucleotide variants, insertions and deletions, and muta- Why Is the Time Right for Implementing
tions in GC-rich genomic regions and to recognize technical MBPT in Molecular Genetic Pathology?
reasons for false positive or false negative results (such as allele
dropout) can systematically be tested by an MBPT program. The rationale for implementing MBPT for molecular
MBPT testing can assess prociency much more efciently and testing addresses several key aspects of sequencing-based

284 - The Journal of Molecular Diagnostics

Methods-Based Prociency Testing

technologies that are unique compared with other types of asked to include all variants identied, to indicate zygosity
molecular testing: i) many clinical laboratories are currently for each, and to provide a basic interpretation for each
performing genomic sequencing analysis for a variety of variant. Interpretations include the categories of pathogenic,
multigene panels by using similar (Sanger or NGS-based) benign, or of unknown clinical signicance, according to the
methods, even though the specic gene(s) tested may be guideline of the American College of Medical Genetics and
different; ii) important aspects of the technical performance Genomics Interpretation of Sequence Variants.16 After 3
and annotation of sequence variants are key to laboratory years of the SEC survey,17 it is clear that MBPT for mo-
prociency and can be tested systematically by MBPT but lecular genetic testing can be successfully developed and
may be agnostic to the specic gene(s) being tested; iii) the implemented and that the feedback to laboratories from this
ability of a laboratory to design, validate, perform, and survey has resulted in improved and more consistent inter-
interpret certain types of genomic sequencing analysis is not pretation of sequence variants.
wholly dependent on the specic gene(s) or disease process The original CAP PT program for sequencing was
being tested but rather on their ability to conduct the type of expanded in 2013 to include a wet lab MBPT challenge
sequencing analysis in its totality; and iv) an MBPT program (SEC-1), which was designed to test not only analytical and
can be efciently designed to test the ability of laboratories to clinical interpretation but also the technical component of
analyze the wide spectrum and types of results that are crucial Sanger sequencing. This survey includes three DNA spec-
to procient laboratory performance in genomic sequencing imens and primers for sequencing, in addition to the mate-
analysis. rials listed above that are supplied in the dry survey.
In 2010, a methods-based Sequencing Educational In SEC-1 the laboratories are asked to both generate the
Challenge (SEC) survey was launched by the Biochemical sequence and to interpret results for nucleotide change,
and Molecular Genetics Committee of the CAP and the zygosity, predicted protein change, and predicted pathoge-
American College of Medical Genetics and Genomics. Of nicity. Participation in the wet survey is encouraged for
note, this was preceded (by years) by European laboratories laboratories that perform sequencing in house, whereas
MBPT that involved many countries.14 Subsequently these laboratories that send out the sequencing but interpret the
SEC surveys have been developed and evaluated by the subsequently received data are encouraged to subscribe to
Biochemical and Molecular Genetics Committee. In the the dry survey. The Biochemical and Molecular Genetics
initial 2 years, participating laboratories interpreted elec- Committee will evaluate the performance of laboratories
tronic sequence traces in a variety of disclosed genes for participating in this survey, going forward.
which these laboratories did not necessarily offer testing. Because MBPT is a reasonable and logical approach in the
With the use of the information provided with these dry era of genomic medicine, CAP has recently initiated the
challenges, laboratories applied the same methods for development of a pilot PT for a methods-based NGS PT
analysis and interpretation as they would for those genes product in which a healthy persons genome, referred to as the
they routinely sequence. The purpose of the CAP dry CAP genome, has been extensively sequenced by the three
sequencing challenges is to test laboratory performance by commonly used commercial NGS platforms. Laboratories will
MBPT, to assess analytical capability, as well as the clinical be evaluated for their prociency to correctly call single
interpretation component for the diversity of sequencing nucleotide variants, insertions, deletions, and nonvariant nu-
results that diagnostic laboratories may encounter when cleotides in this genome. These MBPT pilot studies address
testing by Sanger sequencing methods. Laboratories multiple key aspects of the NGS workow, such as the indi-
participating in the SEC survey ( vidual steps of sequence read generation, sequence mapping,
docs/prociency_testing/2014_surveys_catalog.pdf, last alignment, variant calling, and annotation of pathogenicity.
accessed January 2, 2014) each received a compact disk that They are expected to be completed and evaluated in early 2014.
included three sequence data les that comprise the PT In Europe, the European Molecular Genetics Quality
challenge; three normal sequence data les; additional in- Network and the UK National External Quality Assessment
formation that includes the gene name, the genomic, and the Scheme for molecular genetics recently launched a pilot MBPT
coding RefSeq numbers; a predicted protein translation of for NGS. Thirty laboratories were selected and received a well-
the sequence; a Mutation Surveyor (SoftGenetics, State characterized genome. Individual laboratories were advised to
College, PA) sequence analysis le; web-based references run their smallest panel or their largest gene. This scheme was
to the gene-specic databases and single nucleotide poly- specically designed to assess the technical ability of labo-
morphism databases (National Center for Biotechnology ratories that perform NGS-based diagnostic testing, focusing
Information Entrez SNP Database; http://www.ncbi.nlm.nih. on the sequence quality rather than assessing prociency of
gov/snp?db, last accessed February 6, 2013); and references accurate variant calls. The NGS EQA scheme has been pre-
to Human Genome Variation Society nomenclature rules sented at several conferences but has not yet been published.
(, last accessed February 6, Currently, the scheme report is available only to European
2013).15 The laboratories were directed to use standard Molecular Genetics Quality Network NGS PT participants (S.
Human Genome Variation Society nomenclature to report Patton, personal communication, Director, European Molec-
identied sequence changes. In addition, laboratories were ular Genetics Quality Network).

The Journal of Molecular Diagnostics - 285

Schrijver et al

With the rapid adoption of sequencing-based clinical testing, own setting; ii) laboratories process samples and generate
the number of disease-causing variants in almost every area of sequence data but then outsource the data analysis and
medicine to which molecular diagnostic testing is applied is interpretation components; iii) laboratories outsource the wet
growing and is projected to expand. In genome analysis, the laboratory component but develop the informatics pipeline in
number of variants per person is approximately three million. house; and iv) laboratories and software companies specialize
For exomes, that number approximates 15 to 20,000.18 With in providing data analysis and interpretation. In some sce-
any NGS test, a large number of novel variants will be narios, laboratories that outsource data analysis review the
discovered, some of which will be disease causing. For analyzed data returned to them to arrive at a diagnostic
inherited diseases our earlier understanding of a single gene interpretation. These different approaches were considered
mutation for a single disease has evolved and concludes that when the CAP NGS Working Group developed
there are i) relatively few common disease-causing variants in a NGS-specic accreditation requirements. Thus, the NGS
single gene (eg, broblast growth factor receptor 3, which is Working Group decided to develop separate accreditation
associated with, for example, achondroplasia); ii) multiple rare requirements for the wet bench (inclusive of sequencing) and
disease-causing variants in a single gene (eg, predisposition to the bioinformatics data analysis components. This served the
breast cancer: breast cancer 1 or 2, early onset); iii) a few pragmatic function of accommodating laboratories that pur-
common disease-causing variants in multiple genes associated sue different approaches with respect to implementing NGS.
with a single disease (eg, cardiomyopathy and some of the By extension, EQA testing for NGS should also incor-
complex diseases such as type 2 diabetes); and iv) multiple rare porate diverse implementation approaches. To address this,
disease-causing variants in multiple genes associated with a the CAP NGS Working Group is developing an MBPT
single disease (some common complex diseases such as challenge that will address the total process as described
hypertension). above. Importantly, it should be noted that the different
Therefore, disease-causing mutations will outnumber the approaches to the adoption of NGS (as listed above) pose a
handful of genetic mutations that are currently typically challenge when the wet and the dry bench components are
tested in clinical laboratories. The MBPT approach assesses performed at different addresses. In the United States, PT/
the expertise of laboratories to correctly perform a test EQA referral is a violation of federal law, and under the
method and to accurately identify the types of variants current rule it may not be possible to test the entire NGS
encountered during genomic analysis (single nucleotide process under a single EQA challenge. Given that the
variants, insertions, deletions, duplications, inversions, ho- computing often does not occur at a physical address but
mopolymers, repeated sequence, and chromosomal trans- rather in the cloud, it will be interesting to follow how the
locations), rather than developing PT for each and every interpretation of this rule evolves in the future.
known disease gene and its associated mutations. The Separate dry NGS PT challenges are being discussed that
ability of the operator to correctly call the presence/absence will focus on the bioinformatics component of the work-
of different categories of variants provides a broad assess- ow. The dry challenges could be comprised of sequence
ment and helps determine whether the laboratory is able to data sets (eg, FASTQ, BAM les) derived from a genome,
handle the subtleties related to identifying different types of exome, or gene panel analysis spiked with in silico sequence
genetic mutations by using a specic assay method. reads that contain articially created variants, for example,
single nucleotide variants or insertions and deletions. Such
PT will interrogate a laboratorys ability to properly align
Development of MBPT for NGS and map sequence data and to identify and annotate vari-
ants. In silico dry challenges are particularly useful in mo-
The total process of generating a diagnostic result by using lecular oncology testing or other mixed genotype sample
NGS is comprised of a wet bench workow to generate a testing, including mosaicism and mitochrondrial hetero-
sequencing library and to sequence the library. There also plasmy, in which the limit of detection of the assay is critical
is a dry bench workow, which consists of analysis of for identifying low-frequency alleles. By admixing synthetic
sequence reads and subsequent interpretation of variants sequence reads that contain mutations of interest at varying
obtained. This multistep process is technically complex and ratios with a set of true sequence reads it will be possible to
is being used for the diagnostic evaluation of multigene assess a laboratorys analytical bioinformatics aptitude to
panels, exomes, and genomes for both germline and somatic identify low-frequency mutations, which will be encoun-
variation. Clinical laboratories with experience in high- tered in clinical patient samples. Other advantages of dry
complexity molecular testing are generally well suited for bench testing are related to logistic issues associated with
implementing the wet bench methods and for operating creating wet bench PT samples for cancer. An ideal sample
NGS instrumentation. In contrast, the analysis of NGS read would be a genomic DNA which harbors multiple cancer
data sets is a new subject for clinical laboratories.19 mutations all in the same isogenic background. Such options
In this environment, clinical laboratories may take several are being explored by some commercial vendors whereby
approaches in the adoption of NGS. These approaches several different mutations are knocked-in by site-directed
include i) laboratories perform the total process within their mutagenesis within isogenic cell lines, and the DNA from

286 - The Journal of Molecular Diagnostics

Methods-Based Prociency Testing

these cell lines, each harboring a different mutation, is principles and practices for validating clinical molecular pathology
mixed. This is an area of active research and development. tests. Arch Pathol Lab Med 2009, 133:743e755
6. Jennings LJ, Smith FA, Halling KC, Persons DL, Kamel-Reid S;
Another area for NGS PT will be to assess the ability of Molecular Oncology Resource Committee of the College of American
laboratories to distinguish disease-causing variants from Pathologists: Design and analytic validation of BCR-ABL1 quantita-
variants that are benign. Numerous variants are found in tive reverse transcription polymerase chain reaction assay for moni-
all persons with any type of NGS testing (gene panels or toring minimal residual disease. Arch Pathol Lab Med 2012, 136:
exome- or genome-level analysis); therefore laboratories 33e40
7. Kamel-Reid S, Zhang T, Persons DL, Nikiforova MN, Halling KC;
must evaluate all variants found in clinically relevant genes Molecular Oncology Resource Committee of the College of American
and need to report on variants that may be causative. Bioin- Pathologists: Validation of KRAS testing for anti-EGFR therapeutic
formatics algorithms reduce the variant list to a set of those decisions for patients with metastatic colorectal carcinoma. Arch
variants that are interpreted as causal or potentially causal Pathol Lab Med 2012, 136:26e32
with respect to patient phenotype. Algorithms are used in a 8. Lacbawan FL, Weck KE, Kant JA, Feldman GL, Schrijver I; Bio-
logical and Molecular Genetic Resource Committee of the College of
series of ltering steps such as variant population frequency, American Pathologists: Verication of performance specications of a
in silico tools for predicting protein function change, and molecular test: cystic brosis carrier testing using the Luminex liquid
clinical variant database cross-referencing for previously bead array. Arch Pathol Lab Med 2012, 136:14e19
reported pathogenicity. In addition, laboratories may need to 9. Pont-Kingdon G, Gedge F, Wooderchak-Donahue W, Schrijver I,
identify and report on variants not directly associated with Weck KE, Kant JA, Oglesbee D, Bayrak-Toydemir P, Lyon E;
Biochemical and Molecular Genetic Resource Committee of the Col-
patient phenotype, but present as incidental ndings, lege of American Pathologists: Design and analytical validation of
depending on the laboratorys policies for analyzing inci- clinical DNA sequencing assays. Arch Pathol Lab Med 2012, 136:
dental ndings. Moreover, with the recent discovery that 41e46
apparently healthy persons harbor approximately 100 or 10. Saxe DF, Persons DL, Wolff DJ, Theil KS; Cytogenetics Resource
more deleterious mutations that cause loss of function within Committee of the College of American Pathologists: Validation of
uorescence in situ hybridization using an analyte-specic reagent for
protein coding regions of genes, the interpretive task is now detection of abnormalities involving the mixed lineage leukemia gene.
even more difcult than previously thought. In the era of Arch Pathol Lab Med 2012, 136:47e52
genomic medicine, interpretation of variants has been iden- 11. Schlaberg R, Mitchell MJ, Taggart EW, She RC; Microbiology
tied as one of the critical bottlenecks for translating Resource Committee of the College of American Pathologists: Veri-
sequence information to clinical practice. Therefore, in the cation of performance specications for a US Food and Drug
Administration-approved molecular microbiology test: Clostridium
future, PT challenges are needed to assess the laboratorys difcile cytotoxin B using the Becton, Dickinson and Company
ability to analyze and interpret known and novel variants. GeneOhm Cdiff assay. Arch Pathol Lab Med 2012, 136:20e25
In conclusion, in the rapidly evolving eld of molecular 12. Jennings LJ, Yu M, Fitzpatrick C, Smith FA: Validation of multiplex
pathology with an exponential increase in data generation, ligation-dependent probe amplication for conrmation of array
MBPT is a rational extension of traditional EQA. MBPT is comparative genomic hybridization. Diagn Mol Pathol 2011, 20:
one helpful tool to assess prociency of molecular diagnostic 13. Maddalena A, Bale S, Das S, Grody W, Richards S; ACMG Labora-
laboratories that provide sequence-based testing and can be tory Quality Assurance Committee: Technical standards and guide-
used in conjunction with analyte-specic testing to ensure lines: molecular genetic testing for ultra-rare disorders. Genet Med
optimal preanalytic, analytic, and postanalytic laboratory 2005, 7:571e583
performance. We consider MBPT to be the most efcient, 14. Ahmad-Nejad P, Dorn-Beineke A, Pfeiffer U, Brade J,
Geilenkeuser WJ, Ramsden S, Pazzagli M, Meumaier M: Methodo-
thorough, practical, and cost-effective method to measure logic European external quality assurance for DNA sequencing: the
laboratory prociency in genome-based sequencing analyses. EQUALseq program. Clin Chem 2006, 52:716e727
15. den Dunnen JT, Antonarakis SE: Mutation nomenclature extensions
and suggestions to describe complex mutations: a discussion. Hum
References Mutat 2000, 15:7e12
16. Richards CS, Bale S, Bellissimo DB, Das S, Grody WW, Hegde MR,
1. Kalman L, Lubin IM, Barker S, du Sart D, Elles R, Grody WW, Lyon E, Ward BE; Molecular Subcommittee of the ACMG Laboratory
Pazzagli M, Richards S, Schrijver I, Zehnbauer B: Current landscape Quality Assurance Committee: ACMG recommendations for standards
and new paradigms of prociency testing for molecular genetics. Arch for interpretation and reporting of sequence variations: Revisions 2007.
Pathol Lab Med 2013, 137:983e988 Genet Med 2008, 10:294e300
2. CLSI: Assessment of Laboratory Tests When Prociency Testing Is 17. Richards CS, Palomaki GE, Lacbawan FL, Lyon E, Feldman GL;
Not Available; Approved Guideline, ed 2. CLSI document GP29eA2: CAP/ACMG Biochemical and Molecular Genetics Resource Com-
Wayne, PA: Clinical and Laboratory Standards Institute; 2008 mittee: Three-year experience of a CAP/ACMG methods-based
3. Mattocks CJ, Morris MA, Matthijs G, Swinnen E, Corveleyn A, external prociency testing program for laboratories offering DNA
Dequeker E, Mller CR, Pratt V, Wallace A; EuroGentest Validation sequencing for rare inherited disorders. Genet Med 2014, 16:25e32
Group: A standardized framework for the validation and verication 18. Stitziel NO, Kiezun A, Sunyaev S: Computational and statistical ap-
of clinical molecular genetic tests. Eur J Hum Genet 2010, 18:1276e1288 proaches to analyzing variants identied by exome sequencing.
4. Halling KC, Schrijver I, Persons DL: Test verication and validation Genome Biol 2011, 12:227
for molecular diagnostic assays. Arch Pathol Lab Med 2012, 136: 19. Coonrod EM, Durtschi JD, Margraf RL, Voelkerding KV: Developing
11e13 genome and exome sequencing for candidate gene identication in
5. Jennings L, Van Deerlin VM, Gulley ML; College of American Pa- inherited disorders: an integrated technical and bioinformatics
thologists Molecular Pathology Resource Committee: Recommended approach. Arch Pathol Lab Med 2013, 137:415e433

The Journal of Molecular Diagnostics - 287