Clin Chem Lab Med 2015; 53(6): 887891

Rainer Haeckel*, Werner Wosniok and Thomas Streichert

Optimizing the use of the state-of-the-art

performance criteria
DOI 10.1515/cclm-2014-1201
Received December 5, 2014; accepted February 9, 2015; previously
published online March 20, 2015

Abstract: The organizers of the first EFLM Strategic Conference Defining analytical performance goals identified three models for defining analytical performance
goals in laboratory medicine. Whereas the highest level
of model 1 (outcome studies) is difficult to implement,
the other levels are more or less based on subjective
opinions of experts, with models 2 (based on biological
variation) and 3 (defined by the state-of-the-art) being
more objective. A working group of the German Society
of Clinical Chemistry and Laboratory Medicine (DGKL)
proposes a combination of models 2 and 3 to overcome
some disadvantages inherent to both models. In the
new model, the permissible imprecision is not defined
as a constant proportion of biological variation but by
a non-linear relationship between permissible analytical
and biological variation. Furthermore, the permissible
imprecision is referred to the target quantity value. The
biological variation is derived from the reference interval, if appropriate, after logarithmic transformation of
the reference limits.
Keywords: biological variation; permissible uncertainty;
reference intervals.

Introduction
The organizers of the first EFLM Strategic Conference
Defining analytical performance goals identified three
models for defining analytical performance goals in laboratory medicine. Whereas the highest level of model
*Corresponding author: Rainer Haeckel, Bremer Zentrum
fr Laboratoriumsmedizin, Klinikum Bremen Mitte,
28305Bremen,Germany, Phone: +49 412 273448,
E-mail: rainer.haeckel@t-online.de
Werner Wosniok: Institut fr Statistik, Universitt Bremen, Bremen,
Germany
Thomas Streichert: Institut fr Klinische Chemie, Universittsklinik
Kln, Kln, Germany

1 (outcome studies) is difficult to implement, the other

levels are more or less based on subjective opinions of
experts, with models 2 (based on biological variation) and
3 (defined by the state-of-the art) being more objective.
The benefits and disadvantages of models 2 and 3 have
already been discussed elsewhere [1, 2].
Problems with the biological variation model are the
large variability between studies (e.g., 2.3%31.9% for
triglycerides [3]), data are often generated from relatively
young and healthy subjects, dependence on the time span
studied (hours to years), and possible effects of measurand concentrations. Owing to the great diversity of literature reports, many authors consider biological variation
not suited to set metrological requirements. Problems with
the state-of-the-art model are lack of scientific reasoning,
often based on old data, which may be outdated, lack
of transparency, lack of neutrality (dependency on industry), and the lack of relationship between what is achievable and what is needed clinically.
A working group of the German Society of Clinical
Chemistry and Laboratory Medicine (DGKL) has developed a combination of models 2 and 3 to overcome some
disadvantages inherent to both models [2].

New concept for defining permissible

performance criteria
Three types of biological variation (CVB) have been
described: (1) intra-individual variation (CVI), (2) inter-individual variation (CVG), and (3) combined CVB (combined
CVI and CVG). As a surrogate for the biological variation [2],
the empirical (biological) coefficient of variation (CVE) has
been proposed [1, 2]. According to Simundic etal. [4], CVE
could be termed CVA+I+G.
The empirical biological variation CVE derived from
the reference interval (RI) was proposed because laboratories are obliged to provide RI for all measurands, must
check their transferability (if taken from external sources),
and should review the RI periodically as required by ISO
15189 [5]. Furthermore, reference limits (RL) usually vary
less than data on biological variation in the literature and
often are from consensus groups.

Unauthenticated
Download Date | 5/19/15 5:06 AM

888Haeckel etal.: Optimizing the state-of-the-art performance criteria

sE ,ln = (lnRL2 lnRL1 ) / 3.92. (1)

Then, the CVE of a logarithmic normal distribution

expressed on the linear scale (CVE*) can be calculated from
Eq. (1), as explained in the Appendix:

CVE = (exp(sE,ln2 ) 1) 0.5 100. (2)

CVE* can be considered as a surrogate for the conventional biological variation (CVB), although it also contains the analytical variation. The CVE* values were
compared (Figure 1) with the combined CVB values
[CVB=(CVI2+CVG2)0.5] for 64 quantities (in blood, serum,
and plasma) listed in the RiliBK [8]. The CVB data were
taken from Ricos et al. [7]. Although this list is already
very comprehensive, CVB data are available for only 80%
of the RiliBK measurands. As shown in Figure 1, CVB and
CVE* correlate quite well. The greatest divergences (above
the regression line=black line in Figure 1) are obtained
with C-reactive protein, CA 19-9, and prostate-specific
antigen (PSA). In the Ricos etal. [7] list, a relatively high
CVG value of 130% is presented for CA 19-9. Erden et al.
[9] more recently published a CVG=64.2%, a value that
appears more realistic. In Table 1, the intra-individual
biological variation (CVI) of PSA is compared from several
reports. The most comprehensive survey was published

140

CA 19-9

120

CVB, Ricos et al.

RLs reflect the biological variations, including the

analytical variation. If values are normally distributed, the
empirical (biological) standard deviation, sE, can be estimated by (upper RLlower RL)/3.92 [2]. However, a true
empirical normal distribution does not exist in laboratory
medicine. For small reference ranges and relatively high
mean values (e.g., sodium and chloride concentrations
in plasma or hematological quantities), the distribution
usually looks close to normal, but a log-normal distribution has the same shape under these conditions. For
relatively large reference ranges (e.g., thyreotropin, triglycerides, enzymes in plasma), it is obvious that the data
are not normally distributed. Generally, it may be assumed
that laboratory data follow a power normal distribution,
i.e., (x1)/ has a normal distribution, where the coefficient controls the shape of the distribution. Important
special cases are =1, indicating a normal distribution,
and =0, the lognormal distribution. The value for a
specific data set can be estimated by numerical methods.
If is unknown and cannot be estimated from data,
e.g., because only RLs are given, but there are no individual values, the assumption of a logarithmic distribution
was recommended [6]. Assuming a lognormal distribution, the empirical standard deviation (sE) is calculated
from the lower reference limit (RL1) and the upper reference limit (RL2) by Eq. (1), as recently explained [2]:

Troponin

100

CRP

80

PSA

60
40

Estradiol

20
0

10

-20

20

30

40

50

60

70

CVE*

Figure 1:Comparison of CVE* values with combined CVB values

taken from Ricos etal. [7].
PSA, prostate-specific antigen; CRP, C-reactive protein.

Table 1:The intra-individual variability of plasma PSA reported by

several researchers.

CVIa

CVG

Sletormos etal. [10] (survey of 13 studies)

2.122.9
Further reports (not mentioned by Sletormos etal.)

Ricos etal. [7]

18.1 72.4
Fraser [11]

14.0 72.4
Deijter etal. [12]

17.6
Erden etal. [13]

20.0
Schifman etal. [14]

6.2
Gurr etal. [15]

7.0
CVE*=52.5, CVB=74.6 [7]. aCVI, intra-individual biological variation;
CVG, inter-individual biological variation.

by Sletormos etal. [10] on behalf of the European Group

on Tumor Markers. The CVI values from 13 studies varied
between 2.1% and 22.9%. These large differences between
studies make a choice of the correct CVB value difficult.
Ricos etal. [7] selected a CVI=18.1, whereas a CVI of about
13% (close to the mean of the reported span) would correlate with the CVE* quite well.
The empirical biological variation CVE* derived from
the reference range can be applied to derive permissible
analytical uncertainty and to determine quantity quotients standardizing reporting laboratory results.

Proposal for determining permissible

analytical uncertainty
Various approaches have been presented to define
permissible analytical uncertainty as a proportion of

Unauthenticated
Download Date | 5/19/15 5:06 AM

Haeckel etal.: Optimizing the state-of-the-art performance criteria889

pCVA = (CVE 0.25 ) 0.5 . (3)

This equation approximately describes the empirical relation between CVE* and the major part of the presently used
permissible imprecision values. Eq. (3) describes a curved
relation between permissible analytical CV (pCVA) and
CVE*. It can be easily adapted to future technical improvements by modifying its parameters. The exponent in Eq.
(3) may be reduced from 0.5 to, e.g., 0.45 if one wishes
more stringent permissible limits.
An algorithm to estimate pCVA values for a single
target or measured values has been recently proposed [2].

Permissible limits for combined and

expanded measurement uncertainty
Three types of measurement uncertainty supported by
many international organizations (BIPM, IEC, IFCC,
ISO, IUPAC, IUPAP, and OIML) are described in the
Guide to the Expression of Uncertainty in Measurement
[17]: standard uncertainty uS (imprecision, standard
deviation), combined uncertainty uC (u12+u22+u32)0.5, and
expanded uncertainty U=kuC (if coverage factor k=1.96,
the level of confidence is 95%). In laboratory medicine,
at least two components have to be combined, imprecision and bias.
The permissible uncertainty (of measurement) can be
given by Eq. (3), e.g., u1 in uC. For permissible bias (pB)
(e.g., u2 in uC), 0.5pCVA was previously suggested [2].
According to the GUM concept [17], an uncertainty component should be added for which also 0.5pCVA was proposed [2]:

25

Testosterone

20
pU% , RMSD

biological variation. Cotlove etal. [16] proposed to multiply CVB with a constant factor of 0.5. Fraser [11] suggested
a three-class model with factors 0.25, 0.5, and 0.75, and
Haeckel and Wosniok [1] suggested a five-class model. All
models use more or less arbitrary factors. If permissible
limits derived of the present state-of-the-art [root-meansquare of measurement deviation (RMSD) in Figure 2] are
compared with biological variation data (CVE*), a curved
relationship may be observed (analog to the curved line
in Figure 2). With the Cotlove etal. [16] model (analog to
the straight line in Figure 2), too stringent requirements
are postulated, which can hardly be reached by the
present technology for measurands with relatively small
CVE* values. For measurands with larger biological variation, the requirements may be too permissive. The curved
relationship for the present technology can be simply
described by Eq. (3):

15
10
5
0
0

20

40

CVE*

60

80

100

Figure 2:Comparison of empirical biological variation (CVE*) with

permissible analytical variation derived of the present state of the
art (technical feasibility).
RMSD (circles), root-mean-square measurement deviation (column
3 of Table 2a of the German Guideline 2008); straight line, 0.5CVE*
line; diamonds, present proposal.

pB=(0.5 pCVA 2 +0.5 pCVA 2 ) 0.5 = 0.7 pCVA . (4)

Then, the permissible limit of uC is (pCVA2+pB2)0.5:

uC = 1.22 pCVA .
Considering a 95% probability of uC, the expanded
uncertainty can be calculated by Eq. (5):

pU% = 1.96 puC = 2.39 pCVA , (5)

where pU% corresponds to the RMSD value of the RiliBK

2008 [8, column 3 in Table B1a]. The pU% in relation to
CVE* values were compared with the RMSD for several
measurands (Figure 2). Mean limits for 64 plasma quantities were about 5% lower than those required by the new
RiliBK [8]. The straight line represents the 0.5 multiplication factor for the biological variation as proposed by
Fraser [11]. The problem with this more or less arbitrary
classification is the difficulty to which a single measurand
may be attributed. The expanded uncertainty also leads to
a curved relation with CVE* (like pCVA vs. CVE*).

Permissible limits for ring trials

(EQAS)
Recently, we have proposed that permissible limits in
external quality assessment schemes (EQAS) may be
derived of pCVA, respectively, of pU% values [18]. Analogous to this approach, the 90% interval of the permissible
limits for EQAS may be

pU EQAS % = 1.64 pU% = 3.92 pCVA (6)

Unauthenticated
Download Date | 5/19/15 5:06 AM

890Haeckel etal.: Optimizing the state-of-the-art performance criteria

2. The mean on the logarithmic scale is

and the 95% interval may be

pU EQAS % = 1.96 pU% = 4.68 pCVA . (7)

Examples for the various permissible uncertainties

were recently published [2]. Mean limits for 64 plasma
quantities were similar to those required by the new
RiliBK [8].

Discussion
The empirical (biological) variation (CVE ) derived from
the reference range is suggested as a surrogate for the biological variation. RLs are available to all measurands, and
most probably, the laboratories have more experience with
these data because they have to validate them before their
introduction in the diagnostic service and then to verify
them periodically according to good laboratory practice
[5]. CVE* values can be used to derive permissible uncertainty by algorithms that may reconcile the presently competing biological variation model and the state-of-the-art
model.
Although CVE values correlate quite well with biological variation data (Figure 1), they combine intra- and interindividual variations (CVC). The relation between both is
not constant. The better choice would be to use only CVI
data. Presently, however, these data vary considerably in
the literature (see example presented in Table 1). Therefore, several authors refused to use CVI to derive permissible performance criteria [1, 2, 19]. The application of CVE
data may be an acceptable compromise until more reliable
data on intra-individual variations are available. Furthermore, CVE data vary between laboratories.
The working group Guide limits of the DGKL developed easily to handle Excel tools for the estimation of RIs
from intra-laboratory data pools and the estimation of the
permissible uncertainty. These tools are distributed gratuitously, e.g., via website of the DGKL [20] or from the
authors and could be implemented by software companies in their information systems.
*

Appendix
Estimation of CVE*
1. Assuming a logarithmic normal distribution for the
measurand, the biological standard deviation derived
of the RLs is on the logarithmic scale
sE ,ln = (lnRL2 lnRL1 ) / 3.92.

meanln = (lnRL2 + lnRL1 ) / 2.

3. The corresponding mean on the linear scale is [21]
meanlin = exp(meanln + 0.5 sE ,ln2 ).
 owever, this mean is not needed to calculate CVE*, as
H
is shown below.
4. sE,ln can be transformed to the linear scale according
to Aitchison and Brown [21]:
sE ,lin = meanlin (exp(sE,ln2 ) 1) 0.5 .
5. CVE*=sE,lin100/meanlin.
6. CVE*=meanlin(exp sE,ln21)0.5100/meanlin.
7. CVE*=(exp(sln2)1)0.5100.(2)
Author contributions: All the authors have accepted
responsibility for the entire content of this submitted
manuscript and approved submission.
Financial support: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played
no role in the study design; in the collection, analysis, and
interpretation of data; in the writing of the report; or in the
decision to submit the report for publication.

References
1. Haeckel R, Wosniok W. A new concept to determine permissible
imprecision and bias on the basis of biological variation and
state-of-the-art. Clin Chem Lab Med 2011;49:62335.
2. Haeckel R, Wosniok W, Gurr E, Peil B. Permissible limits for
uncertainty of measurement in laboratory medicine. Clin Chem
Lab Med 2015. doi: 10.1515/cclm-2014-0874 [Epub ahead of print
23 Jan 2015].
3. Smith SJ, Cooper GR, Myers GL, Sampson EJ. Biological variability
in concentrations of serum-lipids: sources of variation among
results from published studies and composite predicted values.
Clin Chem 1993;39:101222.
4. Simundic AM, Kackov S, Miler M, Fraser CG, Hylthoft Petersen P.
Terms and symbols used in studies on biological variation: the
need for harmonization. Clin Chem 2015;61:12.
5. International Standard Medical laboratories particular requirements for quality and competence, ISO 15189-2012.
6. Haeckel R, Wosniok W. Observed, unknown distributions of
clinical chemical quantities should be considered to be lognormal: a proposal. Clin Chem Lab Med 2010;48:13936.
7. Ricos C, etal. Available from: www.westgard.com. Accessed 2014.
8. Richtlinie der Bundesaerztekammer zur Qualittssicherung
laboratoriumsmedizinischer Untersuchungen. Dt Aerzteblatt 2008;105:C30113. Available from: www.aerzteblatt.de/
plus1308.

Unauthenticated
Download Date | 5/19/15 5:06 AM

Haeckel etal.: Optimizing the state-of-the-art performance criteria891

9. Erden G, Barazi AO, Tezcan G, Yildirimkaya MM. Biological
variation and reference change values of CA 19-9, CEA and
AFP in serum of healthy individuals. Scand J Clin Lab Invest
2008;68:2128.
10. Sletormos G, Semjonow A, Sibley PE, Lamerz R,
HylthoftPetersen P, Albrecht W, etal. Biological variation oftotal
prostate-specific antigen. A survey of published estimates and
consequences for clinical practice. Clin Chem 2005;51:134251.
11. Fraser CG. Biological variation: from principles to practice.
Washington, DC: AACC Press, 2001:1151.
12. Deijter SW, Martin JS, McPherson RA, Lynch JH. Daily v ariability
in human serum prostate-specific antigen and prostatic
acidphosphatase: a comparative evaluation. Urology
1988;32:28892.
13. Erden G, zden T, Soydas A, Yildirimkaya MM. Biological variation and reference change value (RCV) of prostate specific
antigen (PSA) levels in the serum of healthy young individuals.
Gazi Med J 2009;20:1526.
14. Schifman RB, Ahmann FR, Elvick A, Ahmann M, Coulis K,
BrawerMK. Analytical and physiological characteristics of
prostate specific antigen and prostatic phosphatase in serum
compared. Clin Chem 1987;33:20868.

15. Gurr E, Krnig F, Golbeck A, Arzideh F. Limits for the determination of guiding values from intralaboratory data basis,
demonstrated by prostate specific antigen (PSA). J Lab Med
2009;33:6770.
16. Cotlove E, Harris EK, Williams GZ. Biological and analytic components of variation in long-term studies of serum constituents
in normal subjects. Clin Chem 1970;16:102832.
17. ISO (International Organisation for Standardisation). ISO/IEC
Guide 98-3:2008 Uncertainty of measurement. Part 3: guide
to the expression of uncertainty in measurement (GUM:1995).
Genf: ISO, ISBN 92-67-10188-9.
18. Haeckel R, Wosniok W, Kratochvila J, Carobene A. A pragmatic approach for permissible limits in external assessment
schemes with a compromise between biological variation and
the state of the art. Clin Chem Lab Med 2012;50:8339.
19. Fuentes-Arderiu X. Variability of the biological variation. Scand J
Clin Lab Invest 2002;62:5614.
20. Permissible imprecision (pCVA) and combined uncertainty (pU%)
for a particular measurand (xi). Available from: www.dgkl.de.
Accessed on 10 January, 2014.
21. Aitchison J, Brown JA. The lognormal distribution. Cambridge:
Cambridge University Press 1969:1176.

Unauthenticated
Download Date | 5/19/15 5:06 AM