12/13/2011

1
Autoverification in
Clinical Chemistry:
From Theory to Practice
By:
Raffick Bowen, MLT (CSMLS), PhD, DClChem, FCACB, DABCC, MHA (c)
Assistant Professor of Pathology
Associate Director of Chemistry and Immunology Laboratory
Stanford University
December 13, 2011
Webinar- Lab Administration
Disclosure
I have NO financial interests in
any of the commercial products
discussed in this presentation.
2
3
Hillview Special Chemistry lab ~ 5 miles away from Stanford Hospital
SUMC Chemistry Lab Information
Stanford University Hospital (SHC) Lucile Packard Childrens Hospital (LPCH)
~ 456 active beds ~ 218 beds
1500 2500 tubes per day
Total of 5-6 million chemistry tests per year
Clinical Chemistry Section:
AM Shift: 7 CLSs and 4 LTs
PM Shift: 4 CLSs and 2 LTs
MN Shift: 3-4 CLSs and 2-3 LTs
Major instruments:
4 Siemens RxLs 2IMx
3 ABL Blood Gas analyzers 2 TDx/FLx
3 Osmometers
1 Immulite 1000
3
SHC Chemistry Volumes
Tests per year at SHC Chemistry: 5 -6 million
30% LPCH specimens
70% SHC specimens
Tubes: 1500 2500 tubes per day
Urines: 40-50 per day
Misc fluids (e.g. CSF, pleural, peritoneal) 10-20/day
40 60% STAT specimens
At peak times (5am 9am and 3pm to 5pm) it is much higher
65 80% AV
Over 180 tests at SHC chemistry
4
12/13/2011
2
SHC Main Chemistry Panels
5
Panel # Per Month
Met B
~ 15,000 per month
Met C
~ 15,000 per month
Chem 23
~ 1000 1500 per month
Blood gases & co-oximetry
~ 4000 ABGs per month
~ 1700 VBGs per month
~ 200 300 cord BGs per month
Other Notable SHC Chemistry Tests
6
Panel # Per Month
Sweat chloride 35 40 per month
Osmolality 440 per month
Cortisol 40 per month
IOPTH 2 5 patients per month
iCa ~ 4000 per month
TnI ~ 2300 per month
Lactates ~ 1600 per month
Fromthe Institute of Medicine (IOM) :
To Err is Human and Crossing the Quality Chasm report in 1999,
it estimated that of 33.6 million admission to US hospitals in 1997
~44,000 to 98,000 admission die as result of medical errors
Cost: 37.6 to 50 billion for adverse events and 17 to 29 billion for preventable adverse events
A 2005 JAMA article suggests that 5 years after the IOM report there have NOT
been substantial improvements in patient safety.
The IOM report has no explicit information on lab errors
~60-80% of clinical decision is based on laboratory data
Bonini et al (1992):
- most lab errors occur in pre-analytical phase (32-75%) - CPOE
- analytical phase (13-32%)- better instruments
- post-analytical phase (9-31%) interfaced instruments
NPSG 2c-Timely reporting of critical values and critical results
(Kohnet al 1999;Leape and Berwick, 2005)
Patient Safety
7
What is Autoverification?
College of American Pathologists (CAP):
Autoverification is the process by which patient results are
generated from interfaced instruments and sent to the LIS, where
they are compared against laboratory-defined acceptance
parameters.
If the results fall within these defined parameters, the results are
automatically released to patient reporting formats without any
additional laboratory staff intervention.
Any data that fall outside the defined parameters is reviewed by
laboratory staff prior to reporting.
http://www.cap.org/apps/docs/laboratory_accreditation/checklists/laboratory_general_sep07.pdf
8
12/13/2011
3
California State Legislation on
Autoverification
Governor Schwarzenegger signed AB 2156 (Niello) into law,
effective January 1, 2007:
9
California Business and Professions Code Section 1209.5
Items discussed in law:
1. Lab Director or Authorized designee to establish, validate, and
document AV criteria
2. Lab Director or Authorized designee must annually revalidate AV
criteria.
3. Authorized designee must be licensed to work in clinical laboratory
according to CA state regulations.
4. A licensed person must be physically present onsite and held
responsible for accuracy and reliability of results
www.dhs.ca.gov/ps.ls.lfsb
Why Autoverification?
Consistency:
Autoverification removes technologists subjectivity and improves
consistency of reporting (regardless of the number and skill set of the
technologists in the lab)
Quality:
Autoverification reduces errors/mistakes, and improves quality
TAT:
Autoverification reduces amount of labor required for validation of results
About 60-80% of results could be automatically verified, while 20-40%
require further attention.
Holy Trinity of Lab Testing
1. Increase Patient Safety
2. Decrease TAT
3. Cost-Savings
10
How to go about implementing Autoverification?
SUMC Autoverification Team Members:
~ 11-15 people:
LIS personnel
IT personnel
Supervisors
Operation Manager
Reference Technologist
Chemistry Medical Director (Dr. Faix)
Chemistry Associate Director (Dr. Bowen)
CLSs
Hospital Administrators (only for approval of project)
Weekly meetings
8-9 months from planning to implementation for each phase
2 months post autoverification implementation for each phase
Phase I (routine plasma chemistry tests) - Jan 2008
Phase II (DOAs, TDMs, Urine chemistry tests) - Jan 2009
11
What Guideline is Available for Autoverification?
CLSI AUTO 10-A; Volume 26 Number 4
Autoverification of Clinical Laboratory Test Results
Guideline
o Provides general framework that will allow labs to
design, implement, validate and customize rules for
autoverification based on the needs of its own patient
population
o Includes supporting sections that deal with different
aspects of regulatory compliance and validation of
algorithms that are essential to establishing and
maintaining a modern autoverification program
12
12/13/2011
4
13
GEN.43850 Autoverification Approval Phase II
There is a policy signed by the laboratory director approving the use of autoverification procedures.
GEN. 43875 Autoverification Validation Phase II
There is documentation that the autoverification process was validated initially, and is tested at least annually and
whenever there is a change to the system that could affect the autoverification logic.
GEN.43878 Autoverification QC Samples Phase II
For all test results subject to autoverification, the laboratory ensures that applicable quality control samples have
been run within an appropriate time period, with acceptable results.
GEN.43881 Autoverification Result Comparability Phase II
Results are compared with an appropriate range of acceptable values prior to autoverification.
GEN.43884 Result Flags Phase II
Results are checked for flags or warnings prior to autoverification.
GEN.43887 Autoverification Audit Trail Phase II
The audit trail in the computer system identifies all test results that were autoverified, and the date/time of
autoverification.
GEN.43890 Delta Checks Phase I
The autoverification process includes all delta checks that the laboratory performs prior to manual release of test r
results.
GEN.43893 Autoverification Suspension Phase I
The laboratory has a procedure for rapid suspension of autoverification.
CAP Checklists Questions
What to autoverify (SUMC AV Phase I)?
14
Analytes autoverified at SUMC clinical chemistry laboratory
What NOT to Autoverify?
Specimens NOT autoverified in Clinical Chemistry
section of the Clinical Laboratory at Stanford
Beta-hCG (plasma)
Ethanol (need osmolality result)
Tricyclic antidepressants (manual tests)
Body fluids-
e.g. CSF, transudates, exudates, etc.
15
Where to implement AV?
Technology available to implement AV:
Technology Layers:
Instrument layer - RxLs, LX-20s, Centaurs, etc
Middleware layer Data Innovations, Dawning Technologies
e.g. Easylink on Siemens Vista instruments
Centralink on Bayer Advia
LIS layer Sunquest (Misys), Meditech, Soft Lab, etc
Combination(s) of the above
16
At SUMC, we built the AV rules in Sunquest (Misys)
12/13/2011
5
17
LIS
LIS
HIS
EPIC-SHC
Cerner LPCH
RxLs
Interfaced
Orders
Interfaced
Results
Host-Query
Download
Results
Upload
Autoverification in our LIS system (Sunquest)
Dynamic
Download
Streamlab
18
How to deactivate AV?
Procedure for AV deactivation
Function: AF (AV functionality) in
Sunquest system
Downtime Protocol unscheduled
scheduled
Of course, you need AV reactivation protocols
Before AV Testing:
Instrument(s) / Method(s) must be thoroughly
validated
Staff must be trained and competent with analyzer(s)
Calibrations and Quality Control must be acceptable
Adequate staff for AV testing
Awareness of pre-analytical, analytical, and post-
analytical issues in your setting.
LIS, LIS, LIS !!! (knowledge of people and system)
Always expect the unexpected!!
19
2007 Training
AV Timeline for RxLs at SUMC
August to December 7 (Friday)
Testing the AV criteria in Misys
December 10 (Monday)
Completed final documentation of AV testing in Misys
December 11 (Tuesday) to December 31 (Monday)
Training of staff in AV testing in Misys (instrument flags, calculations, HIL)
December 17 (Monday) at 11:00am
GO LIVE with instrument flags, calculations, HIL in Misys (NO
AUTOFILING)
January 8 (Tuesday) at 11:00am 2008
GO LIVE with AV in Misys for phase I
20
12/13/2011
6
21
AV Parameters (RULES)
to be developed and tested
22
Quality Control
The control of the testing process to ensure that test results meet
their quality requirements
QC Westgards Rules:
At SUMC, we do NOT have any quality control rule failures built into our
AV parameters (rules)!
QC review is done manually because:
1. To identify any QC specimens that were not performed
against a QC checklist
2. To monitor trends and shifts on LJ charts
Yes
Yes Yes Yes Yes Yes
IN CONTROL ACCEPT RUN
Control
Data
No
No
No
No No No
OUT-OF-CONTROL REJECT RUN
1
2s
1
3s
2
2s
R
4s
4
is
10-
x
Reference Intervals
Based on patients age, gender, etc
Reference interval at SUMC:
e.g. Sodium: 135-145 mmol/L
What would you do with a sodium of 127 mmol/L ?
At SUMC panic sodium values are < 125 and > 160 mmol/L
23 24
12/13/2011
7
25 26
Most panic results are called by
our Customer Service personnel
Delta Check:
The difference between a patient's present laboratory result and
the previous result which exceeds a predefined limit is referred
to as a delta check
Types of Delta Checks - absolute (e.g. 4 mmol/L for sodium)
- % change (e.g. 10%)
- rate change (e.g. 10% over 7 days)
Delta checks are investigated by the lab internally to rule out:
1) Mislabeling
2) Clerical error
3) Possible analytical error- QNS
Delta Checks: How to determine?
Technologist Experience
Data Review
Clinician Input
Literature
Reference Change Values
27
Calculation of RCV (Significant Change):
Calculation of RCV (Significant Change):
1. Analytical Variation (Imprecision)
2. Biologic Variation
Experimental data
RCV = Z x 2
1/2
x (CV
A
2
+ CV
I
2
)
1/2
Z Probability or Degree of Significance
Z= 2.58 at 99% Probability (Highly Significant)
Z = 1.96 at 95% Probability (Significant)
CV
A
Analytical Variation
CV
I
Biological Variation (within-subject)
Fraser CG, Biologic Variation: FromPrinciples to Practice, AACC Press, 2001
Ricos C, et al., Current databases on biologic variation: pros, cons, and progress. Scand J Clin Lab Invest 1999;59:491-
500. www.westgard.com/guest17.htm
28
12/13/2011
8
29
Instrument Flags:
Instrument physical status flags
30
Examples of RxL Instrument Flags and Errors Criteria
Flag Description
Cup or Test
Level Failures
AL Above linearity Test Level
BM Breaker/mixer slipped Cup Level
DE Decode Cup Level
MP Missing pack Cup Level
PL Pump lost steps Cup Level
TE Temperature Cup Level
AB Absorbance Test Level
BL Below linearity Test Level
31 32
Misys/SQ Code
(RXL Code)
Calculation Auto Verification
Impact
AGAP (CALC) Na (Cl +CO2) Results <2 or > 30 will
FAIL auto verification
for the CUP
LDL (CALC) Total Cholesterol (HDL -
TRIG/5)
Results for TG >400
mg/dL will FAIL auto
verification for the
CUP
ALB (CALC) Albumin > Total Protein FAIL auto verification
for the CUP
CK-MB (CALC) CK-MB > Total CK FAIL auto verification
for the CUP
IBIL (CALC) IBIL, DBIL > TBIL FAIL auto verification
for the CUP
Example of Calculation Flags:
12/13/2011
9
HIL Indices Flags:
Examples of Hemolysis, Icterus, and Lipemia messages
Serum Indices Range of HIL
Results
Test codes with
Appended Comments
Code to append
4 (any H value 4) HIL value >410 K , MG HEMIN
33
Serum Indices Range of HIL
Results
Test codes with
Appended Comments
Code to append
4 (any I value 4) 141-146 CHOL, CK, GL, TP ICTDE
Serum Indices Range of HIL
Results
Test codes with
Appended Comments
Code to append
5 (any H value 5) 115-116 GL, PHOS TURIN
Hemolysis (HIL) Related Comments
Icterus (HIL) Related Comments
Lipemia (HIL) Related Comments
LIS automatically appends the appropriate comment(s) with results
(405 nm)
(452 nm)
(700 nm)
34
HIL Comments
(0-1, Slight; 2-4, Moderate; 5-6, Gross)
HEMOLYSIS ICTERUS LIPEMIA
HEMDE: Hemolysis ICTDE: Icteric specimen TURDE: Turbidity present
-may tend to decrease result -may tend to decrease result -may tend to decrease
result
HEMIN: Hemolysis present ICTIN: Icteric specimen TURIN: Turbidity present
-may tend to increase result -may tend to increase result -may tend to increase
result
35
Electronic Simulated Data Validation: Clinical Specimens:
36
12/13/2011
10
Criteria for Autoverifying Specimen
Check to see if calibrators / QC has been run and within
acceptable limits
Check to ensure that there are no instrument errors /
flags
Results are within instrument / AV range
-the results should be held for manual review when the
results are outside the range
Not a critical value
-those results that need immediate attention
Passed delta check criteria
-absolute, percent change, rate change
No calculation errors (e.g. anion gap)
37
(We have TEN 3 inch binders FULL of AV validation documentation)
38
Failed AV (K < 3.0 mmol/L)
Passed AV
39
ED Basic Metabolic Panel (Met B)
40
50
45
41 41 41
35
32
34 34
32 32 32
31
29 29
32
30
29 29
34
28
26
29
31
29
28
31 31
32 32
33
34
35
33
30
37
38
36 36
18
20
12 12 12
10 10
11
12
10
9
9
10
7 8
9
7 8
7
11
7
6 7
7 7 7
8 8 8 8
9
12
8
10
8
11
10 10
8
34
33 33 33
30
26
24
25
25 24
23
26
23 23
24
25 25
25 24
25
23
22
23
26
23 23
26
25
27
26
27
28
29
26
24
30
31
29 29
0
10
20
30
40
50
60
M
I
N
U
T
E
S
,

9
0
t
h

P
e
r
c
e
n
t
i
l
e
Receipt to Verify Receipt to Bench
Bench to Verify Target Receipt to Verify
Target
Streamlab installation project
12/13/2011
11
90
th
percentile ED TAT (CAP Q-Probe Studies)
41
Met B
Met C
Chem. 23
surrogates
CBC
surrogate
42
Total
Specimens
Total
Autofiled
Percent
Autofiled
K 22409 19035 85%
Met B 10190 8711 85%
Met C 9572 8075 84%
TnI 1077 724 67%
Total
Specimens
Total
Autofiled
Percent
Autofiled
K 6374 5371 84%
Met B 1585 1284 81%
Met C 3795 3219 85%
TnI 386 293 76%
Total
Specimens
Total
Autofiled
Percent
Autofiled
K 28783 24406 85%
Met B 11775 9995 85%
Met C 13367 11294 85%
TnI 1463 1017 70%
Routine specimens from SHC and LPCH
October 2011
STAT specimens from SHC and LPCH
October 2011
Total Autofiled
(Routine and STAT specimens)
October 2011
Improve quality of laboratory results
HIL:
- Comments appear with patient results
Delta checks:
- helps identify mislabeled specimens (e.g. sodium +/- 8 mmol/L)
AV range:
- set a AV range to help identify clotted specimens or QNS
e.g. ALP: 5 - 3500 U/L, if < 5 U/L, check specimen
Added comments to CLS:
Check Printout for calculation errors or Repeat Lytes for low anion gaps (< 2)
Standardize reporting of results like anion gaps (<2 instead of
actual number like -15)
Instrument Flags:
on screen, no need to look at instrument printout
43
Some Benefits of Implementing AV in the Chemistry Section of
the Clinical Laboratories at Stanford
o Less phone calls (from 25-30 to <5 per shift); less complaints
o Pending log has been reduced from 15-20 pages to 3-6 pages per shift (20-30
specimens per page)
o Increase staff morale and less fatigue and stress; less sick calls
o Decrease TAT of chemistry results; meeting our TAT goals!
o More organized workflow Standardize SOPs
o More consistent reporting of results-decrease errors and improve quality
o Dynamic screen has less specimens to track
o Less overtime ~ 30 mins/day (7 hours per pay period)
o Identification lab staff who needed some re-education on SOP, policies, etc
44
12/13/2011
12
$$$ COST SAVINGS $$$
1 FTE saved :
2 Chemistry Verifiers Down to 1
The 1 FTE moved to Sweat Chloride bench
and cortisol and IO-PTH bench
We now offer sweat testing everyday
- this was NOT a budgeted position.
45 46
-Chart review (quarterly):
- results (RxL to LIS to HIS)
- check AV rules
- calculations
- HIL comments
Monitoring AV
January: CrCl, AGAP, Globulin, IBIL, BUN/CR, LIPID
April: Urine ratios and excretion
July : TRFSN, eGFR, Delta checks, HIL
October: CK-MB relative index, Osmolal gap, Instrument errors
RxL LIS HIS
47
1. Duca DJ. Autoverification in a laboratory information system. Lab Medicine 2002;33:21-5.
2. Pearlman ES, Bilello L, Stauffer J, Kamarinos A, Miele R, Wolfert MS. Implications of
autoverification for the clinical laboratory. Clin Leadersh Manag Rev 2002;16:237-239
3. Crolla LJ, Westgard JO. Evaluation of rule-based autoverification protocols. Clin Leadersh
Manag Rev. 2003;17:268-72.
4. Clinical and Laboratory Standards Institute (CLSI). Autoverification of Clinical Laboratory
Test Results, Approved Guidelines (AUTO 10-A) Wayne, PA: Clinical and Laboratory
Standards Institute, 2006; Vol 26, No 32.
5. College of American Pathologists Commission on Laboratory Accreditation, Laboratory
General Checklist, 2009.
6. Torke, N. Process Improvement and Operational Efficiency through Test Result
Autoverification. Clin Chem 2005; 51: 2406-2408.
7. Biological Variation: From Principles to Practice. CallumG. Fraser. Washington, DC:
AACC Press, 2001.
8. FraserCG, Stevenson HP, Kennedy IMG. Biological variation data are necessary
prerequisites for objective autoverification of clinical laboratory data. Accred Qual Assur
2002;7: 455-460
Some References:
Any Questions?
Email:
rbowen@stanfordmed.org
48