Вы находитесь на странице: 1из 10

Journal of Medical Imaging

and Radiation Sciences

Journal de l’imagerie médicale


et des sciences de la radiation
Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203
www.elsevier.com/locate/jmir
Review Article

Exploring the Benefits of Magnetic Resonance Imaging Reporting by


Radiographers: A UK Perspective
Paul Lockwood, MSc, FHEA, PGCL&T(HE), BSc(Hons), BA(Hons)*
Allied Health Department, Canterbury Christ Church University, Medway Campus, Chatham Maritime, UK

ABSTRACT RESUME 
Background: The United Kingdom (UK) National Health Service Contexte : Le Rapport sur les activites d’imagerie et de radiodiagnos-
(NHS) Imaging and Radiodiagnostic activity 2013/14 report esti- tic 2013-2014 du National Health Service (NHS) du Royaume-Uni
mates the year-on-year increase of magnetic resonance imaging estime que l’augmentation annuelle des examens d’imagerie par
(MRI) examinations to be 12.3%, with the designated radiologist resonance magnetique (IRM) a ete de 12,3 %, avec une dispropor-
workforce disproportionate to the increase in demand. tion entre le nombre de radiologistes designes et l’augmentation de
la demande en rapports d’imagerie.
Objective: To review the economics, risk, and feasibility of MRI re-
porting by radiographers. Objectif : Examiner l’aspect economique, le risque et la faisabilite de
faire preparer les rapports d’IRM par les radiographes.
Design: A PICO (the four major components of a clinical or
research question: patient [population], intervention, comparison, Conception : Un cadre PICR (les quatre elements d’une question
and outcome) framework using example patient demand from clinique ou d’une recherche: patient (population), intervention, com-
audit data of noncomplex MRI examination attendance (n ¼ paraison et resultat) utilisant la demande des patients tiree des
3,525) over 12 months was used to review costs, potential outcome donnees de verification de presence a un examen d’IRM non com-
risks (diagnostic thresholds), and feasibility (workforce capacity) of plexe (n ¼ 3 525) sur une periode de 12 mois, avec un examen
both interventions. des co^uts, des risques potentiels pour les resultats (seuil de diagnostic)
et de la faisabilite (capacite des effectifs) pour les deux interventions.
Conclusions: The benefits of introducing a skills mix reporting ser-
vice model to the benefit of service delivery in the UK has shown a Conclusions : Les avantages liesa l’introduction d’un service de rapport
potential £145,230–£60,524 per annum cost saving using a generic en competences mixtes pour la prestation des services au Royaume-Uni
acute workload model. Research into recorded discrepancy/error montrent des economies annuelles potentielles de l’ordre de £145 230 -
audit data for potential detrimental risk to patient outcomes identi- £60 524 par annee avec l’utilisation d’un modele generique de charge de
fied a paucity of evidence and recommends that further research is travail aigu€e. Les recherches sur les donnees de verifications des ecarts ou
needed. erreurs enregistrees afin d’etablir le risque potentiel pour les patients ont
permis de constater en manque de donnees probantes, et il est re-
commande de poursuivre les recherches sur cet aspect.

Keywords: Magnetic resonance imaging; reporting radiographer; role extension; advanced practice; economic evaluation

Introduction satisfaction, and decrease service delays, with an emphasis on


investment for local service changes. The English Cancer
The National Health Service (NHS) England released the Five
Strategy 2015–20 [2] followed in 2015 and embraced the
Year Forward View [1] in 2014 to consider possible changes
Five Year Forward View’s [1] three main aims of better preven-
that could be implemented to improve patient outcomes and
tion, swifter diagnosis, and better treatment. The Royal College
The author has no financial disclosures or conflicts of interest to declare. of Radiologists [3] (RCR) endorsed the strategy but insisted the
* Corresponding author: Paul Lockwood, MSc, FHEA, PGCL&T(HE), plan to improve access to scans and reports requires a change in
BSc(Hons), BA(Hons), Allied Health Department, Canterbury Christ Church diagnostic capacity and an increase in radiology staffing.
University, Medway Campus, Rowan Williams Building, 30 Pembr. Chatham
Maritime, ME4 4UF, UK.
The Kings Fund Better Value in the NHS 2015 [4] report
E-mail address: paul.lockwood@canterbury.ac.uk called on NHS staff to engage in delivering better outcomes

1939-8654/$ - see front matter Ó 2016 Canadian Association of Medical Radiation Technologists. Published by Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jmir.2015.12.083
by improving value rather than reducing the costs, in the wake The study used data from a retrospective audit of MRI exam-
of the Five Year Forward View [1] that proposed £22 billion ination attendance at an acute NHS district general hospital
of efficiency savings. The report findings emphasised the need (DGH). A defined time horizon of 12 months (August
to create an environment for change and highlighted major 2014–July 2015; Tables 1 and 2) was used to identify the
service areas where restructuring diagnostic pathways could key resource demand for MRI examinations (n ¼ 12,958).
deliver cost-effective service improvements, increase the speed Using decision tree modelling to illustrate the process
of delivery, reduce length of stay in hospital, and fast-track mapping of the current intervention (Table 3) allowed evalu-
treatment and management for preventable illness. ation of costs and outcomes from each intervention for inter-
Two specific driving factors for change in radiology have nal validity. Using the audit data allowed external validation
been a flexible response to workforce shortages [5–10], and of the model as an example of expected workflow demand
demand for imaging that outstrips capacity [11–13]. The in a generic DGH. A decision tree was chosen over conven-
NHS Imaging and Radiodiagnostic activity 2013/14 report tional Markov models because data for chronic returning pa-
[11] assessed the number of magnetic resonance imaging tients were not available to consider all feasible transitions of
(MRI) examinations from April 2013 to March 2014 to be patient’s health states or cohorts of particular disease categor-
2.7 million, with a 12.3% increase in examinations from ised patients.
the previous year [11], 71.7% over 5 years [14], and 220%
Patient Group
growth over a 10-year period [11], which is a substantial in-
crease in the pattern and trend of imaging demand. Both The sample size from the data collection identified n ¼
the RCR [13] and the Society and College of Radiographers 3,525 noncomplex MRI scans (Table 2). The inclusion
[13] (SCoR) observe that demand in imaging is expected to criteria included knee, lumbar, internal auditory meatus
intensify. The Centre for Workforce Intelligence (CfWI) (IAMs), scaphoid, and breast. The noncomplex examination
[12] predicts the demand for imaging will escalate due to criteria limitations were due to the restricted literature evi-
multiple factors, including growing and/or aging populations, dence available on reported diagnostic thresholds of reporting
rise in cancer diagnosis and chronic illness, screening pro- radiographers and radiologists in MRI reporting.
grammes, introduction of 24/7 working hours, and future im-
The Current Intervention
aging techniques introduced into clinical practice. The CfWI
and the RCR have estimated the total imaging workload The NHS currently employs radiologists to report MRI
could potentially rise from 39 million tests in 2011 to 51 examinations, but drivers for change include the low work-
million by 2025 (an increase of all imaging by 76%, with force numbers of UK registered radiologists [14]. The fifth
MRI alone raising 87%) [12]. RCR workforce report 2012 [14] recorded the number of
The RCR [15] recommend a formal report for diagnostic UK registered radiologists as 2,997 (4.7 radiologists per
examinations within 2 days, but acknowledge that due to 100,000 population); with a current deficit of 421 vacant
workforce shortages, this is not occurring [14]. This causes de- posts [3]. To reach comparable radiologist levels with the
lays in cancer and serious illness diagnosis, hospital stay, and rest of the European Union countries, the RCR estimated it
the subsequent increased listing of radiology departments to would require an 82% increase of radiologists [12].
NHS risk registers [15]. In October 2014, an RCR survey The CfWI report on Clinical Radiology [12] commis-
[16] highlighted a month delay in results (1,697 examina- sioned by the Department of Health (DoH) with multiple
tions) in the 25% of NHS trusts surveyed. The survey was stakeholders, including the RCR and SCoR, reviewed the
repeated in February 2015 [15] with 71% of surveyed trusts RCR 2012 [13] report for the Medical Programme Board
having delays of more than a month, with over 3,277 unre- and the Joint Working Group on Speciality Training
ported MRI scans (estimated for all trusts in England to be Numbers. Recommendations included a proposed, but not
up to 4,268 [15]). implemented, increase of 60 trainee radiology registrars per
year, with the use of radiographers to effectively support the
Methodology
future expansion of radiology.
To define the perspective of the review and the key drivers of
Unit Costs and Discounting
cost-effectiveness (capacity and demand, benefits, and risks) a
PICO framework [17] was adopted. A PICO has four major To ascertain an average hourly price for radiologists, Net-
components, in this case consisting of P for the patient sample ten et al’s Ready Reckoner for staff costs in the NHS [18] and
group defined by the MRI imaging pathway; I for the intervention the Personal Social Services Research Unit (PSSRU) Unit
of radiographers reporting MRI examinations; C for the compar- Costs of Health and Social Care 2014 [19] were adopted
ison to existing intervention of radiologists reporting MRI for the basis of the calculations. The salary was based on a
examinations; and O for the outcome comparison of current full time equivalent (FTE) mean of NHS medical consultant
and alternative service provision through costs, savings, and risk. wages [19]. An additional 33.5% was added for overtime,
This review received university research ethical and gover- shift work and geographic allowances [19], National Insur-
nance approval to calculate a deterministic, scenario-based ance contributions (NI) [20], and employer’s contribution
evaluation of costs for the current and new intervention. to superannuation [21]. The costs for education and training

P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203 195


Table 1
Audit Results of MRI Demand at an Average-Sized Generic DGH (2014–2015)

used PSSRU [19] standard estimation approaches for the and radiologists working in complementary reporting roles
components of training, tuition fees, clinical placement costs, (not substitution or replacement of roles) to sustain service de-
infrastructure (books, journals, and computers), and lost pro- livery. The SCoR scope of practice [27,28] legally entitles UK
duction costs of staff training days. radiographers with accredited training and competence to
The costs incorporated the discounting system used by report MRI examinations. The CfWI [29] have predicted
PSSRU [19] and Her Majesty’s Treasury [22] to transfer all an increase of 17% (to 19,830) of radiographers from 2012
costs and benefits to ‘‘present values’’ to compare, using a to 2016. Currently, the Health and Care Professions Council
3.5% discount rate. This allowed a net present value of the (HCPC) [30] have 29,711 radiographers registered (3,100 are
intervention to be calculated, which is the primary indicator therapeutic radiographers [31]), which is above the projected
used by the UK government to justify action. Furthermore, increase of workforce by the CfWI [29]. In addition, Health
this is the approved system in use by the National Institute Education England (HEE) [32] has increased educational
for Health and Care Excellence (NICE) [23] for all DoH commissioning of places for 2014/2015.
[24] assessment and appraisals of health technologies, tech- The 2014 UK radiographer unfilled vacancy rate was 5.1%
niques, and screening programmes. The hourly unit cost of at Band 7 reporting level [31]; the SCoR [31] estimate 3,662
a radiologist (2014–2015) was calculated at £156 (Table 4). radiographers were in advance practice and 86 in consultant
roles, with a further 1,288 in postgraduate training [31]. The
The New Intervention master’s degree pathway in clinical reporting in our university
The RCR and the SCoR have jointly published guidance currently offers a wide range of options for MRI reporting
[26] to endorse the collaborative skills mix of radiographers modulesdincluding head and neck, IAM, spine, breast,

Table 2
Audit Results of MRI Demand at an Average-Sized Generic DGH (2014–2015)
Area Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15 Mar-15 Apr-15 May-15 Jun-15 Jul-15 Total
Knee 166 110 122 127 125 152 102 150 106 105 116 120 1,501
Lumbar 127 102 115 93 149 103 122 109 88 106 102 104 1,320
IAM 33 39 39 41 34 41 47 47 45 33 38 25 462
Scaphoid 17 11 15 17 14 10 7 7 21 16 13 13 161
Breast 2 9 5 8 5 7 5 6 9 4 11 10 81
All MRI 1,189 1,093 1,136 1,057 1,051 1,109 1,064 1,180 1,016 971 1,035 1,057 12,958
DGH, district general hospital; IAM, internal auditory meatus; MRI, magnetic resonance imaging.

196 P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203


Table 3
Decision Tree Populated with Risk Probabilities

gastrointestinal, knee, foot and ankledwith strong recruitment rate was applied, and the hourly unit cost of a reporting ra-
of students. diographer (2014–2015) was determined as £53 (Table 5).

Unit Costs and Discounting Comparison of Costs per Intervention


To estimate an hourly rate for a reporting radiographer, By applying the unit cost per hour of both interventions,
Netten et al’s Ready Reckoner for staff costs in the NHS estimations of cost per examination for both interventions
[18] and PSSRU Unit Costs of Health and Social Care can be established. The RCR activity reporting guidelines
2014 [19] were applied. The salary was based on a FTE [36] calculate time per test for reporting, which is the measure
mean of band 7 (point 30) on the Agenda for Change [33] for appointing workload standards in radiology (applying a
wages for allied health professionals. A further 7.2% was maximum of 50% of time spent reporting examinations).
added to reflect payments for additional requirements such The RCR recognises that because attempting to identify
as overtime, shift work, and geographic allowances [19], one method to model the costings for reporting is difficult,
with NI contributions [20] and employer’s contribution to and each system had limitations, the RCR elected to calculate
superannuation [21]. The costs for education and training work output using the Gishen’s Ready Reckoner [36]. The
used PSSRU [19] standard estimation approaches to calculate RCR indicative modality-based method estimates against
the components of preregistration and postgraduate training, 1 hour of uninterrupted time a range of 3–6 (noncomplex)
tuition fees, clinical placement costs, infrastructure, and lost MRI reports were possible [36], with three variable time cal-
production costs of staff training days. A 3.5% discounting culations of slow, medium, and fast (20, 13.33, and

Table 4
Consultant Radiologist Hourly Unit Cost Calculation
Unit Costs and Unit Estimation 2014/2015 Value Notes
A Wages/salary (þ) £87,060 per year Medical consultant average [19]
£29,165 per year 33.5% Allowances [19] for overtime/shift work/etc
B Salary oncosts (þ) £5,012 per year National Insurance secondary threshold deduction [20]
(þ) £11,753 per year Superannuation–NHS pensions 13.5%–Tier 6 [21]
London multiplier 1.19  (AþB) and 1.39  G Allow for higher costs of living in London [19]
Non-London multiplier 0.97  (AþB) and 0.97  G Allow for lower costs of living outside London [19]
C Qualifications (þ) £72,197 per year Taken from PSSRU [19], using Netten et al [18] costs from
DoH and HEE consultants ¼ 2 foundation years, 6 speciality registrar years
D Fees (þ) £420 per year GMC [25]
E Overheads, management, Taken from PSSRU–NHS (England) [19]
Administration and estates staff (þ) £20,048 per year Management and noncare staff 19.31% of direct care salary
F Nonstaff (þ) £43,575 per year Nonstaff costs 41.97% of direct salary costs (include costs to provider–office,
travel/transport, telephone, education, training, supplies, services, utilities of
water, gas, electricity) [19]
G Capital overheads (þ) £8,411 per year Capital costs annuitised over 60 years (discount rate of 3.5%) based on
PSSRU [19]. New build and land requirements of NHS hospitals (adjusted
for both treatment and nontreatment space)
H Working time (O) 42.4 wk/y PSSRU [19] calculated unit costs of 1,589 h/y: 212
(O) 37.5 h/wk Working days (minus sickness absence and training) [19]
Unit costs 2014/2015 £156 per hour
DoH, Department of Health; GMC, General Medical Council; HEE, Health Education England; NHS, National Health Service; PSSRU, Personal Social
Services Research Unit.

P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203 197


Table 5
Reporting Radiographer Hourly Unit Cost Calculation
Unit Costs and Unit Estimation 2014/2015 Value Notes
A Wages/salary £35,891 per year AfC band 7 mean–point 30 [33]
£2,584 per year 7.2% Allowances [19] for overtime/shift work/etc
B Salary oncosts (þ) £4,197 per year National Insurance secondary threshold deduction [20]
(þ) £3,337 per year Superannuation–NHS pensions 9.3%–Tier 4 [21]
Inner London multiplier £4,117–£6,342 per year 20% of basic salary [31]
Outer London multiplier £3,483–£4,439 per year 15% of basic salary [31]
Fringe multiplier £951–£1,649 per year 5% of basic salary [31]
C Qualifications (þ) £6,120 per year BSc diagnostic radiography tuition fees, living expenses,
clinical placement [19], and Postgraduate
Clinical placement [19] and Postgraduate Certificate in
Clinical Reporting (MRI) fees [34]–expected annual cost at
3.5%
D Fees (þ) £70 per year HCPC [35]
F Overheads management, administration, and estates staff (þ) £8,385 per year Taken from PSSRU–NHS (England) [19] Management and
noncare staff 19.31% of direct care salary
G Nonstaff (þ) £18,225 per year Nonstaff costs 41.97% of direct salary costs (includes costs to
provider, office, travel, transport, telephone)
Education, training, supplies, services, utilities of water, gas,
electricity [19]
H Capital overheads (þ) £8,411 per year Capital costs annuitised over 60 years (discount rate of 3.5%)
based on PSSRU [19]. New build and land requirements of
NHS hospitals (adjusted for both treatment and non–
treatment space)
I Working time (O) 42.4 wk/y PSSRU calculated unit costs of 1,589 h/y: 212 working d
(minus sickness absence and training) [19]
(O) 37.5 h/wk
Unit costs 2014/2015 £53 per hour
AfC, Agenda for Change; HCPC, Health and Care Professions Council; MRI, magnetic resonance imaging; NHS, National Health Service; PSSRU, Personal
Social Services Research Unit.

10 minutes per examination per report, respectively [12]). of life) from the change of service delivery. The DGH audit
The CfWI and DoH [12] use weighted factors of 24, 16, data did not provide statistics from error and/or discrepancy
and 12 minutes per examination per report. The CfWI [12] meetings to assess the potential for detrimental risk to pa-
calculated that each FTE radiologist was allocated 10.3 pro- tient outcomes through reporting. To estimate the potential
grammed activities (PAs); 2 PAs for nonreporting administra- for errors in reporting that theoretically could affect the
tion of paperwork, teaching, and other duties, with 8 weeks health of patients, a literature review on the diagnostic accu-
deducted for annual leave/study. Likewise, the RCR [14] cal- racy of radiologists and reporting radiographers interpreting
culations use 10.3 PAs (8 PAs over 44 weeks). MRI examinations was performed using Cochrane, MED-
A limitation of this review found the SCoR have no pub- LINE, Europe PubMed Central, CINAHL, Science Direct,
lished costings of reporting radiographers’ unit costs per and Google Scholar. The search results were limited because
noncomplex MRI examination to compare against, so the of the variation and quality of the literature methodologies
RCR [12,36] and CfWI and DoH [12] systems have been and results, and details on sample size and pathology range
adopted for comparison. A literature search using CINAHL, were inconsistent. In addition, reference standards varied,
Cochrane Library, MEDLINE, PubMed, Science Direct, with certain studies only providing agreement levels, mostly
and Google Scholar found no studies on the time taken for without confidence intervals, sensitivity, or specificity.
radiographers to report MRI scans. The study reverted to Observer variation studies on radiologist diagnostic perfor-
the evidence of previously published studies [3739] that mance from a number of published sources comparing set
applied timed reporting of MRI examinations (same case- reference standards for MRI knee studies have identified radi-
loads) of radiographers and radiologists, which produced ologist agreement levels ranging from 48.1% to 96%
near equivalent accuracy, agreement, sensitivity, and speci- [37,4046]. Radiologist sensitivity levels ranged from
ficity results. 73.5% to 88% [42,43,46,47], with specificity between
90.8% to 97% [40,43,46,47]. The introduction of reporting
Comparison of Diagnostic Thresholds per Intervention radiographers to interpreting MRI knee examinations was re-
The risk of discordance in reporting is another important viewed by the university in an academic setting [39],
measure to include in the evaluation of assessing interven- recording a mean sensitivity of 99.4% (95% CI, 97.4–99.8)
tions. This will influence potential impact on patient out- [39] and mean specificity of 95.9% (95% CI, 93.1–97.7)
comes (mortality, morbidity, functional status, and quality [39].

198 P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203


90.5 [42], 94 [46], 94 [43], 97 [47]
Radiologist lumbar spine MRI agreement ranged from
60.8% to 94.4% [37,40,4850]. Sensitivity and specificity
were unrecorded. Reporting radiographer agreement ranged
from 58.6% to 87.2% [38,40], sensitivity and specificity
levels were 99% [38].
Scaphoid reporting by radiologists mean sensitivity rate

Specificity %
ranged from 83.3% to 100% [5153], specificity 90%

% Unknown

% Unknown
100 [52,53]
to 100% [5153], and agreement of 86.65% to 100%

69.2 [56]

96.3 [54]
94 [39]

99 [38]

99 [38]

99 [38]
[5153]. Radiographers demonstrated a mean sensitivity
rate of 100% (95% CI, 82.3–95.1) [54] and specificity
96.3% (95% CI, 90.1–100) [54] and agreement of 92.2%

73.5 [42], 76 [46], 86 [47], 88 [43]


(95% CI, 89.3–95) [54].
MRI lumbar examinations agreement for radiologists have
documented agreement ranges from 60.8% to 94.4%
[37,40,48,50], but no sensitivity or specificity levels were
identified from the literature. Reporting radiographer lumbar
spine agreement ranged from 58.6% to 87.2% [38,40], with

Sensitivity %

% Unknown

% Unknown
sensitivity and specificity levels of 99% [38].

100 [52,53]
88.6 [56]
95.2 [39]

100 [54]
IAM diagnostic threshold studies have identified radiolo-

99 [38]

99 [38]

99 [38]
gist agreement levels between 56% and 100% [37,55], with
no found levels of separate sensitivity or specificity. Reporting
radiographers agreement levels for IAMs were 98.4% [38],

48.1 [40], 68 [41], 86.1 [42], 85 [46], 87 [37], 88 [43],


sensitivity 99% [38], and specificity 99% [38], respectively.
MRI breast observer agreement levels by radiologists were
85% [56], with sensitivity and specificity at 88.6% [56] and
69.2% [56] respectively. Evaluated in comparison to radiog-
raphers’ MRI breast agreement levels of 88.6% [39] and sensi-
tivity and specificity at 95.2% [39] and 94% [39], the results
were comparable. The literature review findings indicated that
radiographer’s MRI results are approaching, and similar to,

60.8 [40], 77 [48], 80 [49],


the range of results identified for radiologists (Table 6), taking
into account the possible variations present in the study 52.6 [40], 86.3 [38]

83.8 [50], 94.4 [38]


58.6 [40], 87.2 [38]
94 [44], 96 [45]

56 [55], 100 [37]


designs.
Agreement %

88.6 [39]

92.2 [54]

98.4 [38]
100 [51]
85 [56]

Outcome Results of Interventions to National Tariffs and


Reference Standards
Estimated Mean Diagnostic Thresholds of Current and New Interventions

The key findings estimating monetary value of the radiol-


Anatomic area

Lumbar spine

Lumbar spine

ogist’s hourly rate against reporting radiographer’s hourly rate


using RCR [36] unit costs per noncomplex MRI report
Scaphoid
Scaphoid

demonstrated a variance of £34.34–17.17 per patient/report.


Breast
Breast
Knee

Knee

IAM
IAM

Applying the CfWI and DoH [12] ranges to the hourly re-
porting rates estimated a cost difference of £41.20–£20.60
Radiographer–reporting

Radiographer–reporting

Radiographer–reporting

Radiographer–reporting

Radiographer–reporting

per patient/MRI report (Table 7).


Radiologist–reporting

Radiologist–reporting

Radiologist–reporting

Radiologist–reporting

Radiologist–reporting

The committed price NHS trusts and commissioners agree


to cost at is set by the sector regulator monitor [57], to reduce
Configuration

anticompetitive practice that is opposed to patient interests.


IAM, internal auditory meatus.

The monitor 2014–2015 direct access and outpatient diag-


nostic imaging services tariff (unbundled) [58] determines
the cost paid by Clinical Commissioning groups for an
MRI scan (one area, no contrast) as £138 [58] with reporting,
Current intervention

Current intervention

Current intervention

Current intervention

Current intervention

and cost of reporting alone as £22 [58]. Although there are


New intervention

New intervention

New intervention

New intervention

New intervention

regional variations of cost and local modifications [59], this


price is set in the current Healthcare Resource Groups
Table 6

(HRG4) costs currently in use by the NHS national tariff pay-


Unit

ment system (2014/15) and is enforced by the Health and

P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203 199


Table 7
Unit Costs of per Examination of Current and New Interventions Using RCR [36], CfWI, and DoH [12] Calculations
Noncomplex MRI Report Configuration Cost per Hour RCR [36] Slow Report RCR3 [2] Medium RCR [36] Fast Report
(20 min) Report (13.33 min) (10 min)

Current intervention Radiologist reporting £156 £52.00 per patient/report £34.66 per patient/report £26.00 per patient/report

Noncomplex MRI Report Configuration Cost per Hour CfWI/DoH [12] Slow CfWI/DoH [12] Medium CfWI/DoH [12] Fast
Report (24 min) Report (16 min) Report (12 min)

Current intervention Radiologist reporting £156 £62.40 per patient/report £41.60 per patient/report £31.20 per patient/report

Noncomplex MRI Report Configuration Cost per Hour RCR [36] Slow Report RCR [36] Medium RCR [36] Fast Report
(20 min) Report (13.33 min) (10 min)

New intervention Radiographer reporting £53 £17.66 per patient/report £11.77 per patient/report £8.83 per patient/report

Noncomplex MRI Report Configuration Cost per Hour CfWI/DoH [12] Slow CfWI/DoH [12] Medium CfWI/DoH [12] Fast
Report (24 min) Report (16 min) Report (12 min)

New intervention Radiographer reporting £53 £21.20 per patient/report £14.13 per patient/report £10.60 per patient/report
CfWI, Centre for Workforce Intelligence; DoH, Department of Health; MRI, magnetic resonance imaging; RCR, Royal College of Radiologists.

Social Care Act 2012 [60] for NHS trusts, NHS foundation using the data (n ¼ 3,525) from the acute DGH 12-
trusts, and private providers. month audit of workload. Calculated savings between
Comparison of the interventional cost of reporting ra- £121,048 - £60,524 could be possible using reporting ra-
diographers to report a noncomplex MRI scan against the diographers against the RCR [36] workload model (fast, me-
national tariff of £22 [58] per report demonstrates cost sav- dium, and slow reporting times). Calculating the reporting
ings between £2.83 and £11.17 per scan, calculated against radiographer’s unit costs against the CfWI and DoH [12] re-
all the proposed RCR [36] and CfWI and DoH [12] time porting ranges give an estimated annual cost saving of
ranges to report an MRI scan (Table 7). Extrapolation of £145,230–£72,615 (Table 8) compared with the current
the data allowed approximation over the observed range intervention of radiologists.

Table 8
Potential Unit Costs of per Annum of Current and New Interventions Using DGH Audit of Workload Against the RCR [36], CfWI, and DoH [12] Calculations
Noncomplex MRI Configuration Annual DGH RCR [36] Slow RCR [36] Medium RCR3 [2] Fast Report
Report Workload Report (20 min) Report (13.33 min) (10 min)

Current intervention Radiologist reporting 3,525 MRI scans Annual cost Annual cost Annual cost
£183,300.00 £122,141.25 £91,650.00

Noncomplex MRI Configuration Annual DGH CfWI/DoH [12] Slow CfWI/DoH [12] CfWI/DoH [12] Fast
Report Workload Report (24 min) Medium Report Report (12 min)
(16 min)

Current intervention Radiologist reporting 3,525 MRI scans Annual cost Annual cost Annual cost
£219,960.00 £146,640.00 £109,980.00

Noncomplex MRI Configuration Annual DGH RCR [36] Slow RCR [36] Medium RCR [36] Fast Report
Report Workload Report (20 min) Report (13.33 min) (10 min)

New intervention Radiographer 3,525 MRI scans Annual cost Annual cost Annual cost
reporting £62,251.50 £41,489.25 £31,125.75

Noncomplex MRI Configuration Annual DGH CfWI/DoH [12] Slow CfWI/DoH [12] CfWI/DoH [12] Fast
Report Workload Report (24 min) Medium Report Report (12 min)
(16 min)

New intervention Radiographer 3,525 MRI scans Annual cost Annual cost Annual cost
reporting £74,730.00 £49,808.25 £37,365.00
CfWI, Centre for Workforce Intelligence; DGH, district general hospital; DoH, Department of Health; MRI, magnetic resonance imaging; RCR, Royal Col-
lege of Radiologists.

200 P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203


Discussion The review of introducing an MRI skills mix reporting ser-
vice model has shown one potential option in tackling the ca-
The RCRs [15] have explored various responses to the
pacity and demand issues faced by NHS imaging
capacity demands of reporting services and acknowledged re-
departments, with a possible £145,230–£60,254 per annum
porting radiographers as one of several solutions (including
cost saving using a generic acute NHS DGH workload model.
out-sourcing, locums, overtime catch up sessions, and review
Research into discrepancy audit data from MRI reporting by
of existing radiologist’s performance). The use of locums and
radiographers and radiologists for potential risk to patient
outsourcing to commercial private companies is not without a
outcomes identified a paucity of evidence on patient mortal-
large financial burden and may not be a sustainable policy
ity/morbidity and quality of life. Further research into this
considering current NHS financial constraints.
area is recommended.
The review has shown that both interventions have the
diagnostic thresholds to achieve similar reporting standards.
The societal benefit to patients from integrating the new inter- Acknowledgements
vention could potentially improve reporting services and result The author would like to thank all the radiographers that
in faster diagnosis. Evidence from studies in x-ray [6164], participated in the study.
computed tomography [65,66], ultrasound [65], and MRI There were no financial conflicts of interest.
[65] supports achievable increases in reporting turnaround Paul Lockwood is a senior lecturer on the MSc Clinical
times. The influence of introducing system efficiencies in report- Reporting, MSc Medical Imaging pathways, and programme
ing enhances patient treatment and management [61,67,68], director for BSc Diagnostic Radiography at Canterbury
which improves quality of care and patient satisfaction. Christ Church University.
Health care economic evaluations normally review the
tradeoff in a comparison of costs, benefits, and harms to
References
determine if current treatment is more expensive and worse
than an alternative, or if a new treatment is better, but [1] National Health Service (NHS) England (2014). Five year forward
more expensive or cheaper and better. There has been prece- view. Available at: https://www.england.nhs.uk/wp-content/uploads/
2014/10/5yfv-web.pdf. Accessed October 13, 2015.
dence in the past from studies in x-ray [63,6871], computed
[2] Independent Cancer Taskforce (2015). Achieving world-class cancer out-
tomography [66] and fluoroscopy [72,73] establishing the comes: a strategy for England 2015-2020. Available at: http://www.
cost effectiveness of radiographer reporting. Any additional cancerresearchuk.org/sites/default/files/achieving_world-class_cancer_out
cost between interventions could not be appropriately calcu- comes_-_a_strategy_for_england_2015-2020.pdf. Accessed October 13,
lated to Quality Adjusted Life Years (QALYs) or Incremental 2015.
[3] The Royal College of Radiologists (2015). Press release: English Cancer
Cost Effectiveness Ratios (ICER), because the sample audit
Strategy 2015-20: the ambition should be followed by action. Available
data did not record the impact of the intervention on care at: https://www.rcr.ac.uk/posts/english-cancer-strategy-2015-20-action-
pathways; and treatment plans, as evidence from discrepancy needed-now. Accessed October 13, 2015.
audit meetings, were unavailable. [4] The King’s Fund (2015). Better value in the NHS: the role of changes in
An additional limitation of this study recognises that some clinical practice. Available at: http://www.kingsfund.org.uk/
publications/better-value-nhs. Accessed October 13, 2015.
NHS teaching hospitals use registrars in training to report
[5] Department of Health (2010). Equity and excellence: liberating the NHS.
MRI examinations and, as such, can be a cost-effective Available at: https://www.gov.uk/government/uploads/system/uploads/
approach to reporting. In justifying why registrars were not attachment_data/file/213823/dh_117794.pdf. Accessed October 13,
evaluated in the data modelling, the DGH where the audit 2015.
was obtained did not commission registrars. In addition, the [6] The Royal College of Radiologists (2010). The RCR response to: equity
and excellence: liberating the NHS. London: The Royal College of
use of registrars could be problematic because of the various
Radiologists.
levels of experience and exposure in reporting; moreover, [7] Department of Health (2011). The operating framework for the NHS in
some will require a level of double reporting at an extra England. Available at: https://www.gov.uk/government/publications/
cost of time and money. the-operating-framework-for-the-nhs-in-england-2012-13. Accessed
October 13, 2015.
[8] The Royal College of Radiologists (2011). The RCR’s response to NHS
Future Forum Further Work on Education and Training. Available at:
Conclusion http://www.rcr.ac.uk/docs/general/pdf/RCR_response-future_Forum_
In summary, the literature [11–14] implies that current education_and_training_Sept2011.pdf. Accessed August 15, 2015.
[9] Department of Health (2011). Improving outcomes: a strategy for can-
practice is not conducive to future service delivery. A consid- cer. London: Department of Health. Available at: https://www.gov.uk/
eration of future workforce planning to cope with capacity government/uploads/system/uploads/attachment_data/file/213785/dh_
and demand should include a whole-team approach to devel- 123394.pdf. Accessed October 13, 2015.
oping an effective service delivery with involvement from pro- [10] Department of Health (2010). The NHS quality, innovation, produc-
tivity and prevention challenge. Available at: http://www.dh.gov.uk/
fessional bodies, commissioners, and stakeholders. The
en/Healthcare/Qualityandproductivity/index.htm. Accessed July 10,
current scope and boundaries of imaging professions will 2015.
need to consider sufficient overlap of roles to enable an effi- [11] National Health Service (NHS) England Analytical Services (Opera-
cient service delivery. tions) (2014). NHS imaging and radiodiagnostic activity 2013/2014.

P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203 201


Available at: http://www.england.nhs.uk/statistics/wp-content/uploads/ [28] Society and College of Radiographers (2013). Preliminary clinical evalua-
sites/2/2013/04/KH12-release-2013-14.pdf. Accessed October 13, tion and clinical reporting by radiographers: policy and practice guidance.
2015. Available at: http://www.sor.org/learning/document-library/preliminary-
[12] Centre for Workforce Intelligence (2012). Securing the future workforce clinical-evaluation-and-clinical-reporting-radiographers-policy-and-prac
supply: clinical radiology stocktake. London: Centre for Workforce Intelli- tice-guidance. Accessed October 13, 2015.
gence. Available at: http://www.cfwi.org.uk/publications/securing-the- [29] Centre for Workforce Intelligence (2012). Workforce risks and oppor-
future-workforce-supply-clinical-radiology-stocktake/@@publication-de tunities: diagnostic radiography (Education commissioning risks sum-
tail. Accessed October 13, 2015. mary from 2012). Available at: http://www.cfwi.org.uk/publications/
[13] Royal College of Radiologists (2012). Investing in the clinical radiology diagnostic-radiographers-workforce-risks-and-opportunities-education-com
workforcedthe quality and efficiency case. Available at: https://www. missioning-risks-summary-from-2012. Accessed October 13, 2015.
rcr.ac.uk/sites/default/files/docs/radiology/pdf/RCR_CRWorkforce_June [30] Health and Care Professions Council (2015). StatisticsdHistoric. Lon-
2012.pdf. Accessed October 13, 2015. don: Health and Care Professions Council. Available at: http://www.
[14] Royal College of Radiologists (2014). Clinical Radiology UK Work- hcpc-uk.org/aboutregistration/theregister/oldstats/. Accessed August
force Census Report 2012. Available at: https://www.rcr.ac.uk/sites/ 10, 2015.
default/files/BFCR(14)14_CR_Census.pdf. Accessed October 13, 2015. [31] Society and College of Radiographers (2014). Diagnostic radiography
[15] The Royal College of Radiologists (2015). Unreported X-rays, UK Workforce Report 2014. Available at: http://www.sor.org/
computed tomography (CT) and magnetic resonance imaging (MRI) learning/document-library/diagnostic-radiography-uk-workforce-report-
scans: results of a snapshot survey of English National Health Service 2014. Accessed October 13, 2015.
(NHS) trusts. Available at: https://www.rcr.ac.uk/sites/default/files/ [32] Health Education England (2013). Workforce plan for England: pro-
publication/Unreported_studies_Feb2015.pdf. Accessed October 13, posed education and training commissions for 2014/15. Available at:
2015. http://hee.nhs.uk/wp-content/uploads/sites/321/2013/12/Workforce-
[16] The Royal College of Radiologists (2014). Press release: patients wait- plan-investing-in-people.pdf. Accessed August 10, 2015.
ing too long for test results. Available at: https://www.rcr.ac.uk/ [33] NHS Careers (2015). Agenda for changedpay rates. Available at: http://
patients-waiting-too-long-test-results. Accessed October 13, 2015. www.nhscareers.nhs.uk/working-in-the-nhs/pay-and-benefits/agenda-for-
[17] National Institute for Health and Care Excellence (NICE) (2006). Methods change-pay-rates/ Accessed August 10, 2015.
for development of NICE public health guidance. Available at: http://www. [34] Canterbury Christ Church University (2014). Postgraduate Pro-
nice.org.uk/guidance/ph1/evidence/methods-for-development-of-nice-pub grammes for Health and Social Care Professionals 2015/15. Available
lic-health-guidance-120988045. Accessed December 18, 2015. at: http://www.canterbury.ac.uk/study-here/docs/Health-and-Social-
[18] Netten, A., Knight, J., Dennett, J., Cooley, R., & Slight, A. (1998). Care-Postgraduate-Booklet.pdf. Accessed August 10, 2015.
A ready reckoner for staff costs in the NHS, Vol 2. Methodology. [35] Health and Care Professions Council. Paying your fees. Available at:
Canterbury: Personal Social Services Research Unit, University of http://www.hpc-uk.org/registrants/fees/paying/. Accessed October
Kent. 13, 2015.
[19] Personal Social Services Research Unit (2014). Unit Costs of Health and [36] The Royal College of Radiologists (2011). Clinical radiology workload:
Social Care 2014. Available at: http://www.pssru.ac.uk/project-pages/ guidance on radiologists’ reporting figures. Available at: https://www.
unit-costs/2014/index.php. Accessed October 13, 2015. rcr.ac.uk/publication/clinical-radiology-workload-guidance-radiologists
[20] HM Revenue and Customs (2015). Business tax guidancedrates and %E2%80%99-reporting-figures. Accessed August 10, 2015.
thresholds for employers 2015 to 2016. Available at: https://www.gov. [37] Piper, K., & Buscall, K. L. (2008). MRI reporting by radiographers: the
uk/rates-and-thresholds-for-employers-2015-to-2016. Accessed October construction of an objective structured examination. Radiography 14(2),
13, 2015. 78–89.
[21] NHS Business Services Authority (2013). NHS Pension Scheme: 2014/ [38] Piper, K., Buscall, K. L., & Thomas, N. (2010). MRI reporting by ra-
15 Tiered Employee Contributions Year Three of Three. Available at: diographers: findings of an accredited postgraduate programme. Radiog-
http://www.nhsbsa.nhs.uk/Pensions/Documents/Pensions/Tiered_Con raphy 16(2), 136–142.
tributions_2014-15_Emplyer_Factsheet_V1_311213.pdf. Accessed [39] Pittock, L., Piper, K., & Rendle, J. (2015). Radiographer reporting of
October 13, 2015. magnetic resonance imaging breast examinations: findings of an Ac-
[22] Treasury, H. M. (2003). Appraisal and evaluation in Central Government credited Postgraduate Programme. Manchester, UK: (UKRC Poster).
(‘The Green Book’). Available at: https://www.gov.uk/government/ UKRC.
uploads/system/uploads/attachment_data/file/220541/green_book_comp [40] Brealey, S., Piper, K., & King, D., et al. (2013). Observer agreement in
lete.pdf. Accessed October 13, 2015. the reporting of knee and lumbar spine magnetic resonance (MR) imag-
[23] National Institute for Health and Care Excellence (NICE) (2013). Pro- ing examinations: selectively trained MR radiographers and consultant
cess and methods guides: guide to the methods of technology appraisal radiologists compared with an index radiologist. Eur J Radiol 82(10),
2013 [PMG9]. Available at: https://www.nice.org.uk/article/pmg9/ e597–e605.
chapter/foreword. Accessed October 13, 2015. [41] Bryan, S., Weatherburn, G., & Bungay, H., et al. (2001). The cost-
[24] Department of Health (2010). Quantifying health impacts of government effectiveness of magnetic resonance imaging for investigation of the
policies: a how to guide to quantifying the health impacts of government knee joint. Health Technol Assess 5, 27.
policies. Available at: https://www.gov.uk/government/uploads/system/ [42] Sonin, A. H., Pensy, R. A., Mulligan, M. E., & Hatem, S. (2002).
uploads/attachment_data/file/216003/dh_120108.pdf. Accessed October Grading articular cartilage of the knee using fast spin-echo proton
13, 2015. density-weighted MR Imaging without fat suppression. Am J Roentgenol
[25] General Medical Council (2015). Fees. Available at: http://www. 197, 1159–1166.
gmc-uk.org/doctors/fees.asp. Accessed October 13, 2015. [43] White, L. M., Schweitzer, M. E., Deely, D. M., & Morrison, W. B.
[26] Royal College of Radiologists & Society and College of Radiography (1997). The effect of training and experience on the magnetic reso-
(2012). Team working in clinical imaging. Available at: http://www. nance imaging interpretation of meniscal tears. Arthroscopy 13(2),
sor.org/sites/default/files/document-versions/BFCR(12)9_Team.pdf. Ac- 224–228.
cessed October 13, 2015. [44] Schweitzer, M. E., Tran, D., Deely, D. M., & Hume, E. L. (1995).
[27] Society and College of Radiographers (2013). Code of professional Medial collateral ligament injuries: evaluation of multiple signs, preva-
conduct. Available at: http://www.sor.org/learning/document-library/ lence and location of associated bone bruises, and assessment with
code-professional-conduct. Accessed October 13, 2015. MR Imaging. Radiology 194, 825–829.

202 P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203


[45] Cotton, A., Delfaut, E., & Demondion, X., et al. (2000). MR imaging [59] National Health Service (NHS) England Publications (2013). 2014/15
of the knee at 0.2 and 1.5T: correlation with surgery. Am J Roentgenol National Tariff payment System: Annex 6A Market forces factor payment
174, 1093–1097. values. gateway Reference 00883. Available at: https://www.gov.uk/
[46] Roos, J. E., Chilla, B., & Zanetti, M., et al. (2006). MRI of meniscal government/uploads/system/uploads/attachment_data/file/300552/
lesions: soft-copy (PACS) and hard-copy evaluation versus reviewer Annex_6A_Market_Forces_Factor_payment_values.xlsx. Accessed August
experience. Am J Roentgenol 186, 786–790. 10, 2015.
[47] Disler, D. G., McCauley, T. R., & Kelman, G. C., et al. (1996). Fat- [60] Department of Health (2012). Health and Social Care Act 2012, Chap-
suppressed three-dimensional spoiled gradient-echo MR imaging of ter 7. Available at: http://www.legislation.gov.uk/ukpga/2012/7/enacted.
hyaline cartilage defects in the knee: comparison with standard MR Accessed October 13, 2015.
imaging and arthroscopy. Am J Roentgenol 167, 127–132. [61] Woodford, A. J. (2006). An investigation of the impact/potential
[48] van Rijn, J. C., Klemets€o, N., & Reitsma, J. B., et al. (2005). Observer impact of a four-tier profession on the practice of radiography: a litera-
variation in MRI evaluation of patients suspected of lumbar disk herni- ture review. Radiography 12, 318–326.
ation. Am J Roentgenol 184, 299–303. [62] Health and Care Professions Council (2015). Paying your fees. Avail-
[49] Brant-Zawadzki, M. N., Jensen, M. C., Obuchowski, N., Ross, J. S., & able at: http://www.hpc-uk.org/registrants/fees/paying/. Accessed August
Modic, M. T. (1995). Interobserver and intraobserver variability in 10, 2015.
interpretation of lumbar disc abnormalities: a comparison of two no- [63] Smith, T. N., & Baird, M. (2007). Radiographers’ role in radiological re-
menclatures. Spine 20(11), 1257–1263. porting: a model to support future demand. Med J Aust 186(12), 629–631.
[50] Pfirrmann, C. W., Metzdorf, A., Zanetti, M., Hodler, J., & Boos, N. [64] Brayley, N. (2000). The need for radiographer reporting: an accident &
(2001). Magnetic resonance classification of lumbar intervertebral disc emergency department (A&E) perspective. Radiography 6(4), 227–229.
degeneration. Spine 26(17), 1873–1878. [65] Woznitza, N., Piper, K., Rowe, S., & West, C. (2004). Optimizing pa-
[51] Breitenseher, M. J., Metz, V. M., & Gilula, L. A., et al. (1997). Radio- tient care in radiology through team-working: a case study from the
graphically occult scaphoid fractures: value of MR imaging in detection. United Kingdom. Radiography 20(3), 258–263.
Radiology 203, 245–250. [66] Lockwood, P. (2015). An economic evaluation of introducing a skills
[52] Gaebler, C., Kukla, C., Breitenseher, M., Mrkonjic, L., Kainberger, F., mix approach to CT head reporting in clinical practice. Radiography
& Vecsei, V. (1998). Limited diagnostic value of macroradiography in e1–e7. Available at: http://www.sciencedirect.com/science/article/pii/
suspected scaphoid fractures. Acta Orthop Scand 69(4), 401–403. S1078817415001169. Accessed October 15, 2015.
[53] Hunter, J. C., Escobedo, E. M., Wilson, A. J., Hanel, D. P., Zink- [67] Paul, D. (2009). An overview of initiatives relating to advanced practice
Brody, C. G., & Mann, F. A. (1997). MR imaging of clinically sus- role development for radiological technologists. J Med Imaging Radiat
pected scaphoid fractures. Am J Roentgenol 168, 1287–1293. Sci 40(3), 90–99.
[54] Moller, J. M., Larsen, L., & Bvin, J., et al. (2004). MRI diagnosis of [68] Wensing, M., Jung, H. P., Mainz, J., Olesen, F., & Grol, R. (1998). A sys-
fracture of the scaphoid bone: impact of a new practice where the images tematic review of the literature on patient priorities for general practice care.
are read by radiographers. Acad Radiol 11(7), 724–728. Part 1: description of the research domain. Soc Sci Med 47(10), 1573–1588.
[55] Zealley, I. A., Cooper, R. C., & Clifford, K. M., et al. (2000). MRI [69] Hardy, M., Hutton, J., & Snaith, B. (2013). Is a radiographer led im-
screening for acoustic neuroma: a comparison of fast spin echo mediate reporting service for emergency department referrals a cost
and contrast enhanced imaging in 1233 patients. Br J Radiol 27, effective initiative? Radiography 19(1), 23–27.
242–247. [70] Brealey, S., King, D. G., & Hahn, S., et al. (2014). The costs and effects
[56] Warren, R., Hayes, C., & Pointon, L., et al. (2006). A test of perfor- of introducing selectively trained radiographers to an A&E reporting
mance of breast MRI interpretation in a multicentre screening study. service: a retrospective controlled before and after study. Br J Radiol
Magn Reson Imaging 24(7), 917–929. 78(930), 499–505.
[57] Monitor (2013). Enforcement of the National Tariff (IRG 29/13). [71] Kelly, J., Piper, K., & Nightingale, J. (2008). Factors influencing the
Available at: https://www.gov.uk/government/uploads/system/uploads/ development and implementation of advanced and consultant radiogra-
attachment_data/file/300865/Enforcement_of_the_national_tariff.pdf. pher practice–A review of the literature. Radiography 14, e71–e78.
Accessed October 13, 2015. [72] Meertens, R., Brealey, S., Nightingale, J., & McCoubrie, P. (2013).
[58] National Health Service (NHS) England Publications (2013). 2014/ Diagnostic accuracy of radiographer reporting of computed tomography
15 National Tariff Payment System: Annex 5A national prices. colonography examinations: a systematic review. Clin Radiol 68(4), e177–
London: NHS England Publications gateway Reference 00883. Avail- e190.
able at: https://www.gov.uk/government/uploads/system/uploads/ [73] Mannion, R. A. J., Bewell, J., Langan, C., Robertson, M., &
attachment_data/file/300551/Annex_5A_National_Prices.xlsx. Ac- Chapman, A. H. (1995). A barium enema training programme for ra-
cessed August 10, 2015. diographers: a pilot study. Clin Radiol 50(10), 715–719.

P. Lockwood/Journal of Medical Imaging and Radiation Sciences 47 (2016) 194-203 203

Вам также может понравиться