INSTITUTE OF PHYSICS PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 51 (2006) 24412463 doi:10.1088/0031-9155/51/10/006

A comparison between objective and subjective image
quality measurements for a full eld digital
mammography system
N W Marshall
Clinical Physics Group, Barts and the London NHS Trust, St Bartholomews Hospital,
London EC1A 7BE, UK
Received 6 September 2005, in nal form 30 March 2006
Published 27 April 2006
Online at stacks.iop.org/PMB/51/2441
Abstract
This paper presents pre-sampling modulation transfer function (MTF),
normalized noise power spectrum (NNPS) and detective quantum efciency
(DQE) results for an amorphous selenium (a-Se) full eld digital
mammography system. MTF was calculated from the image of an angled
0.5 mm thick Cu edge, acquired without additional beam ltration. NNPS
data were acquired at detector air-kerma levels ranging from 9.1 Gy to
331 Gy, using a standard mammography x-ray spectrum of 28 kV, Mo/Mo
target/lter combination and 4 cm of PMMA additional ltration. Prior to
NNPS estimation, the image statistics were assessed using a variance image.
This method was able to easily identify a detector artefact and should prove
useful in routine quality assurance (QA) measurements. Detector DQE,
calculated from the NNPS and MTF data, dropped to 0.3 for low detector
air-kerma settings but reached an approximately constant value of 0.6 above
50 Gy at the detector. Subjective image quality data were also obtained at
these detector air-kerma settings using the CDMAM contrast-detail (c-d) test
object. The c-d data reected the trend seen in DQE, with threshold contrast
increasing at low detector air-kerma values. The c-d data were then compared
against predictions made using two established models, the Rose model and a
standard signal detection theory model. Using DQE(0), the Rose model gave
results within approximately 15% on average for all the detector air-kerma
values studied and for detail diameters down to 0.2 mm. Similar agreement
was also found between the measured c-d data and the signal detection theory
results, which were calculated using an ideal human visual response function
and a system magnication of unity. The use of full spatial frequency DQE
improved the agreement between the calculated and observer results for detail
sizes below 0.13 mm.
0031-9155/06/102441+23$30.00 2006 IOP Publishing Ltd Printed in the UK 2441
2442 N W Marshall
Introduction
The past few years have seen the introduction to the UK of full eld digital mammography
(FFDM) systems, initially for the purpose of breast assessment rather than breast screening.
FFDMtechnology falls into two distinct categoriesthe computed radiography (CR) approach
involving the use of photostimulable phosphor plates held in a cassette and the so-called
integrated detector methods. Integrated detector systems use an x-ray converter bonded to
a thin lm transistor (TFT) array which performs the image readout. One of the major
advantages of integrated digital detectors is that the image is available directly on the hospital
information networkCR systems currently rely on manual handling of the imaging cassette
by the radiographer between the cassette holder and the CR reader. Hence, integrated digital
systems should allow greater patient throughput and enable the radiographer to stay with the
patient in the x-ray room throughout the duration of the examination.
Assessment of conventional lm/screen mammography systems in terms of
commissioning and quality assurance (QA) tests is covered extensively in IPEM Report 89
(2005). One of the challenges of the new full eld digital technology lies in extending existing
test protocols to fully assess the capabilities of these new detectors. One such protocol that
has recently been made available is the EUREF Digital Addendum (2003), which gives tests
suitable for the both the commissioning and routine QA assessment of both CR and direct
detection FFDM systems. This protocol suggests that objective measurements such as the
noise power spectrum(NPS) and modulation transfer function (MTF) may also be measured as
part of the commissioning and routine QA measurements performed on these units. Objective
measures are a well-established and powerful means of assessing image quality (Metz et al
1995, ICRU 1996), especially given the relatively easy access to the required image data now
offered by the DICOMformat. Within the past 15 years or so, these methods have been applied
to various digital imaging modalities (Hillen et al 1987, Workman and Cowen 1993, Dobbins
et al 1995, Flynn and Samei 1999, Stierstorfer and Spahn 1999, Williams et al 1999, Evans
et al 2002, Samei and Flynn 2003). As demonstrated by Samei and Flynn (2003), accurate
measurement of MTF, NPS and DQE is a complex procedure with many important parameters
that affect the nal result.
For imaging modalities such as conventional screen/lm (S/F) mammography or
uoroscopy, it is far harder to apply these objective methods for reasons of image data
access, where images are held by a lm or in the form of a TV voltage. Hence, in the eld
of conventional screen/lm (S/F) mammography, subjective or semi-quantitative test objects
are commonly used to assess image quality. In the UK, this might include the TOR(MAM)
(Cowen et al 1992) test objects, along with the CDMAM 3.4 (Artinis Medical Systems BV,
Netherlands) contrast-detail (c-d) test object. All of these test objects provide a controlled
input signal when imaged under standard conditions and require the observer to perform a
visual assessment of the image. It is likely that some or all of these methods will be carried
over in to the assessment of image quality for FFDM units.
The aim of this work was to assess a Lorad Selenia (Lorad Corp., USA) FFDM imaging
system using both subjective and objective image quality measurements over a range of air-
kerma values at the detector. Subjective image quality was evaluated using c-d data acquired
with CDMAMtest object. While there are known problems with the c-d method (ICRU1996),
current acceptable and achievable image quality standards in the EUREFdigital mammography
protocol are couched in terms of a c-d curve. Objective image quality was measured using
NPS and pre-sampling MTF. Results were then compared in the light of two models: the Rose
model (Rose 1948) and a version of the well-known signal detection theory model as applied
to medical imaging systems (Wagner and Brown 1985) supplied by Aufrichtig (1999).
A comparison between objective and subjective image quality measurements for a FFDM system 2443
Materials and methods
Tube and generator
Tube and generator function was assessed under current protocols (IPEM 2005). Accuracy of
the tube potential was within the expected range of 1 kV of nominal across the range from
24 kV to 34 kV. X-ray output at 50 cm was 232 Gy mAs
1
for a nominal tube potential
of 28 kV. System half value layer (HVL), measured at 30 kV nominal with the compression
plate out of the beam and Mo lter selected was 0.31 mm Al giving an estimated x-ray tube
ltration of 0.54 mm Al equivalent.
Detector
The Lorad Selenia uses a direct conversion at panel detector in which x-rays are directly
converted to charge by the amorphous selenium(a-Se) (Yorker et al 2002). An in-depth review
of this direct detection technology is given by Rowlands (2000). In this type of detector, a
large potential difference (typically 2000 V) is held across the a-Se layer and this ensures
that charge generated in the photoconductor by x-ray interaction is drawn directly towards
the pixel readout system with very little lateral spread. The a-Se x-ray converter layer in the
Lorad Selenia is 200 m thick (CEP Report 05084 2006). The holes generated during x-ray
interaction are collected by pixel electrodes and stored on a capacitor located at each pixel.
Readout of the pixel charge is achieved using an amorphous silicon active matrix thin lm
transistor (TFT) array to switch out the charge, which is then converted to a voltage by an
amplier attached to each data line. The 24 29 cm detector is mapped on to a 3584
4096 pixel array, giving a 70 m pixel size and a sampling limit of 7.1 mm
1
.
Variance
The NPS describes the spectral decomposition of the noise variance in an image as a function of
spatial frequency. When estimating NPS, various assumptions are made regarding the nature
of the stochastic processes that result in the nal image variance. For digital imaging systems,
it is often assumed that the random process generating the variance is wide sense cyclo-
stationary (WSCS) (Cunningham 2000), meaning that the expected value and autocorrelation
of the noise realized in the image are both stationary i.e. they are independent of position in
the image and that these statistics do not change between two positions separated by an integer
multiple of the sample spacing (Cunningham 2003, Maidment et al 2003). The importance
of these assumptions is discussed elsewhere (ICRU 1996, Cunningham 2000, Williams et al
1999).
Most NPS algorithms sample image data from a variety of positions across the image
(Cunningham 2000) and therefore the notion of stationarity is crucial. If we wish to uncover
the spectral decomposition of the image variance using a method that relies on periodicity,
such as Fourier analysis, then it is important the variance remains constant over the region
from which the data are sampled. One of the principal reasons a digital imaging system may
fail this criterion is that of detector artefacts, be it in the form of dead pixels, dead lines or
some other artefact. Before proceeding to full NPS analysis, it has been recommended that an
image of the variance is initially formed (Maidment et al 2003) and assessed for stationarity.
In this study, variance images were produced from a single image using a 10 10 pixel
region of interest (corresponding to 0.7 0.7 mm). The variance of the pixels within this
ROI was calculated and this value was assigned to all the pixels in the ROI. The ROI was then
moved across the image in steps of 10 pixels, so that the entire image was covered. Finally,
2444 N W Marshall
the variance value for each ROI was assigned a grey scale value; low variance equal to black
and high variance equal to white. This image then allows a quick visual inspection of the
uniformity of the variance in the image and whether there are signicant artefacts present.
Signal transfer property (STP)
The rst step in objective image quality analysis is measurement of the system signal transfer
property (STP), or detector response as it is dened in the EUREF protocol. This will tell us
whether the system has a linear relationship between the incident x-ray uence striking the
detector and the output parameter (usually a pixel value). For objective image methods to be
valid the relationship between system input and output must be linear or at least linearizable,
for the case of screen/lm and computed radiography systems. The STP was measured
following the EUREF protocol by placing 4 cm PMMA at the x-ray tube head with the
x-ray grid removed and exposing using standard conditions of 28 kV and Mo/Mo target lter
combination. Initially, a calibrated dose meter was placed 8 cm above the detector at the
centre of the x-ray eld and x-ray output was measured for a range of mAs product values.
The detector was protected using a lead beam block during this procedure. This enabled the
accurate estimation of air-kerma as a function of mAs at the detector input plane. The lead
block was then removed and mAs product varied from 4 to 140 mAs and the pixel value (PV)
fromat eld images (i.e. linear images, corrected for detector artefacts) was measured using
a region of interest (ROI) placed at the centre of the image. The STP was derived by plotting
measured PV against air kerma at the detector.
Modulation transfer function (MTF)
Several methods for measuring MTF have been described in the literature (Judy 1976, Sones
and Barnes 1984, Hillen et al 1987, Fujita et al 1992, Samei et al 1998) and these have been
applied to a range of x-ray imaging systems. In this study, a version of the angled edge was
used to measure pre-sampling MTF i.e. the MTF prior to sampling by the pixel matrix (Giger
et al 1984). It is well known that FFDM systems are not shift invariant (Dobbins et al 1995,
Cunningham 2000, Albert and Maidment 2000), as they are based around discrete digital
detector. Hence, the signal (image) produced by the edge used in the MTF measurement
method will vary depending upon its position relative to the pixel matrix. The angled-edge
method is used to produce a very nely sampled edge spread (Samei et al 1998, Albert and
Maidment 2002) that can be used to estimate the pre-sampling MTF i.e. the MTF prior to
sampling by the pixel matrix. The details of the angled edge method are described fully
elsewhere (Samei et al 1998) but some points relevant to the implementation of this method
are given briey here.
Using this method, the pre-sampling MTF can be estimated in two directions: MTF
parallel to the chest wall edge is dened as the data line direction for this detector and the
direction away from the chest wall towards the nipple edge is the gate line direction. The MTF
was measured at the detector centre; MTF was not studied as a function of position across the
detector. First, a 0.5 mm thick copper sheet was placed on the tabletop at the centre of detector
eld and angled slightly with respect to the pixel matrix. An image of the edge was acquired at
28 kV and 20 mAs and saved to CD-R in DICOM format. As will be shown later, the system
has a linear signal transfer property and therefore it was not necessary to linearize the image
before calculating pre-sampling MTF. Next, the angle of the edge was found by differentiating
along the direction of interest. A simple least-squares method was used to t a rst-order
polynomial y = a + bx to the pixel values lying along the edge. The angle of the edge was
A comparison between objective and subjective image quality measurements for a FFDM system 2445
then calculated as tan
1
(b) where b is the gradient of the line. Knowing the angle of the edge
allows us to re-sample the image using a ner sample distance than the pixel pitch to create
the over-sample. The pixel values from a square region spanning the edge were then assigned
to the over-sample array, with a position along the array given by the distance from the edge.
This distance was calculated using the equation of the line (Rogge et al 2003).The pixel data
in the over-sample array were then re-binned according to a sub-pixel binning factor (Samei
et al 1998). In this work we use = 15, which represents a compromise between resolution
and noise on the MTF result. Finally, the over-sample was normalized for the number of pixels
assigned to each bin.
Before differentiating the over-sampled edge spread function, some ltering can be applied
in order to reduce noise (see the paper by Maidment and Albert (2003) for a discussion on
conditioning of MTF data). Great care should be taken when trying to reduce noise in
the pre-sampling MTF estimateinappropriate choice of lter type could easily bias the pre-
sampling MTF. In this work, a median lter is applied to the over-sample before differentiating
as this a simple and relatively benign lter. After median ltering, the over-sampled edge
spread function was differentiated, Fourier transformed and normalized to obtain the
pre-sampling MTF.
Noise power spectrum (NPS)
As part of the aim of this paper is to examine the feasibility of measuring objective image
quality parameters as prescribed in the EUREF protocol, a simple NPS method suitable for
routine QAmeasurement was sought. Some of the important theoretical aspects of noise power
measurements are detailed in the work of Cunningham (2000) and Dobbins (2000). Williams
et al (1999) give a thorough account of the practicalities of noise power estimation for digital
mammography systems. Noise power spectra in digital mammography are estimated using
at eld images, usually produced by imaging a block of PMMA under standard conditions.
In this study, we used a tube potential of 28 kV, a Mo/Mo target lter combination and the
4 cm PMMA blocks supplied with the CDMAM test object. These blocks were suspended at
the x-ray tube port, generating a uniform image with reduced levels of scatter. Images were
acquired with the anti-scatter grid out of the x-ray beam (similar to the geometry employed
by Evans et al (2002)).
In this study we compared objective and subjective image quality parameters across a
range of input air-kerma values at the image receptor. First, the typical operating point for the
automatic exposure control (AEC) was taken to be the mAs delivered under AEC control with
4 cm PMMA and the CDMAM c-d test object placed on the table and with the anti-scatter grid
in place. AEC sensing region was set to position 2 i.e. approximately 3 cm from chest wall
edge of the detector. For 28 kV, Mo/Mo factors, the AEC delivered 105 mAs. The standard
clinical mAs product for 4 cm PMMA and the CDMAM c-d test object was taken to be
110 mAs (i.e. the nearest mAs station to 105 mAs). This gave a mean PV measured at the
centre of the image of 507. Flat eld images for 4 cm PMMA at an equivalent detector input
air kerma were acquired by matching the PV for this image; in this case 35 mAs gave a PV
of 497. Flat eld NPS images were then acquired at values of 4, 9, 18, 35, 70 and 140 mAs,
corresponding to detector air-kerma values of 9.1, 21.0, 42.3, 82.5 165 and 331 Gy.
The NPS was calculated using a two-dimensional algorithm from a at eld image as
follows. First, a region (e.g. 1024 1024 or 2048 2048 pixel) was extracted from the
central area of the 3584 4096 image. Non-overlapping regions of interest or records of size
128 128 pixels were then taken from this area (Evans et al 2002). Williams et al (1999)
acquired their records from a region of approximately 5 cm 5 cm, however we have found
2446 N W Marshall
that using a central region of 7.2 cm (of 1024 1024 pixels) or 14.4 cm (2048 2048 pixels)
gave similar results.
The NPS data in this study are calculated fromun-subtracted images and therefore contain
contributions fromstochastic, varying non-stochastic (i.e. non-stochastic but varying fromone
image to another, such as slight ghosting artefacts etc) and xed pattern noise sources present
in the system (Williams et al 1999). These can be present, despite the at elding process that
is routinely applied to this system. Evans et al (2002) excluded xed pattern noise sources
from their analysis by using subtracted images. In our analysis, each record was corrected for
the presence of background trends by tting a polynomial and subtracting the tted surface
S(x, y) from the record. For a 128 128 record size, there was some difference between rst-
and second-order polynomials at low spatial frequencies (<0.25 cycles mm
1
). As second-
order polynomials and above gave similar results, the NPS in this study were calculated using
second-order polynomial detrending.
The 2D digital NPS (in terms of pixel value uctuation) was then calculated using the
formula (Williams et al 1999, Dobbins 2000):
NPS
P/P
(u
n
, v
k
) =
lim
N
x
, N
y

lim
M
1
M
N
x
x N
y
y

m=1
|FT
n,k
{p(x, y) S(x, y)}|
2
(1)
where N
x
and N
y
is the record size in the x and y directions and x and y are the x and y
pixel sizes, M is the number of records used in the ensemble average and FT denotes the fast
Fourier transform operation. The limits in this equation were realized by increasing the size
of N
x
and N
y
(i.e. the NPS ROI size) until the NPS converged to some value. Values of N
x
and
N
y
below 128 pixels (approximately 9 mm) resulted in an underestimate of the NPS for spatial
frequencies below approximately 0.7 mm
1
.
The value of M governs the number of NPS spectra in the ensemble; increasing the value
of M reduces the uncertainty on the nal NPS estimate. Using 128 128 pixel records taken
from a 2048 2048 region at the centre of one image gave 256 spectra in the ensemble and
a coefcient of variation of approximately 1/(256)
1/2
6% . For the IDL code used in this
study, FT is given by
F(u) =
1
N
N1

0
p(x) e
(2iux/N)
. (2)
NPS
P/P
was then divided by the (large area signal)
2
to obtain NPS
E/E
, the normalized NPS
expressed in terms of relative input exposure uctuation (Dainty and Shaw 1974, Dobbins
2000):
NPS
E/E
(u, v) =
NPS
P/P
(u, v)
(large area signal)
2
. (3)
The term (large area signal) was taken to be the mean PV in the image at the dose/image of
interest and this can be obtained from the STP (after correction for the offset).
Detective quantum efciency (DQE)
DQE was calculated using the equation:
DQE(u, v) =
MTF
2
(u, v)
q
0
NPS
E/E
(u, v)
. (4)
A comparison between objective and subjective image quality measurements for a FFDM system 2447
Table 1. Pixel value and air kerma at detector as a function of mAs for CDMAM images and
PMMA images used for NPS analysis.
CDMAM 4 cm PMMA
Pixel Air kerma at Pixel Air kerma at
mAs value detector (Gy) mAs value detector (Gy)
14 246 11.9 4 231 9.1
28 269 18.5 9 278 21.0
55 350 41.0 18 353 42.3
110 507 84.8 35 497 82.5
220 828 174 70 796 165
400 1430 342 140 1388 331
As discussed by several authors (Hillen et al 1987, Stiersorfer and Spahn 1999, Williams et al
1999), most current x-ray detectors integrate the energy of the incident x-ray uence, rather
than count the individual x-ray photons. This implies that for DQE, we must compare the
detector against an ideal energy integrating device, rather than an ideal photon counter. Hence,
for an input spectral distribution (E), q
0
in equation (4) is given by
q
0
=
_ _
(E) E dE
_
2
_
(E) E
2
dE
. (5)
We are therefore using the variance of the incident energy uence to quantify the input quantum
noise (Stierstorpher and Spahn 1999). Using the data given by Cranley et al (1997) gives
a value of q
0
= 5300 photons mm
2
Gy
1
for the 28 kV spectrum from a Mo target with
a nominal 30 m Mo lter used in this study. This is close to the value of 5500 photons
mm
2
Gy
1
adopted by Evans et al (2002). There is a carbon bre cover plate of unspecied
thickness protecting the detector and grid assembly. The effect of this cover plate on the
number of transmitted photons was not included on the DQE gures presented in this study.
Contrast-detail data
To study the c-d performance of the system as a function of dose/image, two sets of CDMAM
images were acquired at values of 14, 28, 55, 110, 220 and 400 mAs. The detector input air
kerma for these mAs settings were 12, 18, 41, 85, 174 and 342 Gy. These gures were
estimated from the detector STP measured previously for 4 cm PMMA. The detector STP
gradient changes slightly with mean energy of the x-ray beam, however a beam load of 4 cm
PMMA is sufciently close to 4 cm PMMA with the CDMAM test object not to introduce
signicant errors into this estimation. Table 1 compares air kerma at the detector for 4 cm
PMMA (i.e. for the NPS results) against detector air kerma for the CDMAM images.
The contrast for a gold disc of thickness t
Au
was calculated as follows. First, the number
of photons mm
2
transmitted through the disc at a given energy E (
disc
(E)) was calculated:

disc
(E) =
_
E max
0

inc
(E) exp((
Au
(E)t
Au
+
PMMA
(E)t
PMMA
+
Al
(E)t
Al
)) dE. (6)
The number of photons mm
2
in the background spectrum (no gold disc),
bkg
(E), is

bkg
(E) =
_
E max
0

inc
(E) exp((
PMMA
(E)t
PMMA
+
Al
(E)t
Al
)) dE (7)
2448 N W Marshall
where
Au
(E),
PMMA
(E) and
Al
(E) are the linear attenuation coefcients of gold, PMMA
and aluminium at energy E, respectively.
inc
(E) is the Mo x-ray spectrum incident on to the
CDMAM test object at 28 kV and includes attenuation by 0.69 mm Be and 0.03 mm Mo tube
ltration. The contrast of the gold disc (C
p
) was then calculated in terms of air kerma using
C
p
=
_
E max
0

bkg
(E)K(E) dE
_
Emax
0

disc
(E)K(E) dE
_
E max
0

bkg
(E)K(E) dE
(8)
where K(E) is the photon to kerma conversion factor (Cranley et al 1997). The contrast is
dimensionless and therefore has no physical units.
As CDMAM is placed between 4 cm of PMMA, the radiation contrasts generated (C
P
)
will be degraded by scattered radiation:
C =
C
P
(1 + S/P)
(9)
where S/P is the ratio of scattered to primary radiation (Dance and Day 1984). A typical
factor for 4 cm breast thickness calculated for a linear grid is approximately 0.45. The Selenia
system uses a cellular (HTC) grid (Lorad Corp., USA) however, in the absence of data for this
grid type, a nominal factor of 0.4 was used in this work.
The CDMAM test object is formed of a grid of 1 cm 1 cm squares, with each square
containing two identical circular discs, one at the centre and the other in a randomly selected
corner (the eccentric disc). Included in the CDMAM manual is a scoring and correction
scheme that utilizes the correct identication of the position of eccentric disc in each square.
This method, requiring at least three observers, produces reproducible results but is time
consuming to implement in a QA programme. CDMAM, as used in the QA programme at
our physics centre, is scored as a conventional c-d test object i.e. the observer counts to the
last disc visible at the centre of the square for each detail diameter. The contrast of this disc
is then taken to be the threshold contrast C
T
for that detail diameter. This is the method used
when scoring uoroscopy (Hay et al 1985) and digital subtraction angiography (Cowen et al
1987) c-d test objects. We are therefore comparing subjective and objective measurements
acquired under typical QA conditions.
The c-d images were scored by two experienced observers on the radiologist softcopy
reporting station. This utilizes two Barco (USA) ve megapixel cathode ray tube (CRT)
monitors (2500 2000 pixels with a display bit depth of 10 bits). The luminance response of
these CRT monitors is checked using a luminance meter as part of the routine quality assurance
checks made on the system. The MediCal Pro (Barco, USA) QA package is used for these
checks with the luminance response calibrated against the DICOM luminance response. The
CRT monitors are situated in the radiologist reporting room and therefore a low level of
ambient light was set during the c-d scoring. Care was also taken to keep reections on the
CRT screens to a minimum.
When an image is brought up for display on the softcopy station, the whole image is
displayed at reduced resolution (i.e. not at 1 to 1 pixel resolution) so that the 3584 4096
image can be displayed on the monitor. The software supplied includes a magnication
function that allows a section of the image to be viewed at full resolution and this was
used during the scoring. Before scoring the images, window width and level were set to a
satisfactory level by the observers. C-d images tend to have a narrow dynamic range and
therefore fairly narrow window widths (high contrast setting) can be used when scoring this
type of image.
A comparison between objective and subjective image quality measurements for a FFDM system 2449
Contrast-detail models
The experimentally derived c-d data were compared against two observer models. The c-d
results were modelled using the version of signal detection given by Aufrichtig (1999) for the
task of detecting a static, circular disc shaped object imaged against a homogeneous noisy
background. The SNR for this detection task can be written as
SNR =
C
__
|S(u, v) MTF(u, v)O(u, v)|
2
du dv
___
|S(u, v) MTF
2
(u, v)O
2
(u, v)|
2
(q
0
DQE(u, v))
1
du dv
_
1/2
(10)
where S(u, v) is the spectrumof the target object in frequency space, MTF(u, v) and DQE(u, v)
are the imaging system MTF and DQE, respectively and O(u, v) represents the spatial
frequency response function of the observers visual system. For detection to occur according
to this model, the SNRwill have exceeded some threshold criterion for detection, k say, held by
the observer. At this point, the contrast for the signal detection model will be at the threshold
dened as C
TS
C
TS
= k
_
__
|S(u, v) MTF(u, v)O(u/m, v/m)|
2
du dv
___
|S(u, v) MTF
2
(u, v)O
2
(u/m, v/m)|
2
(q
0
DQE(u, v))
1
du dv
_
1/2
_
1
. (11)
The spatial frequency response of the observer was taken from Kelly (1979). We also studied
the effect of setting O(u, v) = 1 (i.e. with no spatial ltering by the observer).
As the images are presented to the observer with some magnication by the soft copy
workstation (SCW), we could model the effect of display magnication factor using the factor
m in equation (11). A typical viewing distance of 25 cm was set for the calculation of C
TS
.
The software magnication function was used when scoring the test objects so that sections
of the image were displayed at a 1 to 1 pixel resolution. When this was engaged, 8 cm in the
image was displayed as 18.4 cm on the screen giving a magnication m=2.3. With the image
displayed at the default size (i.e. without the magnication function), the SCW magnication
was 1.38.
The model of Rose (1948) was used to calculate the threshold contrast C
TR
for a circular
object of diameter using the using the relationship:
C
TR
=
2k

1/2

_
1
q
0
DQE(0)
_
1/2
(12)
where q
0
is the number of photons mm
2
incident on the x-ray detector and DQE(0) is the
detector DQE at zero spatial frequency. Assumptions required for the Rose approach to be
valid include (1) the photons are assumed to be uncorrelated (2) the signal at the detector
is at-topped and has sharp edges (there is no imaging system blur) (Burgess 1999). The
relationship between the signal detection theory given in equation (10) and the theory of Rose
is discussed at length by Burgess (1999).
In equations (11) and (12), k is the observers threshold signal-to-noise ratio (SNR).
Various values of k have been reported in the literature (Rose 1948, Schnitzler 1973, Moran
1982, Ishida et al 1984). k depends on the detection threshold adopted by the observers
during the period of the experiment. Although trained observers can hold this threshold
reasonably constant over a period of time, the actual value adopted by each observer is not
known or controlled in a contrast-detail measurement of this type. In previous studies we have
found that a value of k = 3.8 (Ishida et al 1984) gives calculated results that are close to the
measured results (Marshall et al 2001). A strict observer would demand that the threshold
disc is extremely well visualized (e.g. a clear, continuous edge and well-dened position) and
2450 N W Marshall
(a) (b)
Figure 1. (a) Variance image produced by imaging a 4 cm thick PMMA block at 28 kV, Mo/Mo,
95 mAs using a 10 10 pixel ROI. (b) The same grey scale data plotted as an SNR image.
have a correspondingly high value of k (around 4 or 5). A more lenient observer would be
operating nearer 50% detectability and have a lower value of k (closer to 2 or 3) (Burgess
1999). Changing k shifts the calculated or expected threshold contrasts for the system with
respect to the y-axis. This is a shortcoming of the contrast-detail method used in the current
study with regard to the prediction of absolute threshold contrasts (ICRU 1996). However,
this type of experiment can still provide valuable qualitative information on the performance
of the system as various parameters are changed (e.g., detector input dose).
Results and discussion
Variance
All FFDM systems have a software map containing the locations of defective pixels. On
calibration of the detector by the engineer, some form of correction algorithm is applied
whereby a pixel value (PV) interpolated fromnearest pixel neighbours is given to the defective
pixel. This is a form of averaging, reducing variance locally and should show up as a region
of reduced variance in the variance image. Defective pixels that have not been mapped by
the engineer, on the other hand, may show up as regions of increased variance in the variance
image.
Figure 1(a) shows a variance image for the Selenia system generated at from one image
acquired at 95 mAs (28 kV, Mo/Mo, 4 cm PMMA). Variance is uniform across the central
and chest wall regions of the image, increasing as we move further away from the chest wall
edge. As a result of the heel effect, the number of x-ray photons mm
2
is greater at the chest
wall than at the nipple edge. X-ray photon noise follows a Poisson distribution and therefore
x-ray variance should increase at the chest wall. This is not seen and is possibly a result of the
at eld and detector gain corrections affecting the noise variance. The variance is lower at
A comparison between objective and subjective image quality measurements for a FFDM system 2451
the image detector edges (top and bottom in this gure) and this may be due to reduced noise
from the lower TFT pixel capacitance at the detector edges.
In addition to the variance image, we can also derive a visual assessment of SNRuniformity
for a single image using this method (gure 1(b)). The SNR in this instance was calculated
from the mean pixel value for the ROI divided by the standard deviation of the pixels within
the ROI. This is of interest as the EUREF Digital Addendum gives a limit of 15% for SNR
uniformity calculated from ve regions in the image. The image of the SNR gives a fuller
realization of the SNR variation across a at eld image and shows that SNR increases at the
chest wall. While we can expect excellent pixel value uniformity for a grey scale image as a
result of the at elding process, the SNR will tend to be less uniform (especially in the chest
wall to nipple direction) because of the heel effect. This should be taken into account when
devising QA protocols for these systems.
Finally, it should be noted that this system developed a fault that was seen by the
radiographer during the at elding QA process (note that this study was performed after
the faulty detector was replaced). A spongy artefact was observed at the chest wall edge of
the detector on the grey scale at eld image. This was investigated by the eld engineer
and ascribed to delamination of the a-Se layer and the TFT array. Ultimately, this fault was
traced by the manufacturer to a known set of detectors via the detector serial numberall
detectors with this fault have since been replaced. The detector fault is clearly demonstrated
by the variance image methodsee gure 2. In fact, the variance image indicates a second
artefact region that had not been noted by the radiographer. The variance image also uncovers
a line of dead pixels at the bottom left of the image. Extracts from the greyscale image are
presented to show the bubble-like appearance of the greyscale x-ray image in the fault region.
The variance image is now included as part of the routine physics QA measurements for the
FFDM systems at this site.
Signal transfer property
Pixel value was plotted against air kerma at the detector to give the STP shown in gure 3.
Coefcient of variation for the air kerma at the detector in this measurement was approximately
0.2%. While there is a linear response between PV and air kerma at the receptor, there is
clearly an offset to the STP. The offset is added at calibration by the engineers and is used to
avoid negative pixel values in the image.
Modulation transfer function
Pre-sampling MTFs calculated using median lter lengths of 5 and 7 pixels showed a slight
reduction at high spatial frequencies for the 7 pixel lter and therefore a median lter of
just 5 pixels was applied when calculating MTF in this study. Figure 4(a) shows the pre-
sampling MTF in the data line and gate line directions, along with the pixel aperture MTF for
a pixel size of 0.07 mm. Results for both directions are similar, indicating that pre-sampling
MTF is fairly isotropic. All three curves touch the x-axis at the same value of approximately
14 mm
1
this is an indication that the actual pixel size is close to the nominal pixel size of
0.07 mm (a rst zero of 14.3 mm
1
is expected for a 0.07 mm pixel). The limit caused by the
aperture response can be higher if the dimension of the sensitive region of the pixel (detector
element) is smaller than the nominal value (Rowlands 2000). The measured pre-sampling
MTF is some way below the limit set by the pixel sampling aperture. Yorker et al (2002) and
Zhao et al (2003) show that re-absorption of K-shell uorescence x-rays at a position remote
from the initial interaction is a source of blurring, leading to a reduction in MTF compared to
2452 N W Marshall
line of dead pixels
greyscale extract
greyscale extract
Figure 2. Variance image of a 4 cm PMMA block acquired at 28 kV, Mo/Mo and 60 mAs clearly
demonstrating two regions of severe artefact. The extracted regions show greyscale images of the
artefact. The variance image also highlights a line of dead pixels.
the ideal aperture. Zhao et al (2003) also suggest that charge trapping in the blocking layer
just above the pixel electrodes may also be a source of MTF degradation.
Plotted in gure 4(b) are data taken from Yorker et al (2002) for an earlier version of this
detector. The measured pre-sampling MTF in this study is somewhat lower at all frequencies
compared to the result of Yorker et al (2002). Factors that could account for such a difference
include changes in design parameters, such as the voltage across the a-Se layer. This gure
also presents the pre-sampling MTF given by Zhao et al (2003) for a 200 m thick a-Se
detector with an 85 m pixel size. For comparison with a typical screen-lm detector, the
MTF of a Min-R 2000 mammography screen-lm system (Bunch 1999) is plotted. The MTF
A comparison between objective and subjective image quality measurements for a FFDM system 2453
y = 3.58x + 203
0
200
400
600
800
1000
1200
1400
1600
0 50 100 150 200 250 300 350
air kerma at detector (Gy)
P
i
x
e
l

V
a
l
u
e
grid out 28 kV, Mo Mo 4 cm
PMMA at tube
Figure 3. Signal transfer property for the a-Se detector.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
spatial f requency (mm
-1
)
M
T
F
data line
gate line
aperture
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
spatial f requency (mm
-1
)
M
T
F
data line (this study)
MinR-2000
Yorker et al
Zhao et al data line
(a) (b)
Figure 4. (a) MTFmeasured in the data line (parallel to the chest wall edge) and gate line directions.
Also plotted is the response for a sinc function of 0.07 mm (the pixel size for this detector).
(b) MTF result in data line direction compared against a-Se digital mammography detector data
from Yorker et al (2002) (14.3 mm
1
sampling limit) and Zhao et al (2003) (11.8 mm
1
sampling
limit). Also shown is the MTF for a Min-R 2000 mammography lm/screen system taken from
Bunch (1999).
of the much thicker a-Se layer (200 m versus approx 80 m for Min-R phosphor screen)
is surprisingly close to screen-lm MTF and is due to electric eld applied to the a-Se layer
stopping lateral charge spread. The screen-lm MTF is superior to that of the a-Se layer at
spatial frequencies greater than 12 cycles mm
1
. The 70 m pixel size in the Selenia system
2454 N W Marshall
1.0E-07
1.0E-06
1.0E-05
1.0E-04
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
spatial f requency (mm
-1
)
N
N
P
S
(
u
)

(
m
m
2
)
9.1 Gy 21.0 Gy 42.3 Gy
82.5 Gy 165.3 Gy 330.9 Gy
gate line
Figure 5. Normalized NPS measured in the data line (symbols) and gate line directions (solid
lines) for a range of detector air-kerma settings.
means that only frequencies up to the sampling limit of 7.1 cycles mm
1
can be displayed in
the digital images.
Noise power spectrum
Figure 5 shows NPS in the data line and gate line directions at detector air-kerma values of
9.1, 21.0, 42.3, 82.5 165 and 331 Gy. A detailed exploration of the noise transfer properties
of integrated x-ray detectors (such as that given by Siewardsen et al (1997), Evans et al (2002)
and Zhao et al (2003)) is beyond the scope of the current workjust a few salient points will
be discussed.
The NPS has similar magnitude in both the gate line and data line directions, indicating
good isotropy (as might be expected from the MTF results). In the data line direction, NPS
at the sampling limit has fallen to approximately 60% of the NPS(0) value, while in the gate
line direction, NPS at the sampling limit is roughly 40% of the NPS(0) value. These results
are close to the values given by Zhao et al (2003) for an a-Se detector.
As this is a digital detector, the measured noise power spectrum will suffer from aliasing
because the noise data have been sampled at discrete intervals by the pixel matrix (Giger et al
1984). Williams et al (1999) discuss aliasing of the NPS for digital mammography systems;
detectors that produce images with signicant spatial frequency content above the sampling
limit prior to sampling will suffer from aliasing. Figure 4(a) shows that the pre-sampling MTF
of the a-Se layer has fallen to only 50% at 7.1 mm
1
and hence some aliasing of the NPS
for this detector is expected. Ultimately, aliasing leads to an increase in noise power at high
spatial frequencies.
A comparison between objective and subjective image quality measurements for a FFDM system 2455
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
spatial f requency (mm
-1
)
D
Q
E
(
u
)
9.1 Gy 21.0 Gy 42.3 Gy
82.5 Gy 165.3 Gy 330.9 Gy
Figure 6. DQE in the data line direction measured for a range of detector air-kerma values. X-ray
spectrum was 28 kV, Mo/Mo target lter and 4 cm PMMA added ltration.
Detective quantum efciency
Figure 6 present DQE(u) measured for the data line direction at detector air-kerma values of
9.1, 21.0, 42.3, 82.5 165 and 331 Gy. The curves have a similar shape, regardless of the
air kerma at which they were measured, dropping by a factor of approximately 3 as spatial
frequency changes from 0 mm
1
to 7.1 mm
1
. It is clear that DQE is dependent on air
kerma to the detector. DQE increases for all spatial frequencies as air kerma at the detector
increases, until reaching a plateau above approximately 50 Gy. At low spatial frequency,
DQE increases fromapproximately 0.3 at 9.1 Gy to approximately 0.6 for detector air-kerma
values of 50 Gy and above. The reduction in DQE at low detector air kerma is due to the
presence of additive noise from the input stages of charge amplier read out circuitry (Yorker
et al 2002). Detector air kerma will be low for regions of the breast that have low x-ray
transmission (probably corresponding to areas of highly glandular or dense tissue). In these
regions, it is likely that DQE will be reduced and this may impact on the ability of the system
to detect clinical details in these areas.
Absolute magnitude of the DQE for air-kerma values above 50 Gy is roughly 0.6 at
low spatial frequenciesin good agreement with data published by Yorker et al (2002) and
Zhao et al (2003) for similar detectors. DQE for this a-Se detector is therefore considerably
greater than the gure of 0.35 (at low frequencies) for a well-established mammography
screen/lm (S/F) system such as Kodak Min-R 2000 (Bunch 1999).The advantage of the
S/F system is that DQE extends above the 7 mm
1
limit imposed by the sampling matrix
for the Selenia system (albeit at the low value of 0.1). As it is thought that detectability
for a given task is related to the DQE (ICRU 1996), we should expect improved detection
performance compared to S/F detectors (certainly for well-dened signal known exactly,
background known exactly (SKE/BKE) tasks). It should also be noted that DQE is calculated
using the pre-sampling MTF in combination with the digital (sampled) NPS (equation (4)).
As noted above, aliasing results in an increase in the high frequency NPS and hence DQE
calculated in this manner may be an underestimate of the true DQE at high frequencies.
2456 N W Marshall
0.1
1.0
10.0
0.01 0.10 1.00 10.00
detail diameter (mm)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
9.1 Gy
theory
42.3 Gy
330.9 Gy
0.1
1.0
10.0
100.0
0.01 0.10 1.00 10.00
detail diameter (mm)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
11.9 Gy
theory
41.0 Gy
342 Gy
Figure 7. Observer c-d results at three detector air-kerma settings (symbols) and c-d curves
calculated according to the Rose theory.
Table 2. DQE(0) found by extrapolating DQE(u) back to the y-axis.
Air kerma at detector (Gy) DQE(0)
9.1 0.29
21.0 0.43
42.3 0.54
82.5 0.61
165 0.62
331 0.59
Contrast-detail results and the Rose model
The observer c-d results are shown in gure 7; for reasons of clarity, only data for detector
air-kerma values of 11.9, 41.0 and 342 Gy are plotted out of the six sets of data available.
Note that for the 342 Gy results, all the discs present in the row for diameters of 1.0 mm
and above were seen and these threshold contrast points have also been excluded from the
plot. The errors bars indicate an uncertainty of 15% (Cohen et al 1984), estimated for two
observers reading two images once
Also plotted are the expected c-d curves calculated from Rose theory (equation (12)) at
air-kerma values of 11.9, 41.0 and 342 Gy. Two assumptions were made regarding DQE
for this calculation. The rst is a practical pointwe are using DQE results obtained at
slightly different detector air-kerma values (9.1, 42.3 and 331 Gy) for the calculation. DQE
is changing quite quickly for the low air-kerma results (11.9 Gy) and this 2 Gy difference
will introduce some error. Above 50 Gy at the detector, DQE reaches a plateau and changes
more slowly with air kerma and virtually no error is introduced by this assumption. Second,
in order to nd DQE(0), a rst-order linear extrapolation to DQE(u) was used (table 2). This
is because the measured value of DQE is often affected by low frequency artefacts present in
the image (Dobbins 2000).
In order to nd a value for k, this equation was tted to the observer threshold contrast
results for the centre set of air-kerma results (42.3 Gy), for detail sizes ranging from 2.0 mm
A comparison between objective and subjective image quality measurements for a FFDM system 2457
0.1
1.0
10.0
100.0
1.0 10.0 100.0 1000.0
air kerma at detector (Gy)
C
t

(
%
)
0.80 mm
limiting DQE=0.6
CTR
(a)

0.1
1.0
10.0
100.0
1.0 10.0 100.0 1000.0
air kerma at detector (Gy)
C
t

(
%
)
0.20 mm
limiting DQE=0.6
CTR
(b)
0.1
1.0
10.0
100.0
1.0 10.0 100.0 1000.0
air kerma at detector (Gy)
C
t

(
%
)
0.13 mm
limiting DQE=0.6
CTR
(c)
Figure 8. Comparison of observer threshold contrast results (squares) and Rose threshold contrast
calculated using DQE(0) (circles) for (a) 0.80 mm, (b) 0.20 mm and (c) 0.13 mm diameter discs.
Dotted line shows result for limiting DQE of 0.60.
down to 0.13 mm. This gave a value of k = 2.56 and therefore an approximate value of 2.5
was used for k for all calculations of C
TR
and C
TS
in this paper.
The model gives results within approximately 15% for the c-d curve with changing
detector air kermathe spread between the low and high air-kerma results simply reects the
changes in quantum noise in the image. The CDMAM test object is sensitive enough (in terms
of percentage contrast step between consecutive discs) to follow these changes in quantum
mottle.
Calculated and measured results were also within approximately 15% for disc diameters
down to 0.2 mm, however there is considerable discrepancy for diameters less than 0.20 mm.
This is examined more closely in gure 8, where the threshold contrast data for a particular
detail diameter have been plotted as a function of detector air kerma. In order to smooth the
measured c-d data, a second-order least-squares t was made to all the c-d curves. Data plotted
in gure 8 are contrasts taken fromthe tted curves rather than the rawthreshold contrast result.
The dotted curve in gure 8(a) is the Rose threshold contrast (C
TR
) calculated for
DQE = 0.6 (we are using this as the approximate limiting value of the DQE) and 0.8 mm
2458 N W Marshall
0.1
1.0
10.0
100.0
0.01 0.10 1.00 10.00
diameter (mm)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
11.9 Gy
41.0 Gy
342 Gy
theory
Figure 9. Observer c-d results at three detector air-kerma settings (symbols) and c-d curves
calculated using signal detection model (equation (11)) with observer visual response (O(u, v)) =
1 and system magnication m = 1.
detail diameter. The measured threshold contrast is well modelled by the theory of Rose
using DQE(0) with an average difference between measurement and theory of 10%. At the
two lowest air-kerma values studied, the DQE drops due to the presence of detector amplier
noise and this clearly results in an increase in threshold contrast. As air kerma increases,
measured and calculated threshold contrast approach the limiting value determined by DQE
of 0.6. Figure 8(b) shows that similar results are found for the 0.2 mm diameter discs.
At the smallest disc diameters (0.13 mm and 0.10 mm), however, the difference between
the measured result and the contrast calculated using DQE(0) is approximately 40% (see
gure 8(c)). One possible reason for this is that the observer visual response (not included
in equation (6)) is starting to limit threshold contrast. There will be some inuence of the
visual response (especially for small detail sizes) on the measured c-d results but the use of
carefully controlled viewing conditions and the software magnication function on the SCW
should keep this to minimum. Another reason might be that detection of such small details
uses a higher spatial frequency region of the DQE (ICRU 1996).
Threshold contrast follows the expected Rose relationship, even for high air-kerma values
at the detector. A previous study (Marshall et al 2001) has discussed the inuence of xed
pattern noise in the form of phosphor structure mottle (Kume et al 1986) on native digital
uorography images. This can limit the threshold contrast resolution of an x-ray system at
high detector air-kerma values. It is clear that is not the case for the a-Se detector in this
system; as expected the a-Se x-ray converter does not suffer from the phosphor granularity
noise that can limit contrast resolution. Low frequency DQE remains close to 0.6, even at
high detector air-kerma values.
Contrast-detail results and signal detection theory model
Threshold contrast (C
TS
) was calculated using equation (11) for detector air-kerma values of
9.1, 42.3 and 331 Gy. Although this equation uses the 2D forms of MTF, NPS and DQE, the
A comparison between objective and subjective image quality measurements for a FFDM system 2459
0.1
1.0
10.0
100.0
1.0 10.0 100.0 1000.0
air kerma at detector (Gy)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
0.80 mm
Rose C%limiting DQE=0.6
signal detection theory
0.1
1.0
10.0
100.0
1.0 10.0 100.0 1000.0
air kerma at detector (Gy)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
0.20 mm
Rose C%limiting DQE=0.6
signal detection theory
0.1
1.0
10.0
100.0
1.0 10.0 100.0 1000.0
air kerma at detector (Gy)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
0.13 mm
Rose C%limiting DQE=0.6
signal detection theory
(a) (b)
(c)
Figure 10. Comparison of observer threshold contrast data (squares) and signal detection model
(triangles) for (a) 0.80 mm, (b) 0.20 mm and (c) 0.13 mm diameter discs. Dotted line shows result
for limiting DQE of 0.60. Signal detection results were calculated with observer visual response
(O(u, v)) = 1 and system magnication m = 1.
results demonstrated reasonably good isotropy and therefore just the results from the data line
direction were used to simplify the calculations. When calculating threshold contrast we set
system magnication m = 1 and O(u) = 1, assuming initially that observers could optimize
their viewing conditions and that the effect of the display/perception would be negligible.
A similar approach was adopted by Workman and Cowen (1993) for computed radiography
systems. Figure 9 plots the observed c-d data along with C
TS
calculated from equation (11)
with an observer spatial frequency response O(u) = 1, m = 1 and k = 2.5. Again, only
data for detector air-kerma values of 9.1, 41.0 and 331 Gy are plotted. Two points can be
made here. First, the calculated curves follow the measured data closely as air kerma at the
detector is changed. Second, the full spatial frequency range DQE is used in the calculations
and therefore somewhat better agreement is seen at small disc sizes compared to the use of
DQE(0) in the Rose theory calculations. For example, the difference between measured and
2460 N W Marshall
0.1
1.0
10.0
100.0
0.01 0.10 1.00 10.00
diameter (mm)
t
h
r
e
s
h
o
l
d

c
o
n
t
r
a
s
t

(
%
)
11.9 Gy
41.0 Gy
342 Gy
theory
Figure 11. Comparison of observer c-d data (symbols) and signal detection model calculated
with system magnication m=1.38, typical observer visual response function O(u, v) taken from
Kelly (1979) and a viewing distance of 25 cm.
calculated data for the 0.13 mm disc at 342 Gy for the detection theory calculation is 26%
compared to a difference of 44% for the Rose theory result.
Figure 10 examines signal detection model results as a function of air kerma at the detector.
Figures 10(a) and (b) showthe measured threshold contrast data along with calculated contrast
C
TS
plotted as a function of air kerma at the detector for 0.8 mm and 0.2 mm discs. For these
diameters, the difference between measured and calculated threshold contrasts was 8% on
average. Figure 10(c) plots the results for 0.13 mm diameter discs where the difference
between measured and calculated results is 20%. There are several possible reasons for this
discrepancy. For example, the MTF and NPS data were acquired under a low scatter geometry
while scattered radiation was present at the c-d acquisition stage. It is possible that the
observers are using a different detection threshold for the smallest discs. Another possibility
is that the observers visual system spatial frequency response is starting to inuence the
results. We should also consider the possibility that internal visual system noise may be
affecting observer performance (Ishida et al 1984, Abbey and Bochud 2000) for detail sizes
less than 0.2 mm.
Finally, C
TS
was calculated using a spatial frequency response curve for a typical human
observer (Kelly 1979), with a viewing distance of 25 cm and a magnication factor m of 1.38
(magnication for an image displayed at the default size)see gure 11. For these conditions,
the observer spatial frequency response used increased from 0.14 at 0.1 mm
1
to a value of
1.0 at 0.67 mm
1
and then fell to 1.70 10
4
at 5 mm
1
. This gives the characteristic bow
often seen in c-d curves, especially for larger detail diameters. Agreement is now closer
at the smallest disc diameters, however, it should be noted that the observed c-d data were
acquired using the software magnication function and hence mshould be 2.3. Setting m=2.3
in equation (11) gives values for C
TS
that are too low at small diameters. There was no
signicant difference (within experimental error) in the observed threshold contrast data
when measured with and without the software magnication tool on this system. Further
modication of equation (11) appears necessary in order to correctly model the effect of the
A comparison between objective and subjective image quality measurements for a FFDM system 2461
observer visual system and system display magnication on these c-d curves. These results
highlight the difculty in using objective measurements such as NPS to model/predict absolute
values, even for the simple c-d task. The objective measurements are an evaluation of detector
performance alone, while the observer threshold contrast results include effects of observer
detectability, observer visual response, display magnication and processing, ambient viewing
conditions including display reection and scattered radiation/grid performance
Conclusions
This study used objective and subjective measures to evaluate the image quality of an a-Se
digital mammography system. The objective analysis began with the generation of variance
images from at eld images. These images were useful in obtaining a visual impression
of the variance across the entire detector before proceeding to perform NPS analysis on the
images. The variance image also clearly identied a severe detector artefact and a line of
dead pixels not included in the system dead pixel map. Results from variance images were
then used to generate SNR images which showed the varying x-ray SNR across image due
to the heel effect, as would be expected for a typical quantum noise dominated x-ray system.
This variation in x-ray SNR should be taken into account when deriving performance limits
in testing protocols.
Both pre-sampling MTF and axially averaged NPS were reasonably isotropic. At the
sampling limit (7.1 mm
1
), the NPS fell to approximately 60% and 40% of the NPS(0) value,
for the data line and gate line directions, respectively. DQE calculated from these parameters
was approximately 0.6 at low spatial frequencies. DQE was also found to change with air
kerma at the image receptor, increasing from 0.3 (at low spatial frequencies) at 9.1 Gy to
approximately 0.6 for air-kerma settings of 50 Gy and above.
Subjective image quality results in the form of c-d measurements were then obtained
at the same detector air-kerma settings used for the objective analysis. The measured c-d
results reected the reduction in DQE found at the low detector air-kerma setting; c-d data
also followed the changes in detector air kerma/image even for the relatively high air kerma of
342 Gy/image. Two models (the Rose model and a version of signal detection theory) were
used to examine the measured c-d data. The Rose model (using DQE(0)) gave results within
approximately 15% for all the detector air-kerma values studied and for detail diameters down
to 0.2 mm. Similar agreement (approximately 15%) was also found between the measured
c-d data and signal detection theory results, which were calculated with an ideal human visual
response function and a system magnication of unity. Using the full spatial frequency DQE
improved the agreement between the calculated and observer results for detail sizes below
0.13 mm.
This study has demonstrated close correspondence, in functional terms, between objective
detector image quality parameters calculated directly fromimage data and subjective (contrast-
detail) image quality results. As QA physicists gain experience with objective image quality
analysis over a wider range of digital mammography systems, standards can be formed using
these objective parameters. Eventually, we may hope to performquality assurance of the x-ray
detector using measured image quality data rather than a carefully controlled, but ultimately
limited subjective task such as the contrast-detail method.
Acknowledgment
I would like to thank Dr Julie Horrocks for her help in preparing this work.
2462 N W Marshall
References
Albert M and Maidment A D 2000 Linear response theory for detectors consisting of discrete arrays Med. Phys.
27 241734
Aufrichtig R 1999 Comparison of low contrast detectability between a digital amorphous silicon and a screenlm
based imaging system for thoracic radiography Med. Phys. 26 134958
Burgess A E 1999 The Rose model, revisited J. Opt. Soc. Am. A 16 63346
Bunch P C 1999 Advances in high-speed mammographic image quality Proc. SPIE 3659 12030
CEP Report 05084 2006 Full Field Digital Mammography Systems Hologic Selenia: A Technical Report (London:
HMSO)
Cohen G, McDaniel D L and Wagner L K 1984 Analysis of variations in contrast-detail experiments Med. Phys.
11 46973
Cowen A R, Brettle D S, Coleman N J and Parkin G J 1992 A preliminary investigation of the imaging performance
of photostimulable phosphor computed radiography using a new design of mammographic quality control test
object Br. J. Radiol. 65 52835
Cowen A R, Haywood J M, Workman A and Clarke O F 1987 A set of X-ray test objects for image quality control in
digital subtraction uorography: I. Design considerations Br. J. Radiol 60 10019
Cranley K, Gilmore B J, Fogarty G W A and Desponds L 1997 Catalogue of diagnostic x-ray spectra and other data
IPEM Report vol 78 (York: Institute of Physics and Engineering in Medicine)
Cunningham I A 2000 Applied linear systems theory Physics and Psychophysics Vol 1 of Handbook of Medical
Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 79159
Cunningham I A and Shaw R 1999 Signal-to-noise optimization of medical imaging systems J. Opt. Soc. Am. A 16
113
Dainty J C and Shaw R 1974 Image SciencePrinciples, Analysis and Evaluation of Photographic-Type Imaging
Processes (London: Academic)
Dance D R and Day G J 1984 The computation of scatter in mammography by Monte Carlo methods Phys. Med.
Biol. 29 23747
Dobbins J T III 2000 Image quality metrics for digital systems Physics and Psychophysics Vol 11 of Handbook of
Medical Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 161222
Dobbins J T III, Ergun D L, Rutz L, Hinshaw D A, Blume H and Clark D C 1995 DQE(f) of four generations of
computed radiography acquisition devices Med. Phys. 22 158193
EUREF Digital Addendum 2003 Addendum on Digital Mammography to Chapter 3 of the European Guidelines for
Quality Assurance in Mammography Screening (Version 1.0, November 2003) (Nijmegen, the Netherlands:
European Reference Organisation for Quality Assured Breast Screening and Diagnostic Services (EUREF))
Evans D S, Workman A and Payne M 2002 A comparison of the imaging properties of CCD-based devices used for
small eld digital mammography Phys. Med. Biol. 47 11735
Flynn M J and Samei E 1999 Experimental comparison of noise and resolution for 2 k and 4 k storage phosphor
radiography systems Med. Phys. 26 161223
Fujita H, Tsai D-Y, Itch T, Doi K, Morishita J, Ueda K and Ohtsuka A 1992 A simple method for determining the
modulation transfer function in digital radiography IEEE Trans. Med. Imaging 11 3439
Giger M L, Doi K and Metz C E 1984 Investigation of basic imaging properties in digital radiography: 2. Noise
Wiener spectrum Med. Phys. 11 797805
Hanson K M 1983 Variations in task and the ideal observer Proc. Soc. Photo-Opt. Instrum. Eng. 419 6067
Hay G A, Clarke O F, Coleman N J and Cowen A R 1985 A set of X-ray test objects for quality control in television
uoroscopy Br. J. Radiol. 58 33544
Hillen W, Schiebel U and Zaengel T 1987 Imaging performance of a digital storage phosphor system Med. Phys.
14 74451
ICRU 1996 Medical imagingthe assessment of image quality ICRU Report 54 (Bethesda, MD: International
Commission on Radiation Units and Measurements (ICRU))
IPEM 2005 Commissioning and routine testing of mammographic X-ray systems IPEM Report 89 ed A C Moore,
D R Dance, D S Evans, C P Lawinski, E M Pitcher, A Rust and K C Young (York: IPEM)
Ishida M, Doi K, Loo L-N, Metz C E and Lehr J L 1984 Digital image processing: effect on detectability of simulated
low-contrast radiographic patterns Radiology 150 56975
Judy P F 1976 The line spread function and modulation transfer function of a computed tomographic scanner Med.
Phys. 3 2336
Kelly D H 1979 Motion and vision II stabilized spatio-temporal threshold surface J. Opt. Soc. Am. 69 13409
Kume Y, Doi K, Ohara K and Giger M L 1986 Investigation of basic imaging properties in digital radiography: 10.
Structure mottle of II-TV digital imaging systems Med. Phys. 13 84349
A comparison between objective and subjective image quality measurements for a FFDM system 2463
Maidment A D and Albert M 2003 Conditioning data for calculation of the modulation transfer function Med.
Phys. 30 24853
Maidment A D A, Albert M, Bunch P C, Cunningham I A, Dobbins J T, Gagne R M, Nishikawa R M, Wagner R F
and Van Metter R L 2003 Standardization of NPS measurement: Interim Report of AAPM TG16 Proc.
SPIE 5030 52332
Marshall N W, Kotre C J, Robson K J and Lecomber A R 2001 Receptor dose in digital uorography: a comparison
between theory and practice Phys. Med. Biol. 46 128396
Metz C E, Wagner R F, Doi K, Brown D G, Nishikawa R M and Myers K J 1995 Toward consensus on quantitative
assessment of medical imaging systems Med. Phys. 22 105761
Moran P R 1982 A physical statistics theory for detectability of target signals in noisy images: I. Mathematical
background, empirical review and development of theory Med. Phys. 9 40113
Rogge F, Bosmans H, Willems P and Marchal G 2003 Use of MTF calculation in global and local resolution
assessment in digital mammography Proc. SPIE 5030 896907
Rose A 1948 The sensitivity performance of the human eye on an absolute scale J. Opt. Soc. Am. 38 196208
Rowlands J A and Yorkston J 2000 Flat panel detectors for digital radiography Physics and Psychophysics Vol 1 of
Handbook of Medical Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 223328
Samei E and Flynn M J 2003 An experimental comparison of detector performance for direct and indirect digital
radiography systems Med. Phys. 30 60822
Samei E, Flynn M J and Reimann D A 1998 A method for measuring the presampled MTF of digital radiographic
systems using an edge test device Med. Phys. 25 10213
Siewerdsen J H and Antonuk L E 1998 DQE and system optimization for indirect-detection at-panel imagers in
diagnostic radiology Proc. SPIE 3336 54655
Siewerdsen J H, Antonuk L E, el-Mohri Y, Yorkston J, Huang W, Boudry J M and Cunningham I A 1997 Empirical
and theoretical investigation of the noise performance of indirect detection, active matrix at-panel imagers
(AMFPIs) for diagnostic radiology Med. Phys. 24 7189
Schnitzler A D 1973 Image-detector model and parameters of the human visual system J. Opt. Soc. Am 63 135768
Sones R A and Barnes G T 1984 A method to measure the MTF of digital x-ray systems Med. Phys. 11 16671
Stierstorfer K and Spahn M 1999 Self-normalizing method to measure the detective quantum efciency of a wide
range of x-ray detectors Med. Phys. 26 13129
Wagner R F and Brown D G 1985 Unied SNR analysis of medical imaging systems Phys. Med. Biol. 30 489518
Williams M B, Mangiaco P A and Simoni P U 1999 Noise power spectra of images from digital mammography
detectors Med. Phys. 26 127993
Workman Aand Cowen AR1993 Signal, noise and SNRtransfer properties of computed radiography Phys. Med. Biol.
38 1789808
Yorker J G, Jeromin L S, Lee D L Y, Palecki E F, Golden K P and Jing Z 2002 Characterization of a full eld digital
mammography detector based on direct x-ray conversion in selenium Proc. SPIE 4682 219
Zhao W, Ji W G, Debrie A and Rowlands J A 2003 Imaging performance of amorphous selenium based at-panel
detectors for digital mammography: characterization of a small area prototype detector Med. Phys. 30 25463