Home Search Collections Journals About Contact us My IOPscience

4D MR imaging of respiratory organ motion and its variability

This content has been downloaded from IOPscience. Please scroll down to see the full text.

2007 Phys. Med. Biol. 52 1547

(http://iopscience.iop.org/0031-9155/52/6/001)

View the table of contents for this issue, or go to the journal homepage for more

Download details:

IP Address: 131.215.225.9
This content was downloaded on 28/09/2013 at 11:50

Please note that terms and conditions apply.

IOP PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 52 (2007) 1547–1564 doi:10.1088/0031-9155/52/6/001

4D MR imaging of respiratory organ motion and its

variability
M von Siebenthal1, G Székely1, U Gamper2, P Boesiger2, A Lomax3
and Ph Cattin1
1 Computer Vision Laboratory, ETH Zurich, 8092 Zurich, Switzerland
2 Institute for Biomedical Engineering, ETH Zurich, 8092 Zurich, Switzerland
3 Paul Scherrer Institute, 5232 Villigen PSI, Switzerland

E-mail: mvonsieb@vision.ee.ethz.ch, szekely@vision.ee.ethz.ch and cattin@vision.ee.ethz.ch

Received 29 June 2006, in final form 3 January 2007

Published 16 February 2007
Online at stacks.iop.org/PMB/52/1547

Abstract
This paper describes a method for 4D imaging, which is used to study
respiratory organ motion, a key problem in various treatments. Whilst the
commonly used imaging methods rely on simplified breathing patterns to
acquire one breathing cycle, the proposed method was developed to study
irregularities in organ motion during free breathing over tens of minutes. The
method does not assume a constant breathing depth or even strict periodicity
and does not depend on an external respiratory signal. Time-resolved 3D image
sequences were reconstructed by retrospective stacking of dynamic 2D images
using internal image-based sorting. The generic method is demonstrated for
the liver and for the lung. Quantitative evaluations of the volume consistency
show the advantages over one-dimensional measurements for image sorting.
Dense deformation fields describing the respiratory motion were estimated
from the reconstructed volumes using non-rigid 3D registration. All obtained
motion fields showed variations in the range of minutes such as drifts and
deformations, which changed both the exhalation position of the liver and the
breathing pattern. The obtained motion data are used in proton therapy planning
to evaluate dose delivery methodologies with respect to their motion sensitivity.
Besides this application, the new possibilities of studying respiratory motion
are valuable for other applications such as the evaluation of gating techniques
with respect to residual motion.
(Some figures in this article are in colour only in the electronic version)

1. Introduction

Respiratory organ motion is a complicating factor in many treatments. One example is

radiation therapy, where respiratory motion can lead to substantial degradation of the dose
0031-9155/07/061547+18$30.00 © 2007 IOP Publishing Ltd Printed in the UK 1547
1548 M von Siebenthal et al

distribution (Lambert et al 2005). The existing approaches to take respiratory motion into
account include choosing larger safety margins to ensure target coverage, voluntary or forced
breath-hold, gating, or tracking of the target. These methods were detailed and compared
in recent publications (Keall et al 2006, Webb 2006). An important prerequisite to evaluate
these methods and to plan treatments in the presence of motion is a detailed knowledge of
the motion and its impact on the resultant dose distribution. Especially for dynamic dose
delivery such as spot scanning in proton therapy (Pedroni et al 1995, Lomax 1999), where
complex interferences between respiratory motion and the treatment pattern are possible, exact
knowledge of organ motion and its influence on the dose distribution is crucial (Gueulette et al
2005, Lambert et al 2005).
A number of groups have looked into the possibilities of 4DCT for this problem, where
multiple dose calculations are performed on time-resolved 3D data sets and then ‘warped’
back into the reference data set (Rietzel et al 2005). However, if one can generate individual
motion vectors for regularly spaced grid points within the patient, then an alternative approach
is to modify the dose calculation in the treatment planning system to deal with the motion of
these points. For example, if one knows when particular portions of a field will be applied,
and one also knows the time-varying position of the individual dose calculation points, one
can directly calculate time-resolved dose distributions in a single dose calculation. The aim
of this work is to develop a 4D imaging method, from which realistic point-by-point motion
vectors can be produced for subsequent use in such a time-resolved dose calculation algorithm
for dynamic proton therapy.
Although the developed technique is applicable to any organ affected by respiratory
motion such as the lung or the kidneys, the details of the method will be discussed for the liver
as an example organ. The respiratory motion of the liver has been studied in numerous works
(Clifford et al 2002, Davies et al 1994). The main component of this motion is a cranio–
caudal shift, usually in the range of 0.5–2.5 cm. As quantified in Rohlfing et al (2001), the
liver additionally shows motion in anterior–posterior (1–12 mm) and the left–right direction
(1–3 mm) as well as non-rigid deformations (up to 2 cm).
Besides the assessments of typical respiratory motion, it is also important to have
knowledge about the reproducibility of breathing. Amplitude, frequency and shape of the
respiratory motion do not only differ between subjects but may vary for a single subject within
short time scales, even with acoustic patient coaching (Nehmeh et al 2004). Phillips et al
(1992) already showed the large influence of sinusoidal motion on the dose distribution for
scanned proton therapy. Thus, it might be dangerous to calculate safety margins and optimize
a treatment plan assuming a certain breathing pattern that might not be representative for the
motion during treatment. One possibility to incorporate the variability of breathing into dose
calculations is to use probability density functions (Engelsman et al 2005), which are, however,
often difficult to determine. Therefore, an important requirement for the developed technique
was the capability to handle and visualize the variability of organ motion. Whilst most imaging
methods were developed for diagnostic purposes and acquire only a few breathing cycles, the
MR approach described in this work is intended to acquire detailed 4D data over tens of
minutes in order to study this intra-subject variability, which is still poorly understood. A
direct application of such realistic data of organ motion and its variability is the evaluation
of gating techniques. For example, detailed motion data can be used to assess the efficacy of
gating techniques that are applicable during treatment, but rely on simplifying assumptions
about the breathing pattern.
The first step towards a better understanding of respiratory motion was to capture the
behaviour of the liver during free breathing in time-resolved volumetric MR images (4D
sequences). In contrast to 4DCT, MRI provides excellent soft tissue contrast, allows for
4D MRI of respiratory organ motion 1549

greater flexibility than CT in selecting image plane positions and orientations and lastly,
volunteers and patients undergoing such examinations are not exposed to ionizing radiation.
Nevertheless, 4DCT is also considered in the following literature overview, because some
synchronization methods are common for MRI and CT.
MRI techniques are constantly evolving, but the unavoidable trade-off between resolution,
acquisition speed and signal-to-noise ratio (SNR) still prevents a detailed examination of
respiratory motion in real time. Dynamic 3D MR sequences such as single shot echo planar
imaging (SSh-EPI) applied in functional MRI achieve high acquisition frequencies, but suffer
from a poor SNR efficiency and reduced resolution if large regions like the lung or the liver
are scanned. Furthermore, these sequences are prone to image artefacts at tissue boundaries
and are therefore less favourable for abdominal imaging. Sequences with high SNR efficiency
and image quality such as steady state free precession (SSFP) achieve only low temporal
resolutions. This is due to the fact that MR data acquisition is a sequential procedure, i.e. the
experiment has to be repeated for every spatial encoding step. The total acquisition time for
one 3D acquisition therefore is the product of this repetition time and the number of encoding
steps. The repetition time for this imaging technique is in the order of milliseconds and cannot
be reduced further due to safety regulations. Since a typical abdominal 3D data set requires
a few thousand encoding steps and an acquisition time in the order of seconds, it is currently
not possible to acquire high resolution 4D data sets without severely compromising image
quality. To overcome this trade-off, there are two main approaches, here denoted as breath-
hold imaging and slice stacking. Breath-hold imaging acquires static 3D volumes to avoid
motion artefacts. The region of interest is scanned at different breathing depths determined for
example with a stretch transducer (Rohlfing et al 2001) or a standard MR navigator (Wang et al
1996). One problem is that subjects may not be able to hold their breath for long acquisition
times, which range up to 40 s (Blackall 2002). Additionally, breath-hold images do not capture
how the organ is deformed during free breathing (Blackall et al 2004). Even if the breath-hold
positions lie within the range of free breathing, the dynamic behaviour of the organ and its
different deformation between inhalation and exhalation cannot be observed.
In slice stacking, 2D slices of a volume are scanned separately during a number of
breathing cycles. Frames from these 2D sequences are retrospectively stacked to 3D
images. Commonly used slice stacking methods apply various measures to find temporally
corresponding 2D images. Examples are the tidal lung volume (Low et al 2003), infrared
reflective markers placed on the abdomen (Ford et al 2003, Pan et al 2004, Keall et al 2004) or
the abdominal skin extracted from CT (McClelland et al 2004). All sorting methods that are
commonly used make strong assumptions about the regularity of the respiratory motion and
parameterize this motion with a one-dimensional amplitude or phase. Although respiration
clearly shows a repetitive character, which is also exploited in this work, the reduction of the
respiratory organ deformation to one parameter neglects all residual variability and may be
too coarse an approximation in some cases. This also leads to artefacts in the reconstructed
images (Keall et al 2004, Rietzel et al 2004). The purpose of the proposed imaging method
is to overcome this problem and specifically study irregularities in respiratory motion. The
proposed technique originates from a method we presented earlier (von Siebenthal et al 2005)
and includes quantitative evaluations as well as improvements in the acquisition sequence.
The method described here reconstructs images with high contrast between blood vessels
and the surrounding tissue. From these images, the deformation between different states
in a respiratory cycle can be determined using intensity-based non-rigid registration. The
algorithm for non-rigid registration that is used in this paper was proposed by Rueckert et al
(1999) and Schnabel et al (2003). The resulting deformations define the trajectories of mass
points during respiration and can provide a basis for motion analysis. A similar free-form
1550 M von Siebenthal et al

1 D1
2 D2 3 2 1
exh.
3 D3 2 1

breathing depth
1
N 1 2 3

schematic
3

1 3

3
2 3 2
1 1
sup. inh.
2

ant. left
(a) (b) time

Figure 1. (a) Sagittal slices covering the volume of interest. One dedicated slice N is used as
navigator slice for image sorting. (b) Interleaved acquisition of data and navigator frames.

registration was applied by Rohlfing et al (2001) to estimate respiratory motion from breath-
hold images of the liver.

2. Materials and methods

2.1. Data acquisition

We propose an imaging method, which follows the slice stacking approach and captures the
respiratory motion during free breathing using dynamic 2D MR images. In contrast to all
techniques cited above, a sagittal slice orientation was chosen. This allows us to track vascular
structures during complete breathing cycles with minimal out-of-plane motion. Moreover, the
sagittal slice orientation requires the smallest field of view and therefore leads to the fastest
acquisition. The volume of interest was covered by sagittal slices, further called data slices
p
Di with p being the position of the sagittal slice and i the acquisition time. The key concept
of the proposed method is to additionally acquire a dedicated so-called navigator slice Nj at a
fixed position, with j indicating the acquisition time (figure 1(a)). This navigator slice will be
used retrospectively to derive a measure for image sorting that determines the state of the liver
on a certain data frame. The medio-lateral position of the navigator slice was chosen such as
to contain vessel cross-sections that are distributed over the entire slice of the liver and which
are easily trackable. This is often the case for orthogonal vessel cross-sections, which can be
found in most sagittal slices. However, vessels that lie completely within a slice may show
large shape changes due to out-of-plane motion. A second criterion for the selection of the
navigator slice was to choose a central slice in order to avoid unnecessarily large distances to
the data slices. Nevertheless, the method has been found to be insensitive to the exact choice
of the navigator position, as
will be shown in section 3.3. 
1 2 3
An alternating scheme Ni –Di+1 –Ni+2 –Di+3 –Ni+4 –Di+5 – . . . was used to acquire data
frames at all slice positions interleaved with navigator frames (figure 1(b)). Such sequences
were acquired on a 1.5 T Philips Achieva whole body MR system (Philips Medical Systems,
Best, NL) with a balanced steady state free precession sequence, SENSE factor 1.7 and
halfscan, flip angle 70◦ , TR = 3.1 ms, with a coil array consisting of four rectangular
elements, and an in-plane resolution of 192 × 192 pixels and 1.8 × 1.8 mm2 . Depending on
the scanned volume, 22–30 slices were acquired with an out-of-plane resolution of 3–4 mm.
The resulting acquisition time per frame was 180–190 ms, yielding a temporal resolution of
4D MRI of respiratory organ motion 1551

3 2 1
1 exh.
2
1

breathing depth
1 2 3
3

schematic
1 3

3
2 3 2
1 1
inh.
2
(a)
time
exh.

breathing depth
comparison
Nj+1

schematic
Ni+1
p Dp,i q Dq,j
comparison

Ni-1 Nj-1 inh.

(b)
i j time

Figure 2. (a) Triplets consisting of data frames and their embracing navigator frames.
(b) The correspondence of two data frames acquired at different slice locations is determined
by comparing the embracing navigator frames. The example navigator frame shows four regions
that were considered for image sorting.

2.6–2.8 Hz for the data slices. At each slice position, 180–240 data frames were acquired
during roughly 1 h.

2.2. Retrospective sorting of 2D images

In order to produce 3D volumes from the acquired 2D images, we have to find those frames
that show the liver in a similar state and which can be stacked together. As discussed above, the
commonly used approach to identify temporally corresponding images is to derive the position
or the phase in a periodic breathing cycle based on a one-dimensional measure such as the
superior–inferior position of the diaphragm. But if the respiratory motion shows different
breathing depths, the phase or the amplitude is not sufficient to determine the shape and the
position of the liver. For example, mid inhalation of a deep breathing cycle may show the
same diaphragm position as maximum inhalation during shallow breathing, where the liver
generally has a different shape. Moreover, the liver may be deformed or tilted during an
acquisition session. Since we explicitly want to study such variations, we further avoid the
notion of the phase in the breathing cycle, which often implies perfect periodicity. Instead,
we denote a certain position and shape of the liver as its state. In general, this state can
be determined only approximately from a single one-dimensional signal. Consequently, a
temporal (one-dimensional) sorting or binning of the acquired images is not sufficient to find
corresponding images showing the same state of the liver. Therefore, a more complex measure,
which considers several regions of the liver, was derived from the acquired navigator images.
The temporal correspondence of data frames is determined by comparing the embracing
navigator frames that were acquired immediately before and after the respective data frames in
the interleaved sequence (figure 2(a)). The underlying assumption is that two data frames show
the same state of the liver, if their embracing navigator frames are sufficiently similar. Since
1552 M von Siebenthal et al

only navigator frames are directly compared and not the data frames themselves, different
image content in distant data slices has no negative effect. Moreover, any difference in
amplitude, phase or the amount of deformation between data and navigator frames can be
handled as long as the state of the liver can be unambiguously defined by the preceding and
the subsequent navigator frames. This assumption will be evaluated in section 3.3.

2.2.1. Frame similarity measure. To describe the similarity measure for data slices in detail,
p
we consider a data frame Di that was acquired at location p and time i (figure 2(b)). Its
embracing navigator frames are Ni−1 and Ni+1 . To find a temporally corresponding data frame
at a different slice position q, we compare (Ni−1 , Ni+1 ) to the preceding and the subsequent
q
navigator frames (Nj −1 , Nj +1 ) of each acquired candidate frame Dj . There are various
possible similarity measures to compare the navigator frames. The evaluated options range
from a simple sum of squared intensity differences to more sophisticated measures like the
Mahalanobis distance in the eigenspace of the navigator frames. Since the goal is accurate
slice fitting, a natural approach is to compare the positions of prominent vascular structures
and thus to directly quantify local displacements. This frame comparison with respect to
vessel displacements produced the best results in a quantitative evaluation and was therefore
chosen for the following experiments. Section 2.4 will show such quantitative comparisons
of measures for image sorting in detail.
Figure 2(b) shows four example regions in the navigator slice that were considered to
derive a measure for image sorting. Three to five such rectangular regions showing prominent
and easily trackable structures were selected manually in one navigator frame. The position
of these regions in all other navigator frames was determined by template matching. The
template region was shifted over the compared navigator frame until maximum normalized
cross correlation was reached, indicating the most probable position of this region. Considering
more than three regions provided more information about the state of the liver and increased
the robustness against the exact choice of the regions and against tracking outliers. On the
other hand, more than five regions did not further improve the reconstruction results in our
experiments. Note that the apparent 2D translation detected by this template matching may
differ from the actual liver deformation due to the out-of-plane component of the liver motion.
However, as long as this apparent motion of all considered regions is similar in both embracing
navigator frames, it can be assumed that the true 3D deformation is also similar.
To determine the frame similarity based on the tracked regions, we define a cost function
p q
c(i, j ), which is small if the data frames Di and Dj show the same respiratory state. Therefore,
each region r should be at a similar position in the preceding navigator frames (Ni−1 , Nj −1 )
as well as in the subsequent navigator frames (Ni+1 , Nj +1 ). Assuming coordinate axes in the
anterior–posterior and cranio–caudal direction, xjr and yir are the coordinates of region r at
time i. With P being the number of tracked regions we define
P 
   r 
  
 xj −1 − xi−1 xj +1 − xi+1  
r r r P
 
X
 −1 r 
 +1
c(i, j ) =  r +  = r
+
X . (1)
 yj −1 − yi−1r
yj +1 − yi+1
r r 
r=1 r=1

In this cost function, we add for each region r the displacements (determined by template
matching) in the preceding navigator frames
X  −1
r
and the displacements in the subsequent
 r
navigator frames
X+1 and sum over all considered regions. A vector addition of the
displacements was chosen, because it captures the motion more consistently and proved
to be more robust than a separate calculation of the Euclidean deviation in the preceding and
in the subsequent frames. If for example two data frames are actually identical but were
acquired during organ movements of different speeds, then the embracing navigator frames
4D MRI of respiratory organ motion 1553

will not be identical and show displacements in opposite directions. With vector addition,
these displacements cancel out each other.
q p
Using cost function (1), the best matching data frame Dj ∗ for a given data frame Di is
∗
then found at time j , where
j ∗ = arg min c(i, j ). (2)
j

This cost function does not only allow for the selection of the most similar data frame,
but also gives a quantitative measure of the remaining deviations. In principle, a minimum
similarity could be enforced by imposing a threshold on the cost function such that, if no frame
was found with a deviation below the threshold, this reconstruction could be ignored. In the
following reconstructions however, such a threshold has not been applied in order to include
the worst case reconstructions in our quantitative analysis.

2.2.2. 4D reconstruction. First, we reconstruct a complete 3D volume for only one specific
state of the liver. With the discussed sorting method, this state is defined by two navigator
frames (Ni−1 , Ni+1 ). In order to reconstruct a complete 3D stack, a corresponding data frame
for each slice position is selected by applying (2) and the selected frames are stacked together
to form a 3D volume.
If a 3D volume is reconstructed for each pair of navigator frames in a sequence
(Ni−1 , Ni+1 ), (Ni+1 , Ni+3 ), (Ni+3 , Ni+5 ), . . . , this yields a complete 4D data set with the
temporal resolution of the navigator frames, which was 2.6–2.8 Hz in the described acquisition
sequence. In addition to the described interleaved sequences, a pure navigator sequence
Nk , Nk+1 , Nk+2 , Nk+3 , Nk+4 , . . . can be acquired twice as fast as the navigator frames in
the interleaved sequences. 3D stacks can then be reconstructed for the navigator pairs
(Nk , Nk+2 ),(Nk+1 , Nk+3 ),(Nk+2 , Nk+4 ). Since the acquisition of the data frames remains
unchanged and shows the same temporal and spatial distance between data and navigator
frames, this is of no advantage for the actual slice selection and the restacking of single
volumes. However, the higher temporal resolution of the navigator pairs in the separate
navigator sequence allows for the reconstruction of volumes with a higher temporal resolution
of 5.2–5.6 Hz.

2.2.3. Noise reduction. Since the acquired images show a considerable amount of noise due
to their short acquisition time and high spatial resolution, a straightforward noise reduction
was applied. Not only the very best matching frame but the M best matching frames were
determined for each slice position in each reconstructed volume. Averaging over these M
frames reduces the variance of the noise by a factor of M and increases the signal-to-noise
ratio compared to single acquisitions. The number of frames that can be averaged is limited
by the available number of images to choose from. In the presented examples, 180–240 data
frames were acquired at each slice position during roughly 1 h and the five best fitting frames
according to the cost function (1) were averaged for noise reduction.

2.3. Motion estimation

The acquisition technique described above provides us with a set of temporally resolved
3D images. In order to estimate the motion vectors of points within the liver from these
data, non-rigid registration between the volumes of different respiratory states was used. For
these registrations, the open source VTK CISG Registration Toolkit (Hartkens et al 2002)
was applied. This software implements intensity-based non-rigid registration as published
by Rueckert et al (1999) and Schnabel et al (2003). The deformation between two volumes
1554 M von Siebenthal et al

(a) 2cm (b) 2cm (c) 5mm

Figure 3. Assessment of local matching errors between pairs of sagittal images that were selected
as best matches. (a) Difference image, (b) result of the non-rigid registration and (c) enlarged
section with arrows showing the estimated local deviations between the compared frames.

is modelled with a grid of control points using the 3D displacements of the control points
as parameters. The motion of any point within the deformed volume is determined from
the control points using B-spline interpolation. For the following examples, we used cross
correlation as similarity measure and a control point resolution of 15 mm. These settings were
confirmed by visual inspection of the registration results, which showed local misregistrations
of less than one pixel.
To generate complete trajectories, one volume V0 at a typical exhalation position was
registered to all consecutive volumes in the reconstructed sequence (V0 → V1 , V0 → V2 ,
V0 → V3 , . . . ). For each registration (V0 → Vm ), m > 1, the previous registration result
(V0 → Vm−1 ) was used as an initial estimate to improve robustness. The resulting sequence of
deformations then defines the trajectory of each point in the volume. The continuous motion
fields resulting from registration of the entire volumes are generally not valid across organ
boundaries. For example, the liver slides along the chest wall, where the motion field must be
discontinuous. Therefore, the liver was manually segmented in V0 and the area outside this
region of interest was ignored during registration. Consequently, the resulting motion fields
are only valid within the liver, but do not suffer from smoothing artefacts across the organ
boundaries.

2.4. Evaluation of the sorting method

In addition to the visual assessment of the reconstructed volumes and motion fields, two
different methods were applied to quantify the robustness of the described sorting method.
The aim of these experiments was to show the accumulated errors in the selection of 2D frames
for volume reconstruction, including errors due to the spatial and temporal distance between
navigator and data slices and errors due to the limited amount of slices to choose from.
First, we tested whether data frames showing the liver in the same state can be found
accurately by the proposed method. This was evaluated with leave-one-out experiments. For
p
a given data frame Di acquired at slice position p and time i, the best matching data frame
p
at the same position Dj ∗ was searched among the remaining frames by applying the cost
p
function c(i, j ) defined in (1). The similarity of the selected frame Dj ∗ to the ideal left-out
p
frame Di was quantified by non-rigid registration using the VTK CISG Registration Toolkit
(Hartkens et al 2002). Figure 3 shows an example for a pair of data frames. The difference
image 3(a) exhibits local deviations between the original data frame and the best match that
was found. Figures 3(b) and (c) show an overview and an enlarged section of the non-rigid
registration result, where the estimated local deviations between the two frames are represented
4D MRI of respiratory organ motion 1555

by arrows. From this registration result, the mean and the maximum local deviations between
the compared slices were determined.
This process was repeated for all 200 data frames at each of the 22 slice positions in
an example data set, yielding the maximum and mean local deviation between each acquired
frame and its best match. For comparison, the same leave-one-out experiments were performed
with simplified one-dimensional sorting, which considered only one tracked navigator region
in the dome of the liver.
In addition to a comparison of different sorting approaches, the same leave-one-out
experiment was performed for different locations of the navigator slice in order to evaluate the
sensitivity of the reconstruction to the selection of the navigator slice. Within one acquisition
session, two data sets, each consisting of 140 data frames at 21 different slice positions,
were produced with two different medio-lateral positions of the navigator slice. Again, the
differences between the ideal left-out data frames and the best matches among the remaining
data frames were determined.
The second method to evaluate the volume consistency compares neighbouring slices in
reconstructed volumes similar to Werner et al (2006). The difference between each pair of
neighbouring slices in a volume is quantified as the sum of squared intensity differences (SSD)
in the liver region. Neighbouring frames generally differ because they show a different slice
of the organ and are subject to acquisition noise. However, additional discontinuities due
to inaccurate frame selection further increase the average SSD and can be quantified by this
measure. This SSD calculation was applied to 1007 volumes, which were reconstructed using
three different sorting methods: (i) random stacking, for which randomly selected frames
were stacked together, producing large discontinuities due to respiratory motion. (ii) One-
dimensional sorting, where matching frames were selected considering one single region in
the dome of the liver. (iii) The proposed sorting method with frame selection according to
the cost function (1). Additionally, the SSD between neighbouring slices in a breath-hold
acquisition was calculated for comparison. This breath-hold volume was acquired with the
same parameters as the dynamic images to obtain a comparable image quality.

3. Results

3.1. 4D reconstruction
The imaging and motion estimation described in the preceding section was performed with 11
healthy volunteers after written consent was obtained. Figure 4 shows an example volume,
which was reconstructed from 30 free-breathing images. Figure 4(a) illustrates the volume
reconstructed by one-dimensional sorting, where only a single region in the dome of the liver
was considered for frame selection. The dome of the liver shows very plausible slice fitting
with a sharp and continuous edge in the coronal view. However, regions that are not fully
defined by a certain diaphragm position show severe mismatches such as discontinuities in the
blood vessels. Figure 4(b) shows the volume for the same respiratory state reconstructed with
the proposed sorting method that considers four regions in the navigator slice. This method
achieves a continuous slice fitting throughout the liver. Figure 4(c) visualizes the effect of
additional noise reduction by averaging over five fitting frames for each slice position. The
signal-to-noise ratio is improved without blurring the image, which shows that the number of
acquisitions at each position was sufficient and the sorting approach is accurate and robust.
Figure 5 shows a part of the right liver lobe of a different volunteer at inhalation and
exhalation in a typical breathing cycle. The two volumes were chosen from a sequence
of 14 reconstructed stacks at 2.8 Hz temporal resolution. The stacks were interpolated to
1556 M von Siebenthal et al

(a) S P

I A
R L A PP R L

(b ) S P

I A
R L A PP R L

(c) S P

I A
R L A PP R L

Figure 4. Orthogonal cuts through reconstructed 3D images of the right liver lobe using (a) sorting
with respect to the diaphragm only, (b) the proposed sorting method, (c) the proposed method and
averaging over the five best matching frames for noise reduction.

an isotropic voxel size using 3D spline interpolation and visualized as maximum intensity
projections (MIP). The orthogonal slices and the MIPs exhibit no major discontinuities in the
blood vessels or the liver boundaries.
The method was also successfully applied to the lung as shown in figure 6. The blood
vessels in the lung were well reconstructed, even in the out-of-plane direction (coronal view)
and the contrast between blood and air is even higher than for the liver.

3.2. Motion estimation

This section shows example deformation fields obtained with the described method. Dense
deformation fields were estimated from the data sets of 11 volunteers by non-rigid registration.
Figure 7 shows the trajectories of selected points for two complete breathing cycles with
different breathing depths. Such cycle-to-cycle variations in amplitude as well as in speed
were expected, because all images were acquired during uncoached free breathing. Both
deformation fields show the non-rigid motion of the liver, which is largest in the cranio–caudal
direction. We also observe substantial left–right motion in the lower part of the right liver lobe,
4D MRI of respiratory organ motion 1557

(a) S S P S S P
(c)

I A I I A
I
R L A P R L R L A P R L

(b) S S P S S P
(d)

I A I I A
I
R L A P R L R L A P R L

Figure 5. Orthogonal cuts through 3D images of the right liver lobe at (a) exhalation and
(b) inhalation. The maximum intensity projections after 3D interpolation show continuous vascular
structures at (c) exhalation and (d) inhalation.

(a) (b)

Figure 6. Sagittal and coronal maximum intensity projections after 3D interpolation showing the
right part of the lung at (a) inhalation and (b) exhalation.

260
100 S 100 S P
240
120 120
220
140 140
200
160 160
180
[mm]
[mm]

[mm]

180 180
160
200 200
140
220 220
120
240 240
100
260 I 260 I A
R L A P R L
80
20 40 60 80 100 80 100 120 140 160 180 200 220 240 260 20 40 60 80 100
[mm] [mm] [mm]

Figure 7. Trajectories of selected points within the liver. Two example respiratory cycles for deeper
breathing (black) and shallower breathing (white). The start/end points (exhalation) are marked
with circles. The background shows the difference between typical inhalation and exhalation
volumes for orientation.
1558 M von Siebenthal et al

160 160
100 S
162 162
164 t0 120 t0 164
166 140 166
168 160 168

[mm]

[mm]

[mm]
170 t3 180 t9 170
172 t9 172
200 t3
174 174
220
176 t6 176
240
178 t6 178
260 I
R L
180 180
20 22 24 26 28 20 40 60 80 100 70 72 74 76 78
[mm] [mm] [mm]

Figure 8. Enlarged views of two point trajectories. Each vector shows the deformation between
two reconstructed volumes. The positions t3 after three time steps and t9 after nine time steps point
out the phase difference between the left and the right trajectory.

260
100 S 100 S P
240
120 120
220
140 140
200
160 160
180
[mm]
[mm]

[mm]
180 180
160
200 200
140
220 220
120
240 240
100
260 I 260 I A
R L A P R L
80
20 40 60 80 100 80 100 120 140 160 180 200 220 240 260 20 40 60 80 100
[mm] [mm] [mm]

Figure 9. Two point trajectories with similar amplitudes before (black) and after (white) a
deformation of the liver due to intestinal activity.

where the tip of the liver lobe slides inwards along the lateral chest wall during inhalation. To
examine the deeper breathing cycle (black trajectories) in more detail, we consider the enlarged
views of two point trajectories (figure 8). These trajectories show that the deformation of the
liver is different during inhalation and exhalation. A phase shift between the right and the left
trajectory can be observed. The right part of the liver lobe is pushed down later than the left
part during inhalation and it also lags behind during exhalation.
Next, we consider the respiratory motion over larger time scales in the range of minutes,
for example for the duration of a radiation treatment session. Figure 9 shows two example
breathing cycles with similar breathing depths that were acquired 25 min apart. The lower
part of the liver was deformed due to intestinal activity, which locally changed the exhalation
position by more than one centimetre and affected the shape of the motion trajectories. These
changes occurred in addition to the cycle-to-cycle variability in amplitude and speed. Another
example of a varying breathing pattern is shown in figure 10. These trajectories show how
the exhalation position drifted in the inferior direction by 5 mm within 35 min due to general
muscle relaxation. All 11 acquired data sets showed variations in the breathing pattern that
substantially changed the point trajectories in at least a part of the liver. Table 1 shows the
4D MRI of respiratory organ motion 1559

S S P
100 100 220

120 120 200

140 140 180

160 160 160

[mm]

[mm]

[mm]
180 180 140

200 200 120

220 220 100

240 240 80 A
I I
R L A P R L
20 40 60 80 100 120 80 100 120 140 160 180 200 220 20 40 60 80 100 120
[mm] [mm] [mm]

Figure 10. Point trajectories with similar amplitudes before (black) and after (white) a drift of the
exhalation position due to muscle relaxation.

Table 1. Drift of the exhalation position over 20 min. For each subject, the root-mean square of the
Euclidian deviation and the maximum Euclidian deviation within the right liver lobe are shown.

Subject 1 2 3 4 5 6 7 8 9 10 11
RMS drift (mm) 4.6 2.9 1.2 3.4 2.9 2.6 2.0 3.9 4.4 1.9 4.5
Maximum drift (mm) 12.8 5.9 4.3 7.6 5.7 4.8 6.0 8.5 8.6 3.1 6.3

Euclidian drift of the exhalation position in each subject over 20 min. Whilst the root-mean-
square drift over the entire right liver lobe was below 5 mm for all subjects, the maximum
drift ranged up to 12.8 mm. Although the presented examples showed changes over several
minutes, similar variations can occur more abruptly, for example due to sudden intestinal
activity.

3.3. Evaluation of the sorting method

This section describes the results of the leave-one-out experiments to quantify the volume
consistency as described above. The mean and maximum local deviations between the left-
out frames and the selected best matching frames are shown in figure 11. The deviations are
given per slice position with their mean and standard deviation over all 200 experiments. The
considered medio-lateral region is the same as for the reconstructions shown in figure 4 and
ranges from the vena cava inferior (position 0) to the rightmost end of the liver. Figure 11(a)
shows the results for one-dimensional sorting. The mean local deviation over all experiments
and all slice positions is 0.96 ± 0.53 mm and the maximum local deviation is 2.91 ± 2.07 mm.
In contrast, figure 11(b) illustrates the local deviations obtained with the described sorting
method. By considering four regions in the navigator slice, matching frames with much lower
local deviations were found. The remaining mean deviations of 0.68 ± 0.26 mm are below
one pixel (1.35 mm) and the maximum deviations of 1.63 ± 0.80 mm are mostly below two
pixels (2.7 mm).
Figure 12 shows the resulting local deviations for two different positions of the navigator
slice. For the navigator position shown in figure 12(a), the mean local deviation over all
experiments and all slice positions is 0.45 ± 0.24 mm, and the maximum local deviation is
1.19 ± 0.73 mm. The results for the second navigator position illustrated in figure 12(b) show
1560 M von Siebenthal et al

(a) 8 (b) 8
max navigator max
mean slice mean
7 7

local deviation [mm]

local deviation [mm]

6 6

5 5 navigator
slice
4 4

3 3

2 2

1 1

0 0
0 20 40 60 80 0 20 40 60 80
left slice position [mm] right left slice position [mm] right

Figure 11. Resulting maximum and mean local deviations between the selected frames and the ideal
left-out frames. The error bars show the standard deviations over all leave-one-out experiments
at a certain slice position. (a) One-dimensional sorting and (b) the described image-based sorting
method, which takes multiple regions into account.

(a) 8 (b) 8
max max
7 mean 7 mean
local deviation [mm]
local deviation [mm]

6 6

5 navigator 5
slice
4 4 navigator
slice
3 3

2 2

1 1

0 0
0 20 40 60 80 100 0 20 40 60 80 100
left slice position [mm] right left slice position [mm] right

Figure 12. Maximum and mean local deviations between the selected frames and the ideal left-out
frames. The two experiments differ in the position of the navigator slice, which lies in the left part
of the liver lobe for (a) and in the right part for (b), at a distance of 5 cm.

only slight differences with a mean local deviation of 0.46 ± 0.24 mm and a maximum local
deviation of 1.40 ± 0.86 mm. This specific example of robustness with respect to the exact
choice of the navigator position is consistent with our general experience. When applying
our method to a number of volunteers, we did not encounter any problems in reconstruction,
although the position of the navigator slice varied within the liver between subjects.
All leave-one-out experiments show lower matching errors towards the right end of
the liver, where the respiratory motion is smaller than around the dome of the right
liver lobe. The local deviations obtained with the described method for slice selection
(figures 11(b), 12(a), (b)) clearly increase towards the vena cava inferior (position 0), where the
liver is affected by the beating heart. These additional deformations introduce local deviations,
as they are uncorrelated to the respiration and not reflected in the navigator slice. Besides this
4D MRI of respiratory organ motion 1561

800
random stacking
700
1D sorting
600 proposed sorting

SSD per pixel

breath hold
500

400

300

200

100
− 40 − 20 0 20
left slice position [mm] right

Figure 13. The SSD similarity of neighbouring slices was normalized to SSD per pixel (intensity
range 0–255) and averaged over 1007 reconstructed volumes. Additionally, the SSD values
between the slices of a breath-hold volume are given for comparison.

clear effect of heart motion, the results illustrate that the data frames were selected accurately
by considering the navigator frames only, even though the navigator frames were acquired
at slightly different points in time than the data frames and at locations that may be several
centimetres apart from the data frames. This shows that there is sufficient correlation between
data and navigator slices and that the described method accurately determines the respiratory
state of the entire liver.
The second evaluation method, where neighbouring frames in reconstructed volumes
were compared by SSD, also shows the advantages of the proposed sorting method.
Figure 13 illustrates the SSD per pixel between neighbouring slices averaged over all 1007
volumes that were reconstructed with the compared sorting methods. The SSD values between
the slices of a breath-hold volume show the achievable similarity between neighbouring slices.
Not surprisingly, random stacking leads to the largest SDD values between neighbouring
slices (mean and standard deviation 470 ± 234). One-dimensional sorting achieves much
lower values (341 ± 140), because many discontinuities can be reduced by using this simple
approach. But considering more tracked regions as in the described method leads to even
lower SSD values (294 ± 100), except from the right part where the methods perform equally.
The right part of the liver lobe contains smaller blood vessels and typically shows smaller
motion than the central area beneath the dome of the liver. The rightmost slice, which contains
no moving tissue, shows of course hardly any difference between the four methods. Also in
these experiments, the increasing influence of the beating heart towards the left side is visible.

4. Discussion

The described method uses MR images to obtain motion data for research in the field of
radiation therapy planning and treatment, where dose calculations have to be performed.
Although the reconstructed MR images cannot be used directly for dose calculations, the
deformation vectors resulting from the motion estimation are independent of the imaging
modality. On the one hand, these motion vectors can be used to deform planning CT images by
transferring the motion vectors from MR to CT using non-rigid registration (Andronache 2006).
On the other hand, the displacement of calculation points according to the deformation fields
can be integrated directly in the dynamic dose calculation as mentioned in the introduction.
1562 M von Siebenthal et al

A common limitation of all slice stacking methods is that only breathing cycles can be
reconstructed, for which all necessary frames were captured across the liver. If particularly
deep breathing is only acquired at some of the slice positions, the method fails to reconstruct
a complete 4D sequence for this extreme breathing depth. In our experiments with free
uncoached breathing, 3D images around exhalation could be reconstructed with better fitting
of the slices than volumes showing extreme inhalation states, for which the probability of
finding accurately corresponding slices was lower. As mentioned in section 2.2.1, large
values of the cost function (1), which indicate missing data, could be detected by imposing a
threshold. Because of the large amount of data to select from, and in order to keep the worst
case reconstructions for our quantitative analysis, this was not done in the presented examples.
Nevertheless, drifts and other variations could be fully captured and reconstructed for typical
ranges of breathing depths without significant artefacts due to missing data.
Another limitation, which is common to all respiratory gated imaging methods, was
illustrated in the evaluation experiments. Deformations due to the beating heart are
uncorrelated to respiration and are thus ignored by respiratory synchronization. The applied
noise reduction by averaging similar frames alleviates this problem to a certain degree, because
the resulting errors are averaged as well. For a better handling of heart motion, the navigator
slice can be placed close to the heart, such that motion due to the beating heart is captured. If
the described method is performed with such a navigator slice, then fitting frames are selected,
for which not only the respiratory state but also the state of the heart matches. This additional
constraint requires more frames to select from but becomes interesting if the acquisition speed
can be further increased.
The evaluation experiments have shown that the proposed method achieved better slice
fitting in the reconstructed volumes than sorting based on the superior–inferior motion of the
diaphragm only. These results demonstrate that sorting according to this single region of
the liver allows for residual motion in other regions. Such residual motion has to be taken
into account whenever surrogates are considered to derive the respiratory state for imaging
or treatment. Potential gating techniques for radiation therapy, which are applicable in real
time but make more restrictive assumptions about the breathing pattern, are currently being
evaluated with respect to residual motion and optimal placement of surrogates.
A point that distinguishes the described method from clinically used imaging is the long
acquisition time of up to 1 h. Although this method was developed for motion studies over
tens of minutes and not for patient specific planning, it can also be used to acquire 4D data sets
in much shorter acquisition sessions. In our experiments, 15 min of acquisition were enough
to collect sufficient data for the reconstruction of typical respiratory cycles with different
breathing depths. A further speed-up can be achieved by acquiring only a few slices covering
a tumour instead of 30 slices across the entire right liver lobe. Lastly, interpolation of missing
frames from available data, which is the subject of ongoing work, aims at a further reduction
of this acquisition time.

5. Conclusions

The developed technique allows for the reconstruction of 4D data sets showing the detailed
deformation of an organ during free breathing. Whilst variations beyond a regular breathing
cycle are ignored by other approaches and lead to artefacts, they are addressed specifically
with the proposed method. Irregularities such as drifts and deformations are recognized and
handled by the image-based sorting method. In contrast to 4DCT, volunteer studies over long
acquisition sessions are possible. The motion estimation based on non-rigid registration can
determine point trajectories that show the detailed organ motion. This permits in particular
4D MRI of respiratory organ motion 1563

to study the changes in motion within the range of minutes, which may occur during a
treatment session. Further studies of this variability are strongly motivated by the large
range of observed changes in all acquired data sets. To support further research in the field
of organ motion, we made the data obtained with the described method available online
(http://www.vision.ethz.ch/4dmri).
The technique described in this paper is applicable to any organ that undergoes respiratory
motion and can be implemented on a standard MR scanner without additional equipment. The
method was developed for research in the field of proton therapy, where the motion data
are currently being used to evaluate dose delivery methodologies with respect to motion
sensitivity and to assess the residual motion under different gating techniques. However, the
new possibilities of studying realistic respiratory motion are valuable to improve many other
applications such as gating in MR-guided focused ultrasound (MRgFUS) or motion correction
in radiofrequency tumour ablation. Furthermore, data obtained with the proposed method can
be used to model generic organ motion, which is required for anatomical simulations and for
applications such as model-based segmentation or tracking.

Acknowledgment

This work has been supported by the CO-ME/NCCR research network of the Swiss National
Science Foundation (http://co-me.ch).

References

Andronache A 2006 Multi-modal non-rigid registration of volumetric medical images PhD Thesis ETH Zurich. ETH
No. 16601
Blackall J M 2002 Respiratory motion in image-guided interventions of the liver PhD Thesis University of London
Blackall J M, Ahmad S, Miquel M, Landau D and Hawkes D 2004 Modelling respiratory motion for optimisation of
lung cancer radiotherapy using fast MR imaging and intensity-based image registration Proc. Int. Soc. Magn.
Reson. Med. 11 2610
Clifford M A, Banovac F, Levy E and Cleary K 2002 Assessment of hepatic motion secondary to respiration for
computer assisted interventions Comput. Aided Surg. 7 291–9
Davies S C, Hill A L, Holmes R B, Halliwell M and Jackson P C 1994 Ultrasound quantitation of respiratory organ
motion in the upper abdomen Br. J. Radiol. 67 1096–102
Engelsman M, Sharp G C, Bortfeld T, Onimaru R and Shirato H 2005 How much margin reduction is possible through
gating or breath hold? Phys. Med. Biol. 50 477–90
Ford E C, Mageras G S, Yorke E and Ling C C 2003 Respiration-correlated spiral CT: a method of measuring
respiratory-induced anatomic motion for radiation treatment planning Med. Phys. 30 88–97
Gueulette J, Blattmann H, Pedroni E, Coray A, Coster B M D, Mahy P, Wambersie A and Goitein G 2005 Relative
biologic effectiveness determination in mouse intestine for scanning proton beam at Paul Scherrer Institute,
Switzerland. Influence of motion Int. J. Radiat. Oncol. Biol. Phys. 62 838–45
Hartkens T, Rueckert D, Schnabel J A, Hawkes D J and Hill D L G 2002 VTK CISG registration toolkit: an open source
software package for affine and non-rigid registration of single- and multimodal 3D images Bildverarbeitung
für die Medizin 2002 (Berlin: Springer) pp 409–12
Keall P J, Starkschall G, Shukla H, Forster K M, Ortiz V, Stevens C W, Vedam S S, George R, Guerrero T and
Mohan R 2004 Acquiring 4D thoracic CT scans using a multislice helical method Phys. Med. Biol. 49 2053–67
Keall P J et al 2006 The management of respiratory motion in radiation oncology, report of the AAPM task group 76
Technical Report 91 (American Association of Physicists in Medicine)
Lambert J, Suchowerska N, McKenzie D R and Jackson M 2005 Intrafractional motion during proton beam scanning
Phys. Med. Biol. 50 4853–62
Lomax A 1999 Intensity modulated methods for proton therapy Phys. Med. Biol. 44 185–205
Low D A et al 2003 A method for the reconstruction of four-dimensional synchronized CT scans acquired during
free breathing Med. Phys. 30 1254–63
1564 M von Siebenthal et al

McClelland J R, Chandler A G, Blackall J M, Ahmad S, Landau D B and Hawkes D J 2004 Feasibility study of a
novel technique for constructing respiratory motion models, for use in 4D lung cancer radiotherapy planning
Proc. MIUA 2004 (London: Imperial College) pp 220–3
Nehmeh S A et al 2004 Quantitation of respiratory motion during 4D-PET/CT acquisition Med. Phys. 31 1333–8
Pan T, Lee T Y, Rietzel E and Chen G T Y 2004 4D-CT imaging of a volume influenced by respiratory motion on
multi-slice CT Med. Phys. 31 333–40
Pedroni E et al 1995 The 200-MeV proton therapy project at the Paul Scherrer Institute: conceptual design and
practical realization Med. Phys. 22 37–53
Phillips M H, Pedroni E, Blattmann H, Boehringer T, Coray A and Scheib S 1992 Effects of respiratory motion on
dose uniformity with a charged particle scanning system Phys. Med. Biol. 37 223–33
Rietzel E, Chen G T Y, Choi N C and Willet C G 2005 Four-dimensional image-based treatment planning: target
volume segmentation and dose calculation in the presence of respiratory motion Int. J. Radiat. Oncol. Biol.
Phys. 61 1535–50
Rietzel E, Rosenthal S J, Gierga D P, Willett C G and Chen G T Y 2004 Moving targets: detection and tracking of
internal organ motion for treatment planning and patient set-up Radiother. Oncol. 73 (Suppl. 2) S68–72
Rohlfing T, Maurer C R Jr, O’Dell W G and Zhong J 2001 Modeling liver motion and deformation during
the respiratory cycle using intensity-based free-form registration of gated MR images Proc. SPIE Med.
Imaging 4319 337–48
Rueckert D, Sonoda L I, Hayes C, Hill D L G, Leach M O and Hawkes D J 1999 Nonrigid registration using free-form
deformations: application to breast MR images IEEE Trans. Med. Imaging 18 712–21
Schnabel J A, Tanner C, Castellano-Smith A D, Degenhard A, Leach M O, Hose D R, Hill D L G and Hawkes D J
2003 Validation of nonrigid image registration using finite-element methods: application to breast MR images
IEEE Trans. Med. Imaging 22 238–47
von Siebenthal M, Cattin P, Gamper U, Lomax A and Székely G 2005 4D MR imaging using internal respiratory
gating Proc. MICCAI vol 3750 (Berlin: Springer) pp 336–43
Wang Y, Rossman P J, Grimm R C, Riederer S J and Ehman R L 1996 Navigator-echo-based real-time respiratory
gating and triggering for reduction of respiration effects in three-dimensional coronary MR angiography
Radiology 198 55–60
Webb S 2006 Motion effects in (intensity modulated) radiation therapy: a review Phys. Med. Biol. 51 R403–25
Werner R, Erhardt J, Frenzel T, Saering D, Low D and Handels H 2006 Rekonstruktion von 4D-CT-Daten aus
räumlich-zeitlichen CT-Segmentfolgen zur analyse atmungsbedingter Organbewegungen Bildverarbeitung für
die Medizin 2006 (Berlin: Springer) pp 6–10