YNIMG-12897; No.

of pages: 14; 4C: 6, 7, 8, 9, 11

NeuroImage xxx (2016) xxxxxx

Contents lists available at ScienceDirect

NeuroImage

journal homepage: www.elsevier.com/locate/ynimg

1Q2 Hippocampus and amygdala volumes from magnetic resonance images

2 in children: Assessing accuracy of FreeSurfer and FSL against

F
3 manual segmentation

O
4Q3 Dorothee Schoemaker a,b, Claudia Buss c,d, Kevin Head c, Curt A. Sandman c, Elysia P. Davis c,e,
5 Mallar M. Chakravarty b,f, Serge Gauthier a, Jens Pruessner a,b,

O
6 a
McGill Centre for Studies in Aging, McGill University, Montreal, QC, Canada
7 b
Douglas Hospital Research Centre, Psychiatry Department, McGill University, Montreal, QC, Canada
8 c
University of California at Irvine, CA, USA

R
9 d
Charit, Berlin, Germany
10 e
University of Denver, CO, USA
11 f
Biomedical Engineering Department, McGill University, Montreal, QC, Canada

P
12

1 3 a r t i c l e i n f o a b s t r a c t D
14
19 Article history: The volumetric quantication of brain structures is of great interest in pediatric populations because it allows the
15 Received 21 August 2015 investigation of different factors inuencing neurodevelopment. FreeSurfer and FSL both provide frequently used 20
E
16 Accepted 15 January 2016 packages for automatic segmentation of brain structures. In this study, we examined the accuracy and consisten- 21
17 Available online xxxx
cy of those two automated protocols relative to manual segmentation, commonly considered as the gold stan- 22
18
T

dard technique, for estimating hippocampus and amygdala volumes in a sample of preadolescent children aged 23
Keywords: 49
50 Segmentation techniques
between 6 to 11 years. The volumes obtained with FreeSurfer and FSL-FIRST were evaluated and compared with 24
51 manual segmentations with respect to volume difference, spatial agreement and between- and within-method 25
C

Pediatric population
52 Hippocampus correlations. 26
53 Amygdala Results highlighted a tendency for both automated techniques to overestimate hippocampus and amygdala vol- 27
54 28
E

FSL-FIRST umes, in comparison to manual segmentation. This was more pronounced when using FreeSurfer than FSL-FIRST
55 FreeSurfer and, for both techniques, the overestimation was more marked for the amygdala than the hippocampus. Pearson 29
correlations support moderate associations between manual tracing and FreeSurfer for hippocampus (right r = 30
R

0.69, p b 0.001; left r = 0.77, p b 0.001) and amygdala (right r = 0.61, p b 0.001; left r = 0.67, p b 0.001) volumes. 31
Correlation coefcients between manual segmentation and FSL-FIRST were statistically signicant (right hippo- 32
campus r = 0.59, p b 0.001; left hippocampus r = 0.51, p b 0.001; right amygdala r = 0.35, p b 0.001; left amyg- 33
R

dala r = 0.31, p b 0.001) but were signicantly weaker, for all investigated structures. When computing intraclass 34
correlation coefcients between manual tracing and automatic segmentation, all comparisons, except for left hip- 35
36
O

pocampus volume estimated with FreeSurfer, failed to reach 0.70. When looking at each method separately, cor-
relations between left and right hemispheric volumes showed strong associations between bilateral 37
hippocampus and bilateral amygdala volumes when assessed using manual segmentation or FreeSurfer. These 38
39
C

correlations were signicantly weaker when volumes were assessed with FSL-FIRST. Finally, BlandAltman
plots suggest that the difference between manual and automatic segmentation might be inuenced by the vol- 40
ume of the structure, because smaller volumes were associated with larger volume differences between 41
N

techniques. 42
These results demonstrate that, at least in a pediatric population, the agreement between amygdala and hippo- 43
campus volumes obtained with automated FSL-FIRST and FreeSurfer protocols and those obtained with manual 44
U

segmentation is not strong. Visual inspection by an informed individual and, if necessary, manual correction of 45
automated segmentation outputs are important to ensure validity of volumetric results and interpretation of re- 46
lated ndings. 47
2016 Published by Elsevier Inc. 48
59
57
56
58

1. Introduction 60

Corresponding author at: McGill Centre for Studies in Aging, 6825 Boulevard LaSalle, Childhood is a period of great relevance in the development of risk 61
Montreal, QC H4H1R3, Canada. factors for various neuropsychiatric conditions (Paus et al., 2008). 62

http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
1053-8119/ 2016 Published by Elsevier Inc.

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 2 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

63 Together with increased efforts in prevention, many large-scale longitu- relations between structures are dened according using a training set 129
64 dinal studies, starting in early childhood, are currently being undertak- of manually labeled brains. The second commonly used automated seg- 130
65 en to reveal the impact of environmental, behavioral and biological mentation protocol is FIRST, provided as part of the FSL software li- 131
66 factors on subsequent developmental outcomes (Chakravarty et al., brary (http://fsl.fmrib.ox.ac.uk) (Patenaude, 2007; Patenaude et al., 132
67 2014; Giedd et al., 2015; Raznahan et al., 2014). Due to rapid advances 2011). Using a probabilistic framework, this software estimates bound- 133
68 of in-vivo brain imaging technologies, volumetric quantication of aries of brain structures based on the signal intensity of the T1 image as 134
69 brain structures from structural Magnetic Resonance Imaging (MRI) is well as the expected shape of structures to be segmented. 135
70 more accessible than ever. Thus, large-scale studies often acquire MRI It is well known that neuroanatomical variations are found not only 136
71 to investigate relations between volume of specic brain structures in clinical populations, but also when comparing brains of normal indi- 137
72 and different aspects of behavior. viduals (Pruessner et al., 2002). Automated segmentation approaches 138
73 Due to their involvement in multiple neuropsychiatric and neuro- are based on the questionable assumption that computer algorithms 139
74 logical conditions, the medial temporal lobe structures hippocampus can reliably differentiate and delimitate anatomical regions regardless 140
75 and amygdala have received a considerable amount of attention. The of inter-individual differences in neuroanatomy, scan quality, image 141
76 hippocampus is one of the most commonly studied and cited brain contrast, etc. While we did not nd any studies comparing the perfor- 142

F
77 structures in the scientic literature. Its involvement in basic cognitive mance of automated segmentation performed with FSL-FIRST and/or 143
78 functions, such as memory consolidation (Squire, 1992), psychopathol- FreeSurfer to manual segmentation in pediatric populations, the validity 144

O
79 ogies such as PTSD (Bonne et al., 2001), major depression (Campbell of these protocols has previously been assessed in healthy adult controls 145
80 and MacQueen, 2004), and neurological disorders, such as Alzheimer (Cherbuin et al., 2009; Morey et al., 2009; Patenaude et al., 2011) as well 146
81 disease (Fox et al., 1996), is well established. The amygdala is the as different clinical populations, such as Alzheimer Disease (Pipitone 147

O
82 main structure of the limbic system associated with fear (Adolphs et al., 2014; Snchez-Benavides et al., 2010; Shen et al., 2010), mood dis- 148
83 et al., 1994; Davis and Whalen, 2001). It has been linked to many psy- orders (Doring et al., 2011; Nugent et al., 2013; Tae et al., 2008), 149
84 chopathologies including borderline personality disorder (Donegan temporal-lobe epilepsy (Akhondi-Asl et al., 2011; Pardoe et al., 2009) 150

R
85 et al., 2003; Herpertz et al., 2001), PTSD (Rauch et al., 2000) and social and psychosis (Pipitone et al., 2014). These reports generally support 151
86 phobia (Stein et al., 2002). The association between negative life events the ability of automated methods to detect volume difference between 152

P
87 during childhood, such as abuse and traumatic experiences, and the in- clinical groups. However, many articles have highlighted a tendency for 153
88 creased risk of developing psychiatric disorders later in life is well doc- FreeSurfer and FSL-FIRST to overestimate volume of brain structures 154
89 umented (Janssen et al., 2004; Johnson et al., 1999; MacMillan et al., (Cherbuin et al., 2009; Doring et al., 2011; Morey et al., 2009; Nugent
D 155
90 2001; Springer et al., 2007). It has been hypothesized that the relations et al., 2013; Pipitone et al., 2014; Snchez-Benavides et al., 2010; Shen 156
91 between severe childhood stressors and vulnerability to psychopathol- et al., 2010; Tae et al., 2008). When assessing the correspondence be- 157
92 ogies might be mediated trough an impaired development of the hippo- tween volumes derived from these two automated protocols and man- 158
E
93 campus and/or amygdala (Pynoos et al., 1999; Teicher et al., 2003; ual segmentation earlier ndings are variable. For the hippocampus 159
94 Woon and Hedges, 2008). Thus, many efforts are directed at dening region, results usually support moderate to strong associations between 160
T

95 and clarifying the roles of the amygdala and the hippocampus in pediat- manual tracing and FreeSurfer, with Pearson correlation coefcients 161
96 ric samples. From a structural neuroimaging perspective, an important ranging from 0.71 (Cherbuin et al., 2009; Snchez-Benavides et al., 162
C

97 challenge lies in the reliable and valid volumetric quantication of 2010) to 0.90 (Shen et al., 2010). Studies looking at the association be- 163
98 these brain regions. However, reliable volumetric estimation is method- tween hippocampus volumes derived from FSL-FIRST and manual seg- 164
99 ologically limited by the anatomical complexity of these two structures. mentation report Pearson correlations ranging from 0.47 (Pardoe 165
E

100 Manual segmentation is currently considered the gold standard for et al., 2009) to 0.67 (Nugent et al., 2013). Few studies have looked at 166
101 volumetric quantication of brain structures (Pardoe et al., 2009; the agreement between amygdala volumes derived from automated 167
R

102 Rodionov et al., 2009). However, this procedure requires sufcient ana- segmentation protocols and manual tracing. A study by Morey et al. 168
103 tomical and MR methodological expertise, is difcult and time- (2009) revealed weaker associations between manual segmentation 169
104 consuming to learn, and can be associated with intra- and inter-rater and both FSL-FIRST and FreeSurfer when estimating the amygdala vol- 170
R

105 variability if not performed using a consistent approach (Jack Jr. et al., ume than when estimating the hippocampus volume (Morey et al., 171
106 1995). In order to increase reliability and reduce potential biases associ- 2009). Taken together, these results seem to indicate that the concor- 172
O

107 ated with manual segmentation procedures, multiple protocols have dance between volumes derived from manual segmentation versus au- 173
108 been established and described in the literature for specic target re- tomatic protocols depend on the segmented structure as well as the 174
109 gions (Jack et al., 1990; Matsuoka et al., 2003; Pruessner et al., 2000; protocol used. Further, a report by Snchez-Benavides suggests that 175
C

110 Watson et al., 1992). Studies have demonstrated that using these proto- the accuracy of automated protocols may vary depending on neuroana- 176
111 cols signicantly improve intra- and inter-rater agreement (Jack et al., tomical characteristics of studied populations (Snchez-Benavides et al., 177
N

112 1990; Matsuoka et al., 2003; Pruessner et al., 2000; Watson et al., 2010). More precisely, this later study highlights a larger discrepancy 178
113 1992). However, these protocols require a considerable amount of between manually and automatically segmented volumes when used 179
114 training and thus further increase time demands of manual segmenta- on atrophic brains. Previous reports assessing the validity and accuracy 180
U

115 tion procedures. In contrast, protocols that offer the fully automated of FSL-FIRST and FreeSurfer were based on adult brains; it remains un- 181
116 processing and segmentation of target structures from MR images are certain whether smaller brain volumes and potential changes in gray / 182
117 fast (speed is only limited by CPU power and availability), have excel- white matter contrasts in pediatric brains negatively affect the perfor- 183
118 lent reproducibility and require little anatomical expertise from the mance of these two automated segmentation software. Thus, studies in- 184
119 end user. As a result, a number of automated protocols have recently vestigating the validity of automated segmentation in children are 185
120 been developed, published and received favorably by the research com- needed. 186
121 munity. In part because they are easily and freely accessible to the re- The goal of this article was to explore the validity of FSL-FIRST and 187
122 search community and provide detailed documentation on usage, two FreeSurfer in estimating hippocampus and amygdala volumes in chil- 188
123 of these automated procedures have gained a considerable amount of dren. To do so, we compared volumes generated by these two automat- 189
124 popularity. The rst one is FreeSurfer (http://surfer.nmr.mgh.harvard. ed techniques to volumes obtained by manual segmentation, which is 190
125 edu), a software developed by the Martinos Center for Biomedical Imag- considered to be the gold standard approach. The validity of the 191
126 ing (Fischl et al., 2002). FreeSurfer automatically assigns a label to each segmentation methods was investigated by means of three different ap- 192
127 voxel from the anatomical image based on probabilistic estimations re- proaches. First, we established discrepancies between volumes derived 193
128 lying on Markov random elds (MRFs). The localisation and spatial from manual segmentation and automated methods. Second, to 194

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx 3

195 estimate the consistency between manual and automated segmenta- rater achieved an intraclass correlation coefcient of 0.90, and an 241
196 tion, we assessed between- and within-method associations of intrarater reliability of 0.92. One of the main objectives of this study 242
197 hippocampus and amygdala volumes. Finally, to explore agreement be- was to dene whether smaller brain volumes could affect the accuracy 243
198 tween volumes and estimate possible proportional and xed biases in of FSL-FIRST and/or FreeSurfer in estimating hippocampus and amygda- 244
199 volume estimation we computed BlandAltman plots. la volumes. Therefore, we used original MR T1 images from children 245
participants as input for both automated protocols. Consequently, to 246
200 2. Methods be able to compare all segmentation methods within the same space, la- 247
bels from manual segmentation were resampled to the native space 248
201 2.1. Subjects using the inversion of the matrix le designed to perform the linear 249
transformation prior to the manual segmentation. Native labels from 250
202 Anatomical MRI scans were collected in preadolescent children as the specic structures (left/right amygdala and hippocampus) were 251
203 part of two studies on child neurodevelopment that applied the same saved as four distinct binary masks, each representing a single structure. 252
204 MRI acquisition protocol conducted at the University of California Irvine A voxel count was then used to estimate volumes from manually seg- 253
205 (Buss et al., 2012; Davis et al., 2013). Institutional review boards from all mented structures. To verify that the resampling of labels did not inu- 254

F
206 participating institutions approved all study procedures. All T1 images ence our results and conclusions, we also computed native volumes by 255
207 were visually inspected for quality of the image and for absence of ap- dividing the original segmentation volume in standard space by the 256

O
208 parent motion artifacts. 153 scans judged to be of good quality were global scale factor associated with the linear transformation (native vol- 257
209 retained and used in this study. Two scans were removed due to co- ume = standard volume/[x y z scale factors]). Volume difference 258
210 registration issues when using FSL and 4 others were removed due to and between-methods correlation analyses described below were also 259

O
211 inadequate processing with FreeSurfer, leaving 147 subjects for nal performed with native volumes obtained the using the global scaling 260
212 analyses. Following quality control, the nal sample included 65 girls factor. 261

R
213 and 82 boys study (age range: 6 to 11 years, mean age = 8.47 years
214 1.37 SD). These children were predominantly right-handed (n = 2.3.2. Automated segmentation of the hippocampus and amygdala using 262
215 130). The demographic information of subjects used in analyses is sum- FreeSurfer 263

P
216 marized in Table 1. The segmentation of the hippocampus and amygdala were also per- 264
formed using the FreeSurfer recon-all pipeline (v.4.4.0; http://surfer. 265
217 2.2. MRI acquisition nmr.mgh.harvard.edu/). In brief, this technique estimates the probabil-
D 266
ity of each voxel to belong to a certain structure, based on a-priori 267
218 T1 anatomical imaging was performed on a 3-T Philips Achieva MRI knowledge of spatial relationships acquired with a training set. It uses 268
219 scanner, at 1 mm isotropic resolution. Images were acquired in the sag- differences in voxel intensity to locate and parcelate subcortical struc- 269
E
220 ittal plane with the following parameters: repetition time 11 ms; echo tures and afne registration to the Talairach space. The FreeSurfer pro- 270
221 time 3.3 ms; inversion time 100 ms; turbo eld echo factor 192; 150 cessing stages are fully described in Fischl, B., et al. (2002). All les 271
Q5
T

222 slices; sensitivity encoding for fast MRI acceleration; and ip angle 18. were visually inspected to ensure adequate registration. Four subjects 272
were removed from the analysis due to poor co-registration. The vol- 273
C

223 2.3. Volumetric quantication umes provided in the aseg.stats le were used in the analysis, because 274
these take into account partial volume estimation and are judged to 275
224 2.3.1. Manual segmentation of the hippocampus and amygdala be more accurate than the voxel count of label les. For visualization, 276
E

225 Before proceeding to the manual segmentation, anatomical images segmentation les in the native space were converted into the MINC 277
226 were corrected for intensity non-uniformity (Sled et al., 1998) and reg- format. Labels from the specic structures (left/right amygdala and hip- 278
R

227 istered to the stereotaxic space (MNI152 template) (Collins et al., 1994) pocampus) were also saved as four distinct binary masks in the native 279
228 using a linear transformation. This pre-processing was performed to fa- space. 280
229 cilitate the identication of key structures and improve segmentation
R

230 consistency between scans. The hippocampus and amygdala were seg- 2.3.3. Automated segmentation of the hippocampus and amygdala using 281
231 mented by a single expert rater using the software DISPLAY (www.bic. FSL 282
O

232 mni.mcgill.ca/software/Display/Display.html). The anatomical borders Hippocampus and amygdala volumes were further obtained using 283
233 of the two key structures were dened and segmented according to FSL-FIRST (v.1.2; http://fsl.fmrib.ox.ac.uk/). In brief, following registra- 284
234 the protocol described by Pruessner et al. (Pruessner et al., 2000). As tion to a standard template this software uses a Bayesian probabilistic 285
C

235 the structural characteristics, delineations and boundaries of the hippo- model that relies on shape and intensity to infer the location of struc- 286
236 campus and amygdala in children aged over 6 years old are fully devel- tures of interest. For each structure a pre-dened number of modes is 287
N

237 oped (Arnold and Trojanowski, 1996), the segmentation protocol was applied to ensure the best t. More documentation on the processing 288
238 used as described in the article and no specic modications were nec- steps of FIRST can be found in Patenaude's articles (Patenaude, 2007; 289
239 essary for the population of interest. This protocol has been shown to Patenaude et al., 2011). Finally, segmentation labels in the native 290
U

240 allow good intra- and inter-rater reliability. Consistently, the present space were converted in the MINC le format. All les were visually 291
inspected to ensure correct registration. Two subjects were removed 292
from subsequent analyses due to inadequate co-registration and poor 293
t1:1 Table 1
processing. Labels from the specic structures (left/right amygdala 294
t1:2 Demographic information.
and hippocampus) were saved as binary masks, generating four sepa- 295
t1:3 Mean (SD) rate masks. A voxel count was then used to estimate volumes of struc- 296
t1:4 N 147 tures segmented using FSL-FIRST. 297
t1:5 Age 8.47
(1.37)
2.4. Statistical analysis 298
t1:6 Gender 82/65
t1:7 (M/F)
t1:8 Handedness 130/17 Volumes used for method comparisons and statistical analyses were 299
t1:9 (R/L) in the native space. Due to the absence of group comparisons or correla- 300
t1:10 Subject demographics. M male. F female. R right tions with external factors in the current analyses, we did not correct for 301
t1:11 handed. R left handed. SD standard deviation. intracranial volume as there was no specic need to control for this 302

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 4 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

303 variable. All the following statistical analyses were performed using IBM bilateral structures segmented within a same technique. Past research 367
304 SPSS statistics version 20. indicates that, in a single subject, a moderate to strong association is ex- 368
pected between homotopic (left versus right hemisphere) volumes 369
305 2.4.1. Analysis of volume difference (Allen et al., 2002). Weak left versus right hemisphere correlations 370
306 The percentage of difference between volumes obtained with auto- would indirectly suggest a lack of consistency or the presence of errors 371
307 mated methods and manual segmentation was computed using the fol- in volume estimation within the assessed method. Further, if the two 372
308 lowing formula: %VD = [(Va Vm)/Vm] 100%. In the event that the automated segmentation protocols are consistent with manual seg- 373
309 automated (Va) method reaches an identical volume as manual seg- mentation, similar associations between left and right hemisphere vol- 374
310 mentation (Vm), the resulting percentage of volume difference (VD) umes are expected when comparing theses methods. Thus difference in 375
311 would be 0%. Hence, larger percentages of VD indicate increased dis- magnitude between within-method correlations was also assessed ac- 376
312 crepancy between the volume derived from manual segmentation and cording to the statistical procedure described in Raghunathan et al. 377
313 volumes derived from automated methods. Negative values are indica- (1996) article and based on the Fisher r-to-Z transform (ZPF) 378
314 tive of an underestimation of volumes, in comparison with manual seg- (Raghunathan et al., 1996). In comparison to the Steiger's z statistical 379
315 mentation, while positive values suggest an overestimation of volumes test, this procedure is designed to assess differences between correlated 380

F
316 computed automatically relative to manual segmentation. In order to but nonoverlapping correlation coefcients. 381
317 investigate potential interactions between methods and segmented

O
318 area (as expressed in percentages of volume difference), we conducted 2.4.3. Analysis of estimation biases 382
319 a two-factor (Method x Area) repeated measure ANOVA. Signicant To further investigate agreement between manual segmentation 383
320 main effects were explored using post hoc Bonferroni-corrected volumes and volumes derived from automated protocols, we computed 384

O
321 paired-samples t tests with a signicance threshold adjusted to BlandAltman plots. This graphical method is used to illustrate differ- 385
322 p b 0.01 to account for the four (k = 4) performed comparisons. To lo- ences in estimation between two techniques or raters (Bland and 386
323 cate regions of disagreement between volumes derived from automatic Altman, 1986). BlandAltman plots are sometimes created using the 387

R
324 methods and manual segmentation, 3D maps of regional differences mean of the two studied techniques as the estimation of reference. 388
325 were prepared. For each subject, the transformation matrix associated However, as manual segmentation is accepted and viewed as the gold 389

P
326 with registration to the MNI152 space was estimated using the standard of technique for hippocampus and amygdala volumes estima- 390
327 mritotal tool of the MINC Tool Kit. Binary masks representing labels tion, we plotted the difference between automated and manually seg- 391
328 from each of the three segmentation methods were then resampled to mentation volumes against the volumes obtained with manual
D 392
329 the MNI152 space, using the same transformation matrix. Using the segmentation. Arguments in favor of this procedure can be found in 393
Q7
330 mincmath tool of the MINC Tool Kit maps of regional agreement be- Krouwer, J.S., 2008 (Krouwer, 2008). We further integrated a regression 394
331 tween manual segmentation and both automated techniques were line to the plot to explore possible biases in volume estimation and ob- 395
E
332 computed. Specically, these maps were constructed so that each serve whether characteristics of studied brain structures, as dened 396
333 voxel represents the average percent of volume difference between la- using the gold standard technique, inuence the discrepancy between 397
T

334 bels from manual segmentation and the automated method (100%, indi- manually and automatically segmented volumes. 398
335 cating a total disagreement that a specic voxel belongs to the
C

336 segmented structure and 0%, indicating a total agreement). Thus, a 3. Results 399
337 voxel with a percentage difference value of 25% would indicate that in
338 25% of the subjects where this specic voxel is inconsistently labeled be- 3.1. Analysis of volume differences 400
E

339 tween techniques, while in 75% of subjects, this voxel is labeled by both
340 techniques. For visualization, the maps are presented on the average Percentages of volume difference were computed separately for the 401
R

341 standardized brain of all participants included in the analyses. left and right hippocampus and the left and right amygdala. The mean 402
percentage of volume difference of FreeSurfer-derived volumes relative 403
342 2.4.2. Correlation analysis to manually segmented volumes was of 60.38% (SD = 13.04) and 404
R

343 Pearson correlations were conducted to estimate associations be- 51.53% (SD = 13.17) for the left and right hippocampi, respectively, 405
344 tween manual and automated techniques and to establish whether vol- and 100.29% (SD = 24.56) and 93.56% (SD = 25.78) for the left and 406
O

345 umes derived from automated methods are signicantly associated right amygdala, respectively. When computing the difference between 407
346 volumes obtained with manual segmentation. A strong correlation FSL-FIRST and manual segmentation, the mean percentage of volume 408
347 would conrm a good consistency between automated techniques difference was of 27.61% (SD = 14.49) and 28.39% (SD = 13.07) for 409
C

348 and manual segmentation. To compare the two automated segmenta- the left and right hippocampi, respectively and of 50.32% (SD = 410
349 tion techniques with regards to their correlation with manual segmen- 27.65) and 40.29% (SD = 26.09) for the left and right amygdala, respec- 411
N

350 tation, we computed Steiger's z test, a test recommended to assess the tively. The mean hippocampus and amygdala volumes as well as per- 412
351 difference in magnitude between correlated and overlapping correla- centage of volume difference derived from each technique are 413
352 tion coefcients (Meng et al., 1992; Steiger, 1980). Further, to obtain a presented in Table 2. The effects of the segmentation technique (FSL- 414
U

353 concurrent estimate of consistency and agreement between volumes FIRST versus FreeSurfer) and the segmented area (average left and 415
354 derived from the different segmentation techniques, we computed right hippocampus volume respectively average left and right amygda- 416
355 intraclass correlation coefcients (ICC) (Shrout and Fleiss, 1979). An la) on the obtained percentage of volume difference were tested with a 417
356 ICC value of 1 indicates a perfect reproducibility between two (or two-way repeated measure ANOVA. This analysis revealed a signicant 418
357 more) raters and of 0 or less, a reproducibility that is lower than what effect of the technique F(1146) = 1555.65, p b 0001. Post-hoc 419
358 is expected on the basis of chance alone. While there is no ofcial guide- Bonferroni-corrected pairwise comparisons further revealed that 420
359 line for the interpretation of ICCs, it has previously been suggested that a FreeSurfer leads to signicantly larger percentage of volume difference 421
360 ICC denoting a good reproducibility between measurements should be than FSL-FIRST for both the hippocampus (t(146) = 38.24, p b 0001) 422
361 equal to or higher than 0.75 (Burdock et al., 1963). Further, 0.70 has and the amydgala (t(146) = 29.52, p b 0001). A highly signicant effect 423
362
Q6 often been considered as the minimum standard for adequate reliability of the segmented area was also noted (F(1146) = 395.22, p b 0001). 424
363 (Nunnally, J.C., et al., 1967; Terwee et al., 2007). ICCs were computed au- Bonferroni-corrected pairwise comparisons showed that the amygdala 425
364 tomatically with SPSS and, specifying a mixed-effect model as per yielded signicantly larger percentage of volume difference than the 426
365 Shrout and Fleiss (1979) guidelines. Finally, to assess within-method hippocampus when segmented with both FSL-FIRST (t(146) = 9.85, 427
366 consistency, Pearson correlations were performed between volumes of p b 0001) and FreeSurfer (t(146) = 24.11, p b 0001). Further, there 428

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx 5

t2:1 Table 2
t2:2 Comparison of volumes between methods.

t2:3 Manual FSL-FIRST FreeSurfer

t2:4 Volume (SD) Volume (SD) % volume diff. (SD) Volume (SD) % volume diff. (SD)

t2:5 L-hippocampus 2746,29 (347,73) 3475,44 (378,37) 27.61 (14.49) 4378,05 (445,69) 60.38
(13.04)
t2:6 R-hippocampus 2786,92 (337,88) 3553,06 (372,55) 28.39 (13.07) 4194,63 (390,10) 51.53 (13.17)
t2:7 L-amygdala 777,27 (134,99) 1144,02 (163,19) 50.32 (27.65) 1532,65 (171,49) 100.29 (24.56)
t2:8 R-amygdala 832,92 (137,71) 1148,97 (194,78) 40.29 (26.09) 1586,05 (170,55) 93.56
(25.78)

t2:9 Description of mean volumes derived from each technique as well as mean percentage of volume difference (% volume diff.) obtained between FreeSurfer/FSL-FIRST and manual segmen-
t2:10 tation. L left. R right. SD standard deviation.

429 was a signicant interaction effect between the automated segmenta- FIRST and the one obtained with Freesurfer was tested using the 464

F
430 tion method and the area (F(1146) = 180.27, p b 0001), due to the Steiger's z test. Since a total of four comparisons were performed, the 465
431 fact that the difference in volume differences between the hippocampus alpha was adjusted to p b .01 for statistical signicance, applying the 466

O
432 and the amygdala was even more pronounced when using FreeSurfer Bonferroni correction. Correlations between manual and automated 467
433 than FSL-FIRST. Results of this analysis are summarized in Fig. 1. To segmentation volumes were signicantly stronger for Freesurfer than 468
434 obtain a visual estimation of areas of discrepancy between manual seg- FSL-FIRST for the left (Z = 4.83, p b 0.001) and right (Z = 3.31, 469

O
435 mentation and the two studied automated methods, 3D-maps where p b 0.001) amygdala and the left hippocampus (Z = 5.05, p b 0.001). 470
436 computed for FSL-FIRST vs. manual segmentation (Fig. 2), and For the right hippocampus, the difference in correlations obtained 471
FreeSurfer (Fig 3) vs. manual segmentation using the mincmath com-

R
437 with manual segmentation obtained with FSL-FIRST and FreeSurfer 472
438 mand, as part of the Minc ToolKit for manipulating 3D images (http:// did not reach our corrected signicance threshold (Z = 2.28, p = 473
439 www.bic.mni.mcgill.ca/ServicesSoftware/MINC). As expected, these 0.01). To investigate causes of incongruity between segmentation vol- 474

P
440 maps showed that, while the agreement between manual and automat- umes, outliers were identied using the magnitude of the residuals 475
441 ed segmentation is usually satisfactory towards the inner sections of the and selecting individuals that were at the furthest distance from the re- 476
442 structures, especially at the cores, the disagreement increases linearly gression line. Illustrations of the segmentation obtained from these out-
D 477
443 towards the lateral and medial, superior and inferior, and anterior and liers are presented in Figs. 5 and 6 for FSL-First and FreeSurfer, 478
444 posterior borders of the target structures. From Figs. 2 and 3, it appears respectively. 479
445 that higher percentages of difference appear in the hippocampal tail as
E
446 compared to the head area. For the amygdala, when comparing FSL- 3.2.1. Intraclass correlation coefcient 480
447 FIRST against manual segmentation, higher percentages of difference The ICC between manual segmentation and Freesurfer was rlhc = 481
T

448 are noted in superior boundaries. The same comparison between 0.74 (CI: 0.660.81) for the left hippocampus, rrhc = 0.68 (CI: 0.59 482
449 FreeSurfer and manual segmentation shows differences in both superior 0.76) for the right hippocampus, rlag = 0.65 (CI: 0.550.74) for the left 483
C

450
Q8 and inferior boundaries. (See Table 3.) amygdala and rrag = 0.60 (CI: 0.480.69) for the right amygdala. 484
When comparing manual segmentation and FSL-FIRST volumes, the 485
451 3.2. Correlation analysis. ICC for the left hippocampus was rlhc = 0.51 (CI: 0.380.62), rrhc = 486
E

0.59 (CI: 0.470.68) for the right hippocampus, rlag = 0.30 (CI: 0.15 487
452 3.2.1 Between-method correlations 0.44) for the left amygdala, and rrag = 0.33 (CI: 0.170.46) for the 488
R

453 Pearson correlations between manual segmentation and FreeSurfer right amygdala. 489
454 volumes were rrhc = 0.69 and rlhc = 0.77 for right and left hippocampus,
455 respectively and rrag = 0.61 and rlag = 0.67 for right and left amygdala, 3.2.2. Within-method correlation analysis 490
R

456 respectively. Correlations between FSL-FIRST and manually segmented Pearson correlations between volumes in the left and right hemi- 491
457 volumes were rrhc = 0.59 and rlhc = 0.51 for the right and left hippo- sphere derived from each technique were calculated to estimate 492
O

458 campus, respectively and rrag = 0.35 and rlag = 0.31 for the right and within-method consistency. Results of this analysis are presented in 493
459 left amygdala, respectively. All correlations reached a p b 0.0001 thresh- Fig. 7 (A to F). The association between interhemispheric (left versus 494
460 old. Correlations between volumes obtained with manual segmentation right) volumes was r = 0.85 (p b 0.0001) for hippocampus and r = 495
C

461 and automatic protocols for FreeSurfer and in for FSL-FIRST are summa- 0.75 (p b 0.0001) for amygdala volumes estimated with manual segmen- 496
462 rized in Fig. 4 A and B, respectively. For each region (lhc, lag, rhc, rag), tation, r = 0.83 (p b 0.0001) for hippocampus and r = 0.77 (p b 0.0001) 497
N

463 the difference in magnitude between correlations obtained with FSL- for amygdala volumes estimated with FreeSurfer, and r = 0.53 498
(p b 0.0001) for hippocampus and r = 0.59 (p b 0.0001) for amygdala 499
volumes estimated with FSL-FIRST. The difference in magnitude between 500
U

the computed correlations was tested with the ZPF statistic. Overall, 4 501
comparisons were performed: correlations between bilateral hippocam- 502
pi (bHC) volumes estimated with manual segmentation versus correla- 503
tions between bHC volumes estimated with FSL-FIRST/FreeSurfer; 504
correlations between bilateral amygdala (bAG) volumes estimated 505
with manual segmentation versus correlations between bAG volumes 506
estimated with FSL-FIRST/FreeSurfer. Consequently, the alpha was ad- 507
justed to p b .01 for statistical signicance, as per the Bonferroni proce- 508
Fig. 1. Percentage of volume difference between automatic protocols and manual dure. Using this criterion, signicant differences were observed only 509
segmentation for the combined left and right hippocampus and amygdala volumes. Two between within-method correlations of volumes estimated with manual 510
asterisks indicate a signicant difference (at the p b 0.0001 level). Percent volume segmentation and with FSL-FIRST. More precisely, the results suggest a 511
differences are signicantly larger for volumes estimated with FreeSurfer than FSL-
stronger association between bi-hemispheric volumes when estimated 512
FIRST, for both the amygdala and the hippocampus. Further, the amygdala leads to
signicantly larger percent volume differences than the hippocampus, for FreeSurfer with manual segmentation than FSL-FIRST. This was true for both the 513
and FSL-FIRST. bAG (ZPF = 2.55, p b .01) and bHC (ZPF = 6.21, p b .01) volumes. No 514

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 6 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

F
O
O
R
Fig. 2. Satistical maps representing, for each voxel, the average percentage of difference between manual segmentation and FSL-FIRST volumes for the A left hippocampus, B left

P
amygdala, C right hippocampus, D right amygdala. The maps are displayed on the average standardized brain of all subjects.

515 signicant difference was found between the strength of within-method volume difference and baseline manual segmentation volume. This sug-
D 526
516 correlations of bHC (ZPF = 0.66, p N .05) and bAG (ZPF = 0.58, p N .05) gests that smaller volume of the studied structures leads to larger differ- 527
517 volumes when estimated with manual segmentation or with FreeSurfer. ence in volume estimation when comparing automatic to manual 528
tracing. Thus, this seems to indicate that neuroanatomical features pos- 529
E
518 3.3. Analysis of estimation biases sibly systematically inuence outputs from automatic segmentation 530
protocols. 531
T

519 BlandAltman graphs plotting raw volume difference between man-

520 ual and automatic segmentation volumes against manual segmentation 4. Discussion 532
C

521 volume, considered to be the gold standard measure, conrm that

522 both FreeSurfer and FSL-FIRST (Fig. 8 A and B) yielded larger volumes Here we compared two widely used automated segmentation tools, 533
523 than manual segmentation. In all plots but the one comparing left hip- FSL-FIRST and FreeSurfer, against manual segmentation, the current 534
E

524 pocampus volumes between FreeSurfer and manual segmentation, the gold standard technique, for estimating hippocampus and amygdala 535
525 incorporated regression line highlights a negative linear trend between volumes in a population of preadolescent children. To our knowledge 536
R
R
O
C
N
U

Fig. 3. Satistical maps representing, for each voxel, the average percentage of difference between manual segmentation and Freesurfer volumes for the various structures. A left
hippocampus, B left amygdala, C right hippocampus, D right amygdala. The maps are displayed on the average standardized brain of all subjects.

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx 7

t3:1 Table 3 commonly studied in the eld of neuroscience. We also dened manual 541
t3:2 Comparison of inter-hemispheric volumes correlations derived from each method. volumes as the standard of reference, because its validity has been 542
t3:3 Leftright hippocampus Leftright amygdala established in previous articles (Pardoe et al., 2009; Rodionov et al., 543
2009). 544
t3:4 Manual segmentation 0.85 0.75
t3:5 FreeSurfer 0.83 0.77 Our results highlight important differences between volumes de- 545
t3:6 FSL-FIRST 0.53 0.59 rived from manual segmentation and the two studied automated tech- 546
t3:7 Pearson correlations of left against right hemispheric volumes obtained within a same seg- niques. Indeed, both FreeSurfer and FSL-FIRST overestimated total 547
t3:8 mentation method. hippocampus and amygdala volumes in comparison with the manual 548
t3:9 Indicates a signicant difference (at the p b 0.0001 level) in the magnitude of the segmentation protocol used in the current study. When the same vol- 549
t3:10 correlation, as compared with the correlation coefcients obtained with manual segmen- ume difference analyses were performed using native volumes obtained 550
t3:11 tation, as dened with the Fisher r-to-Z transform (ZPF) statistical test.
by dividing the volume of labels manually segmented in the standard 551
space by scale factors of the linear transformation (x y z), the results 552
537 this is the rst study looking at the validity of automated segmentation were highly similar and also suggested that FreeSurfer and FSL-FIRST 553
538 tools in a large pediatric sample. In this study, we decided to focus on overestimated hippocampus and amygdala volumes in comparison to 554

F
539 hippocampus and amygdala volumes because these regions are impli- manual segmentation. This suggests that large volume differences be- 555
540 cated in multiple psychopathologies and are among the most tween manual and automated segmentation were not due to biases 556

O
O
R
P
D
E
T
C
E
R
R
O
C
N
U

Fig. 4. Pearson correlations between volumes obtained with manual segmentation and with FreeSurfer (A) and FSL-FIRST (B). Plots are presented separately for i-right hippocampus ii- left
hippocampus, iii-right amygdala, iv-left amygdala. r pearson correlation coefcient. Outliers, dened using the magnitude of the residuals, are circled in red and identied in a red
rectangle.

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 8 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

F
O
O
R
P
D
E
T
C

Fig. 5. Visual comparison of amygdala (blue) and hippocampus (red) volume estimation in a single subject using manual segmentation and FSL-FIRST. These subjects were selected on the
E

basis of linear regression analyses, due to a poor correspondance between manually segmented and FSL-FIRST derived volumes. A left amygdala, B left hippocampus, C right
amygdala, D right hippocampus.
R

557 associated to the resampling of labels. Further, this tendency for volume However, it cannot be excluded that differences in volumes obtained 582
R

558 overestimation has been reported in earlier work in non-pediatric pop- between manual segmentation and automated protocols reect varia- 583
559 ulations (Cherbuin et al., 2009; Doring et al., 2011; Morey et al., 2009; tions in the denition of anatomical boundaries between segmentation 584
O

560 Nugent et al., 2013; Pipitone et al., 2014; Snchez-Benavides et al., protocols. Manual segmentation of the hippocampus and amygdala 585
561 2010; Shen et al., 2010; Tae et al., 2008). Between the two automated performed in this study was based on the protocol established by 586
Q9
562 approaches, FreeSurfer was found to yield the largest volume estimates. Pruessner, J., et al., 2000 (Pruessner et al., 2000). FreeSurfer and FSL- 587
C

563 Our results further showed that the overestimation of volumes associat- FIRST pipelines are based on manual labels provided by the Center for 588
564 ed with automated segmentation was more pronounced for the amyg- Morphometric Analysis, part of the Massachusetts General Hospital. 589
N

565 dala than for the hippocampus. This was true for both automated More details on the segmentation protocols used by this Center can be 590
566 methods, but was also more pronounced with the FreeSurfer method. found at www.cma.mgh.harvard.edu/manuals/segmentation. The pro- 591
567 To better understand the origin of volumetric overestimation that tocol used for the manual segmentations in this article systematically 592
U

568 occur with these automated techniques, 3D neuroanatomical maps excludes the Andreas-Retzius and the Fasciolar gyrus from the tail of 593
569 representing the average percentage of difference between automatic the hippocampus. Also, this protocol takes extra care to avoid including 594
570 and manual segmentation were computed to localize areas of disagree- the inferior horn of the lateral ventricle, even in subjects where it might 595
571 ment. A qualitative revision of those maps revealed that areas of dis- not be clearly apparent, by excluding voxels in the infero-lateral portion 596
572 agreement were located at the border of the target structures, found of the hippocampus with ambiguous signal intensity. This exclusion 597
573 in all dimensions (x-y-z axis), rather than in one specic location, or takes place even if in one slice these voxels appear as gray matter, but 598
574 in only one dimension. This suggests that the difference in volumes the existence of the inferior horn can be extrapolated from neighboring 599
575 was likely not a result of differences in the anatomical denition of the slices. Such an approach is likely not present in automatic segmentation 600
576 target structures, but rather a too liberal inclusion of voxels towards methods for hippocampal volumes, and thus can be expected to result 601
577 the structure boundaries. This might perhaps be explained by partial in somewhat larger volume estimates. However, the amount of volume 602
578 volume effects, which can lead to incorrect inclusion of voxels neighbor- that would be generated by the inclusion of the AndreasRetzius gyrus 603
579 ing the target structure. Thus, it appears likely that automatic segmenta- and the lateral ventricle can be estimated not to be more than 5% addi- 604
580 tion techniques that were tested are more susceptible to partial volume tional volume, which is far inferior to the volume differences observed 605
581 segmentation faults when compared to manual segmentation. between the automated methods and the manual one in the current 606

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx 9

F
O
O
R
P
D
E
T
C

Fig. 6. Visual comparison of amygdala (blue) and hippocampus (red) volume estimation in a single subject using manual segmentation and FreeSurfer. These subjects were selected on the
basis of linear regression analyses, due to a poor correspondence between manually segmented and FreeSurfer derived volumes. A left amygdala, B left hippocampus, C right
E

amygdala, D right hippocampus.

R

607 study. In addition, other anatomical boundaries present in the manual control to remove scans with apparent motion artifacts, it cannot be ex- 634
608 method protocol appear to match well with those of the automated cluded that motion affected the quality of the results from the two auto- 635
609 ones. These areas include the superiolateral white matter bands of the matic segmentation protocols. Further, even in scans of high quality, 636
R

610 hippocampus, the fornix and more anterior, the mbria and the alveus. superior and lateral boundaries of the amygdala with the basal ganglia, 637
611 Also, both the manual and the automated segmentation method include inferior boundaries with the hippocampus and lateral-inferior bound- 638
O

612 at least part of the subiculum. Thus, differences anatomical boundaries aries with the entorhinal cortex can be difcult to dene based on signal 639
613 between segmentation protocols could be expected to result in volume intensity, and are highly variable across subjects due to anatomical het- 640
614 changes of around 5%, with the automated methods generating larger erogeneity. Consequently, manual segmentation protocols often rely on 641
C

615 volumes than the manual one. This is clearly not what is seen, as the au- the visualization of the area by a trained anatomist, recognition of the 642
616 tomated methods generate hippocampus volumes that are approxi- various structures in the eld of view, and an expert decision as to 643
N

617 mately 28% (FSL) and 55% (Freesurfer) larger than the manual ones. where exactly the boundary to surrounding structures is located for 644
618 This additional overestimation could be the consequence of using a that particular subject. This is a procedure that is time and labor inten- 645
619 standard brain template derived from mature adult brains compared sive but favors anatomical precision and validity. Automated methods, 646
U

620 to a pediatric population. Future studies should determine whether in comparison, can't rely on an expert rater's decision in ambiguous cir- 647
621 using a common space based on pediatric brains, which would be cumstances, and have to employ probabilities and intensity distribu- 648
622 more representative of this population's neuroanatomy, could poten- tions instead. Future studies investigating differences in the 3-D shape 649
623 tially improve the accuracy of automated segmentation techniques. An- of the hippocampus and amygdala segmented manually or with auto- 650
624 other possible cause for this additional discrepancy can be seen in Fig. 6, mated techniques could allow a better understanding of the discrepan- 651
625 which illustrates for selected subjects that both automated methods cy in volume observed when comparing manual and automatic 652
626 suffer from inclusion of ventricle space, neighboring gray matter struc- segmentation. 653
627 tures, and white matter in their segmentations. There are probably mul- Volume overestimation does not necessarily imply a lack of validity 654
628 tiple reasons for the inclusion of these structures and areas not part of of automatic segmentation as long as it is done in a consistent manner. 655
629 the target structure. Signal intensity might vary depending on scan Thus, to assess consistency in volume estimation, we computed pearson 656
630 quality and motion artifacts, which may lead to a less precise differenti- correlations between volumes derived from automatic methods and 657
631 ation and classication of structures by automated techniques. This manual segmentation. The guiding idea was that a consistent overesti- 658
632 might be especially signicant in children, who are more likely to mation of volumes would not weaken correlations between segmenta- 659
633 move during scan acquisition. Although, we performed a visual quality tion techniques and could thus still support the validity of automated 660

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 10 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

F
O
O
R
P
D
E
T
C
E
R
R

Fig. 7. Within-method correlations of left versus right structure volumes. A Correlation between left and right hippocampus volumes segmented manually. B Correlation between left
and right amygdala volumes segmented manually. C Correlation between left and right hippocampus volumes segmented automatically with FreeSurfer. D Correlation between left
and right amygdala volumes segmented automatically with FreeSurfer. E Correlation between left and right hippocampus volumes segmented automatically with FSL-FIRST. F
O

Correlation between left and right amygdala volumes segmented automatically with FSL-FIRST. r Pearson correlation coefcient.

661 techniques relative to manual segmentation. Associations between contributed to lower correlations found in this study, similar correla- 681
C

662 FreeSurfer and manual segmentation were satisfactory for the hippo- tions between FSL-FIRST and manual segmentation have been 682
663 campus volumes and ranged between r = 0.69 to r = 0.77. These cor- highlighted in past studies performed on adults/mature brains 683
N

664 relations are consistent with what as previously been reported in the (Pardoe et al., 2009; Doring et al., 2011). For amygdala volumes derived 684
665 literature (Cherbuin et al., 2009; Doring et al., 2011; Morey et al., using FSL-FIRST, correlations with manual segmentation and FSL-FIRST 685
666 2009; Pardoe et al., 2009; Pipitone et al., 2014; Snchez-Benavides can be considered weak (r = 0.31 to r = 035). The poor correlation be- 686
U

667 et al., 2010), which usually supports correlation coefcients surround- tween FSL-FIRST and manual segmentation for assessment of the amyg- 687
668 ing r = 0.75. Correlations between amygdala volumes derived dala volume has also been reported previously (Morey et al., 2009). The 688
669 FreeSurfer and manual segmentation were weaker than for the hippo- assessment of reproducibility of measurements with ICC suggests a 689
670 campus and ranged between r = 0.61 and r = 0.67. These estimates weak agreement between manual segmentation and automated 690
671 are consistent with what has been found by Morey et al. (2009). How- methods. The only comparison that reached or exceeded a coefcient 691
672 ever, few studies have looked at the accuracy of FreeSurfer to estimate of 0.70, a threshold previously dened as the minimum to dene reli- 692
673 amygdala volume, thus it is difcult to compare our results with previ- ability between measures (Nunnally, J.C., et al., 1967; Terwee et al., 693
Q10
674 ous ndings. Past studies comparing manually segmented hippocam- 2007), was the left hippocampus volume measured with FreeSurfer 694
675 pus volumes to volumes obtained with FSL-FIRST reported Pearson and manual segmentation. All other volumes, from FreeSurfer or FSL- 695
676 correlation coefcients varied between r = 0.47(Pardoe et al., 2009) FIRST, failed to reach this minimum standard to support adequate 696
677 and r = 0.67(Nugent et al., 2013). The results we obtained performing agreement with manual segmentation. Two key observations could be 697
678 similar analyses highlight correlations closer to lower estimates that derived from Pearson correlations and ICCs analyses. First, the agree- 698
679 have been reported in the past (r = 0.51 to r = 059). While neuroana- ment between manual and automated segmentation tended to be 699
680 tomical characteristics of the studied pediatric population could have stronger for hippocampus than amygdala volumes. This amygdala 700

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx 11

F
O
O
R
P
D
E
T
C
E
R
R
O
C
N

Fig. 8. BlandAltman plots of volume difference estimation between manual segmentation and (A) FreeSurfer or (B) FSL-FIRST. Plots are presented separately for i left hippocampus ii
right hippocampus, iii left amygdala, iv right amygdala. A red regression line was integrated to each plot to illustrate potential biases in volume estimation.
U

701 hippocampus discrepancy was also observed in previous articles study- the amygdala and hippocampus volume. Indeed, FreeSurfer consistent- 715
702 ing the validity of automated segmentation in medial temporal lobe ly yielded larger correlations and ICC coefcients with manual segmen- 716
703 structures (Morey et al., 2009). Poor associations found with amygdala tation than FSL-FIRST. It seems that the advantage of FreeSurfer over 717
704 volumes are possibly the consequence of the neuroanatomical com- FSL-FIRST is not specic to our population as it was outlined in previous 718
705 plexity of this structure. In addition to poor agreement with manual articles comparing results from both segmentation techniques as well 719
706 segmentation, the amygdala volume was also shown to have a low (Doring et al., 2011; Morey et al., 2009; Pardoe et al., 2009). Using native 720
707 scan-rescan reliability when estimated with automatic techniques manual segmentation volumes computed either by resampling labels to 721
708 (Morey et al., 2010), most likely due to a high susceptibility to small var- the native space or by dividing volumes of labels in the standard space 722
709 iations in image intensity. This suggests that the amygdala volume is by scale factors associated with the linear transformation did not signif- 723
710 particularly difcult to assess reliably and vulnerable to errors when es- icantly alter results of these analyses and did not change our ndings 724
711 timated with automated methods. The second observation that was (see Table 4). 725
712 noted both in between-method correlations and ICC analyses was that An approach commonly used to establish the validity of automated 726
713 the association between automatic and manual segmentation volumes techniques is to dene their accuracy in distinguishing individuals 727
714 was stronger with FreeSurfer than FSL-FIRST. This was true for both from different clinical groups (eg. Alzheimer Disease versus Normal 728

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 12 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

t4:1 Table 4

t4:2 Native manual segmentation volumes based on Native manual segmentation volumes based on
resampling of labels calculation of scaling factors
(as presented in the article)

t4:3 FreeSurfer FSL-FIRST FreeSurfer FSL-FIRST

t4:4 % volume diff. (SD)

t4:5 L-hippocampus 60.38 (13.04) 27.61(14.49) 60.16 (12.90) 27.43 (14.28)
t4:6 R-hippocampus 51.53 (13.17) 28.39 (13.07) 51.41 (13.09) 28.28 (12.89)
t4:7 L-amygdala 100.29 (24.56) 50.32 (27.65) 99.53 (24.27) 49.76 (27.55)
t4:8 R-amygdala 93.56 (25.78) 40.29 (26.09) 93.11 (24.56) 40.03 (25.83)
t4:9 PCC with manual seg.
t4:10 L-hippocampus 0.77 0.51 0.77 0.52
t4:11 R-hippocampus 0.69 0.59 0.70 0.60
t4:12 L-amygdala 0.67 0.31 0.66 0.30
t4:13 R-amygdala 0.61 0.35 0.62 0.35

F
t4:14 ICC with manual seg.
t4:15 L-hippocampus 0.74 0.51 0.74 0.52
t4:16 R-hippocampus 0.68 0.59 0.69 0.60

O
t4:17 L-amygdala 0.65 0.30 0.64 0.28
t4:18 R-amygdala 0.60 0.33 0.61 0.33

t4:19 Percentage of volume difference (% volume diff.), Pearson correlation coefcients (PCC) and intraclass correlation coefcients (ICC) computed between manual segmentation volumes and

O
t4:20 automatic protocols. Results are presented with native manual segmentation volumes obtained by resampling labels in the native space using an inversion of the linear transformation (left
t4:21 column) and with manual volumes obtained by dividing volumes segmented in the standard space by scale factors associated with the linear transformation in the x,y,z directions (right
Q1
t4:22 column). This table shows that both methods of estimating manual segmentation volumes in the native space lead to highly similar results. L left. R right. SD standard deviation.

R
P
729 aging patients). A limitation associated with this study lies in the ab- reveal important between-subject heterogeneity in the development of 771
730 sence subgroups in the studied population. However, to arrive at an as- the hippocampal structure during brain development. Automated tech- 772
731 sessment of consistency for each method independently, we used niques are likely to be less exible and accurate when dealing with irreg-
D 773
732 between hemisphere correlations to demonstrate the differences be- ular shape. On the other hand, an expert in neuroanatomy and 774
733 tween methods. If manual and automated segmentations were inter- hippocampus segmentation should not be affected by variance in 775
734 changeable, it would be expected that the associations between left shape. Future studies should aim to investigate the impact associated 776
E
735 and right volumes would be similar regardless of the difference in struc- with variations in the shape of neuroanatomical structures in the context 777
736 tural denition associated with the segmentation protocol. Further, it of automatic segmentation validation. To illustrate cases were there is an 778
T

737 can be expected that, within the individual subject, left versus right important discrepancy between automated and manual segmentation, 779
738 hemispheric volumes are moderately to strongly associated (Allen we selected subjects that deviated from the regression line and visually 780
C

739 et al., 2002). Both manual segmentation and FreeSurfer seemed to sup- compared labels obtained with both techniques. In addition to corrobo- 781
740 port this last statement, with results showing strong correlations be- rating the overestimation reported in previous analyses, these images 782
741 tween left versus right hemisphere for both the amygdala and the show a tendency for automated methods to miss the borders of target 783
E

742 hippocampus. Correlations between bilateral amygdala and hippocam- structures and expand into adjacent areas, including ventricular space. 784
743 pus volumes were signicantly weaker when estimated with FSL-FIRST. For those subjects, the obtained volume is not anatomically valid and 785
R

744 Thus, within-method correlations suggest that the FSL-FIRST method should not be used in subsequent analyses. This highlights the impor- 786
745 might be prone to inconsistencies in segmentation within the same sub- tance of quality control and, when needed, corrections of labels obtained 787
746 ject. The scope of this study was to investigate two key structures of the automatically. This process is time and labor intensive and is rarely per- 788
R

747 medial temporal lobe, the hippocampus and amygdala. While results formed thoroughly. Both FreeSurfer and FSL-FIRST include documenta- 789
748 highlighted in this article are likely to extend to adjacent structures in tion and guidelines on quality control. However, to reect the way 790
O

749 the medial temporal lobe, and perhaps to the rest of the cortex, our nd- groups lacking the training and expertise in anatomy would use these 791
750 ings remain specic to those two key structures. Future studies investi- tools, we did not apply any form of correction of the labels derived 792
751 gating the agreement between manual and automated segmentation from automated methods in the current article. A careful and informed 793
C

752 using a more global approach and looking as spatial relationships be- quality control and manual corrections of automatically obtained labels 794
753 tween segmented structures would provide important additional by a trained individual would likely lead to signicantly improved associ- 795
N

754 information. ations between manual segmentation and automatic techniques, espe- 796
755 When looking at associations between manual segmentation and cially when used in special populations like the one used in the current 797
756 both automated techniques and the overall t to the regression line, a study. 798
U

757 considerable number of outliers could be visually identied. In these

758 outliers, a marked discrepancy between automatic and manual segmen-
759 tation volume estimates is observed - contributing to a limited explana- 5. Conclusion 799
760 tion of variance. It is thus possible that the automated segmentation
761 tends to be particularly inaccurate for some subjects. The BlandAltman In this study we highlight differences in volumes of structures seg- 800
762 diagrams seem to support that notion by indicating a trend for larger vol- mented manually or obtained with automatic techniques, in this case 801
763 ume difference between manual and automated segmentation for indi- FreeSurfer and FSL-FIRST. We provide evidence that, in a pediatric pop- 802
764 viduals with smaller structure volumes. Variations in scan quality or ulation, volumes obtained with those techniques might not always be 803
765 even in anatomy could contribute to this variability in performance. The equivalent to volumes obtained when manually segmented by an ana- 804
766 hippocampus shape and volume are known to be highly variable across tomical expert. This is especially true for more complex structures, 805
767 normal subjects (Bouix et al., 2005; Lupien et al., 2007). Studies looking such as the amygdala. Our results also support a better consistency be- 806
768 at hippocampal shape in pediatric populations highlighted variations in tween manual segmentation and FreeSurfer than FSL-FIRST. With these 807
769 the hippocampal shape over the course of normal development results, we hope to emphasize the importance of performing quality 808
770 (Gogtay et al., 2006; Lin et al., 2013). Additionally, Gogtay et al. (2006) control on volumes obtained automatically. A validated and well- 809

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx 13

810 established quality control protocol could signicantly improve the cor- Janssen, I., Krabbendam, L., Bak, M., Hanssen, M., Vollebergh, W., Graaf, R.d., Os, J.v., 2004. 887
Childhood abuse as a risk factor for psychotic experiences. Acta Psychiatr. Scand. 109, 888
811 respondence between automatic and manual segmentation volumes. 3845. 889
Johnson, J.G., Cohen, P., Brown, J., Smailes, E.M., Bernstein, D.P., 1999. Childhood maltreat- 890
ment increases risk for personality disorders during early adulthood. Arch. Gen. Psy- 891
812 Acknowledgements chiatry 56, 600606. 892
Krouwer, J.S., 2008. Why BlandAltman plots should use X, not (Y + X) / 2 when X is a 893
reference method. Stat. Med. 27, 778780. 894
813 This work was supported by the NIH R01 HD 50662 (EPD), R01 HD Lin, M., Fwu, P.T., Buss, C., Davis, E.P., Head, K., Muftuler, L.T., Sandman, C.A., Su, M.-Y., 895
814 51852 (CAS), P50 MH 096889 (EPD, CAS). DS doctoral training is funded 2013. Developmental changes in hippocampal shape among preadolescent children. 896
Q11 by Fonds de recherche du Qubec Sant (FRQS) doctoral award. JCP is
815 Int. J. Dev. Neurosci. 31, 473481. 897
Lupien, S., Evans, A., Lord, C., Miles, J., Pruessner, M., Pike, B., Pruessner, J., 2007. Hippo- 898
816
Q12 supported by an FRQS Chercheur National salary award. campal volume is as variable in young as in older adults: implications for the notion 899
of hippocampal atrophy in humans. NeuroImage 34, 479485. 900
MacMillan, H.L., Fleming, J.E., Streiner, D.L., Lin, E., Boyle, M.H., Jamieson, E., Duku, E.K., 901
817 References Walsh, C.A., Wong, M.Y.-Y., Beardslee, W.R., 2001. Childhood abuse and lifetime psy- 902
chopathology in a community sample. Childhood 158. 903
818 Adolphs, R., Tranel, D., Damasio, H., Damasio, A., 1994. Impaired recognition of emotion in Matsuoka, Y., Mori, E., Inagaki, M., Kozaki, Y., Nakano, T., Wenner, M., Uchitomi, Y., 2003. 904
819 905

F
facial expressions following bilateral damage to the human amygdala. Nature 372, Manual tracing guideline for volumetry of hippocampus and amygdala with high-
820 669672. resolution MRI. No To Shinkei 55, 690697. 906
821 Akhondi-Asl, A., Jafari-Khouzani, K., Elisevich, K., Soltanian-Zadeh, H., 2011. Hippocampal Meng, X.-L., Rosenthal, R., Rubin, D.B., 1992. Comparing correlated correlation coefcients. 907

O
822 volumetry for lateralization of temporal lobe epilepsy: automated versus manual Psychol. Bull. 111, 172. 908
823 methods. NeuroImage 54, S218S226. Morey, R.A., Petty, C.M., Xu, Y., Pannu Hayes, J., Wagner II, H.R., Lewis, D.V., LaBar, K.S., Styner, 909
824 Allen, J.S., Damasio, H., Grabowski, T.J., 2002. Normal neuroanatomical variation in the M., McCarthy, G., 2009. A comparison of automated segmentation and manual tracing 910
825 human brain: an MRI-volumetric study. Am. J. Phys. Anthropol. 118, 341358. for quantifying hippocampal and amygdala volumes. NeuroImage 45, 855866. 911

O
826 Arnold, S.E., Trojanowski, J.Q., 1996. Human fetal hippocampal development: I. Morey, R.A., Selgrade, E.S., Wagner, H.R., Huettel, S.A., Wang, L., McCarthy, G., 2010. Scan 912
827 Cytoarchitecture, myeloarchitecture, and neuronal morphologic features. J. Comp. rescan reliability of subcortical brain volumes derived from automated segmentation. 913
828 Neurol. 367, 274292. Hum. Brain Mapp. 31, 17511762. 914

R
829 Bland, J.M., Altman, D., 1986. Statistical methods for assessing agreement between two Nugent, A.C., Luckenbaugh, D.A., Wood, S.E., Bogers, W., Zarate, C.A., Drevets, W.C., 2013. Au- 915
830 methods of clinical measurement. Lancet 327, 307310. tomated subcortical segmentation using FIRST: testretest reliability, interscanner reli- 916
831 Bonne, O., Brandes, D., Gilboa, A., Gomori, J.M., Shenton, M.E., Pitman, R.K., Shalev, A.Y., ability, and comparison to manual segmentation. Hum. Brain Mapp. 34, 23132329. 917
832 918

P
2001. Longitudinal MRI study of hippocampal volume in trauma survivors with Nunnally, J.C., Bernstein, I.H., Berge, J.M.t., 1967. Psychometric Theory. McGraw-Hill, New
833 PTSD. Am. J. Psychiatry 158, 1248. York. 919
834 Bouix, S., Pruessner, J.C., Collins, D.L., Siddiqi, K., 2005. Hippocampal shape analysis using Pardoe, H.R., Pell, G.S., Abbott, D.F., Jackson, G.D., 2009. Hippocampal volume assessment 920
835 medial surfaces. NeuroImage 25, 10771089. in temporal lobe epilepsy: how good is automated segmentation? Epilepsia 50, 921
836 Burdock, E.I., Fleiss, J.L., Hardesty, A.S., 1963. A new view of inter-observer agreement1. D 25862592. 922
837 Pers. Psychol. 16, 373384. Patenaude, B., 2007. Bayesian Statistical Models of Shape and Appearance for Subcortical 923
838 Buss, C., Davis, E.P., Shahbaba, B., Pruessner, J.C., Head, K., Sandman, C.A., 2012. Maternal Brain Segmentation. University of Oxford. 924
839 cortisol over the course of pregnancy and subsequent child amygdala and hippocam- Patenaude, B., Smith, S.M., Kennedy, D.N., Jenkinson, M., 2011. A Bayesian model of shape 925
E
840 pus volumes and affective problems. Proc. Natl. Acad. Sci. 109, E1312E1319. and appearance for subcortical brain segmentation. NeuroImage 56, 907922. 926
841 Campbell, S., MacQueen, G., 2004. The role of the hippocampus in the pathophysiology of Paus, T., Keshavan, M., Giedd, J.N., 2008. Why do many psychiatric disorders emerge dur- 927
842 major depression. J. Psychiatry Neurosci. 29, 417. ing adolescence? Nat. Rev. Neurosci. 9, 947957. 928
T

843 Chakravarty, M.M., Rapoport, J.L., Giedd, J.N., Raznahan, A., Shaw, P., Collins, D.L., Lerch, Pipitone, J., Park, M.T.M., Winterburn, J., Lett, T.A., Lerch, J.P., Pruessner, J.C., Lepage, M., 929
844 J.P., Gogtay, N., 2014. Striatal shape abnormalities as novel neurodevelopmental Voineskos, A.N., Chakravarty, M.M., Initiative, A.s.D.N., 2014. Multi-atlas segmenta- 930
845 endophenotypes in schizophrenia: a longitudinal study. Hum. Brain Mapp. tion of the whole hippocampus and subelds using multiple automatically generated 931
C

846 Cherbuin, N., Anstey, K.J., Rglade-Meslin, C., Sachdev, P.S., 2009. In vivo hippocampal templates. NeuroImage 101, 494512. 932
847 measurement and memory: a comparison of manual tracing and automated segmen- Pruessner, J., Li, L., Serles, W., Pruessner, M., Collins, D., Kabani, N., Lupien, S., Evans, A., 933
848 tation in a large community-based sample. PLoS One 4, e5265. 2000. Volumetry of hippocampus and amygdala with high-resolution MRI and 934
E

849 Collins, D.L., Neelin, P., Peters, T.M., Evans, A.C., 1994. Automatic 3D intersubject registra- three-dimensional analysis software: minimizing the discrepancies between labora- 935
850 tion of MR volumetric data in standardized talairach space. J. Comput. Assist. Tomogr. tories. Cereb. Cortex 10, 433442. 936
851 18, 192205. Pruessner, J.C., Khler, S., Crane, J., Pruessner, M., Lord, C., Byrne, A., Kabani, N., Collins, D.L., 937
R

852 Davis, E.P., Sandman, C.A., Buss, C., Wing, D.A., Head, K., 2013. Fetal glucocorticoid expo- Evans, A.C., 2002. Volumetry of temporopolar, perirhinal, entorhinal and 938
853 sure is associated with preadolescent brain development. Biol. Psychiatry 74, parahippocampal cortex from high-resolution MR images: considering the variability 939
854 647655. of the collateral sulcus. Cereb. Cortex 12, 13421353. 940
855 Davis, M., Whalen, P.J., 2001. The amygdala: vigilance and emotion. Mol. Psychiatry 6, Pynoos, R.S., Steinberg, A.M., Piacentini, J.C., 1999. A developmental psychopathology 941
R

856 1334. model of childhood traumatic stress and intersection with anxiety disorders. Biol. 942
857 Donegan, N.H., Sanislow, C.A., Blumberg, H.P., Fulbright, R.K., Lacadie, C., Skudlarski, P., Psychiatry 46, 15421554. 943
858 Gore, J.C., Olson, I.R., McGlashan, T.H., Wexler, B.E., 2003. Amygdala hyperreactivity Raghunathan, T.E., Rosenthal, R., Rubin, D.B., 1996. Comparing correlated but nonoverlap- 944
O

859 in borderline personality disorder: implications for emotional dysregulation. Biol. ping correlations. Psychol. Methods 1, 178. 945
860 Psychiatry 54, 12841293. Rauch, S.L., Whalen, P.J., Shin, L.M., McInerney, S.C., Macklin, M.L., Lasko, N.B., Orr, S.P., 946
861 Doring, T.M., Kubo, T.T., Cruz, L.C.H., Juruena, M.F., Fainberg, J., Domingues, R.C., Pitman, R.K., 2000. Exaggerated amygdala response to masked facial stimuli in post- 947
862 948
C

Gasparetto, E.L., 2011. Evaluation of hippocampal volume based on MR imaging in traumatic stress disorder: a functional MRI study. Biol. Psychiatry 47, 769776.
863 patients with bipolar affective disorder applying manual and automatic segmentation Raznahan, A., Shaw, P.W., Lerch, J.P., Clasen, L.S., Greenstein, D., Berman, R., Pipitone, J., 949
864 techniques. J. Magn. Reson. Imaging 33, 565572. Chakravarty, M.M., Giedd, J.N., 2014. Longitudinal four-dimensional mapping of sub- 950
865 Fischl, B., Salat, D.H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., cortical anatomy in human development. Proc. Natl. Acad. Sci. 111, 15921597. 951
N

866 Killiany, R., Kennedy, D., Klaveness, S., 2002. Whole brain segmentation: automated Rodionov, R., Chupin, M., Williams, E., Hammers, A., Kesavadas, C., Lemieux, L., 2009. Eval- 952
867 labeling of neuroanatomical structures in the human brain. Neuron 33, 341355. uation of atlas-based segmentation of hippocampi in healthy humans. Magn. Reson. 953
868 Fox, N., Warrington, E., Freeborough, P., Hartikainen, P., Kennedy, A., Stevens, J., Rossor, Imaging 27, 11041109. 954
U

869 M.N., 1996. Presymptomatic hippocampal atrophy in Alzheimer's disease a longitudi- Snchez-Benavides, G., Gmez-Ansn, B., Sainz, A., Vives, Y., Delno, M., Pea-Casanova, J., 955
870 nal MRI study. Brain 119, 20012007. 2010. Manual validation of FreeSurfer's automated hippocampal segmentation in 956
871 Giedd, J.N., Raznahan, A., Alexander-Bloch, A., Schmitt, E., Gogtay, N., Rapoport, J.L., 2015. normal aging, mild cognitive impairment, and Alzheimer disease subjects. Psychiatry 957
872 Child psychiatry branch of the national institute of mental health longitudinal struc- Res. Neuroimaging 181, 219225. 958
873 tural magnetic resonance imaging study of human brain development. Shen, L., Saykin, A.J., Kim, S., Firpi, H.A., West, J.D., Risacher, S.L., McDonald, B.C., McHugh, T.L., 959
874 Neuropsychopharmacology 40, 4349. Wishart, H.A., Flashman, L.A., 2010. Comparison of manual and automated determina- 960
875 Gogtay, N., Nugent, T.F., Herman, D.H., Ordonez, A., Greenstein, D., Hayashi, K.M., Clasen, tion of hippocampal volumes in MCI and early AD. Brain Imaging Behav. 4, 8695. 961
876 L., Toga, A.W., Giedd, J.N., Rapoport, J.L., 2006. Dynamic mapping of normal human Shrout, P.E., Fleiss, J.L., 1979. Intraclass correlations: uses in assessing rater reliability. 962
877 hippocampal development. Hippocampus 16, 664672. Psychol. Bull. 86, 420. 963
878 Herpertz, S.C., Dietrich, T.M., Wenning, B., Krings, T., Erberich, S.G., Willmes, K., Thron, A., Sled, J.G., Zijdenbos, A.P., Evans, A.C., 1998. A nonparametric method for automatic correc- 964
879 Sass, H., 2001. Evidence of abnormal amygdala functioning in borderline personality tion of intensity nonuniformity in MRI data. IEEE Trans Med Imaging 17, 8797. 965
880 disorder: a functional MRI study. Biol. Psychiatry 50, 292298. Springer, K.W., Sheridan, J., Kuo, D., Carnes, M., 2007. Long-term physical and mental 966
881 Jack, C., Bentley, M., Twomey, C., Zinsmeister, A., 1990. MR imaging-based volume mea- health consequences of childhood physical abuse: results from a large population- 967
882 surements of the hippocampal formation and anterior temporal lobe: validation based sample of men and women. Child Abuse Negl. 31, 517530. 968
883 studies. Radiology 176, 205209. Squire, L.R., 1992. Memory and the hippocampus: a synthesis from ndings with rats, 969
884 Jack Jr., C.R., Theodore, W.H., Cook, M., McCarthy, G., 1995. MRI-based hippocampal volu- monkeys, and humans. Psychol. Rev. 99, 195. 970
885 metrics: data acquisition, normal ranges, and optimal protocol. Magn. Reson. Imaging Steiger, J.H., 1980. Tests for comparing elements of a correlation matrix. Psychol. Bull. 87, 971
886 13, 10571064. 245. 972

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038
 14 D. Schoemaker et al. / NeuroImage xxx (2016) xxxxxx

973 Stein, M.B., Goldin, P.R., Sareen, J., Zorrilla, L.T.E., Brown, G.G., 2002. Increased amygdala Terwee, C.B., Bot, S.D., de Boer, M.R., van der Windt, D.A., Knol, D.L., Dekker, J., Bouter, L.M., 983
974 activation to angry and contemptuous faces in generalized social phobia. Arch. Gen. de Vet, H.C., 2007. Quality criteria were proposed for measurement properties of 984
975 Psychiatry 59, 1027. health status questionnaires. J. Clin. Epidemiol. 60, 3442. 985
976 Tae, W.S., Kim, S.S., Lee, K.U., Nam, E.-C., Kim, K.W., 2008. Validation of hippocampal Watson, C., Andermann, F., Gloor, P., Jones-Gotman, M., Peters, T., Evans, A., Olivier, A., 986
977 volumes measured using a manual method and two automated methods Melanson, D., Leroux, G., 1992. Anatomic basis of amygdaloid and hippocampal vol- 987
978 (FreeSurfer and IBASPM) in chronic major depressive disorder. Neuroradiology ume measurement by magnetic resonance imaging. Neurology 42, 1743-1743. 988
979 50, 569581. Woon, F.L., Hedges, D.W., 2008. Hippocampal and amygdala volumes in children and 989
980 Teicher, M.H., Andersen, S.L., Polcari, A., Anderson, C.M., Navalta, C.P., Kim, D.M., 2003. The adults with childhood maltreatment-related posttraumatic stress disorder: a meta- 990
981 neurobiological consequences of early stress and childhood maltreatment. Neurosci. analysis. Hippocampus 18, 729736. 991
982 Biobehav. Rev. 27, 3344.

992

F
O
O
R
P
D
E
T
C
E
R
R
O
C
N
U

Please cite this article as: Schoemaker, D., et al., Hippocampus and amygdala volumes from magnetic resonance images in children: Assessing
accuracy of FreeSurfer and FSL against manu..., NeuroImage (2016), http://dx.doi.org/10.1016/j.neuroimage.2016.01.038