NeuroImage 47 (2009) 15321544

Contents lists available at ScienceDirect

NeuroImage
j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / y n i m g

Single-trial P3 amplitude and latency informed event-related fMRI models yield different BOLD response patterns to a target detection task
T. Warbrick a,b,, A. Mobascher a,b, J. Brinkmeyer a,b, F. Musso a, N. Richter a, T. Stoecker b, G.R. Fink b,c, N.J. Shah b, G. Winterer a,b
a b c

Neuropsychiatric Research Laboratory, Department of Psychiatry, Heinrich-Heine University, Bergische Landstr. 2, 40629 Duesseldorf, Germany Institute of Neurosciences and Medicine, Helmholtz Research Center Juelich, Germany Department of Neurology, University of Cologne, Germany

a r t i c l e

i n f o

a b s t r a c t
Using single-trial parameters as a regressor in the General Linear Model (GLM) is becoming an increasingly popular method for informing fMRI analysis. However, the parameter used to characterise or to differentiate brain regions involved in the response to a particular task varies across studies (e.g. ERP amplitude, ERP latency, reaction time). Furthermore, the way in which the single-trial information is used in the fMRI analysis is also important. For example, the single-trial parameters can be used as regressors in the GLM or to modify the duration of the events modelled in the GLM. The aim of this study was to investigate the BOLD response to a target detection task when including P3 amplitude, P3 latency and reaction time parameters in the GLM. Simultaneous EEG-fMRI was recorded from fteen subjects in response to a visual choice reaction time task. Including P3 amplitude as a regressor in the GLM yielded activation in left central opercular cortex, left postcentral gyrus, left insula, left middle frontal gyrus, left insula and left parietal operculum. Using P3 latency and reaction time as an additional regressor yielded no additional activation in comparison with the conventional fMRI analysis. However, when P3 latency or reaction time was used to determine the duration of events at a single-trial level, additional activation was observed in the left postcentral gyrus, left precentral gyrus, anterior cingulate cortex and supramarginal gyrus. Our ndings suggest that ERP amplitudes and latencies can yield different activation patterns when used to modify relevant aspects of the GLM. 2009 Elsevier Inc. All rights reserved.

Article history: Received 4 December 2008 Revised 19 May 2009 Accepted 26 May 2009 Available online 6 June 2009 Keywords: Event Related Potentials (ERPs) functional Magnetic Resonance Imaging (fMRI) Single-trial P3 amplitude P3 latency Multimodal imaging

Introduction The attraction of simultaneous electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) recording lies in the combination of high spatial resolution of fMRI and superb temporal resolution of EEG for responses to the same task under the same conditions. A number of approaches have been used to fuse EEG and fMRI data; for example fMRI informed source modelling (Strobel et al., 2008) and modulation of separate session EEG and fMRI by parametric task manipulation (Horovitz et al., 2002). While these approaches have yielded valuable ndings, an approach receiving an increasing amount of attention is the use of single-trial parameters from the EEG signal as a regressor in the General Linear Model (GLM) when analysing fMRI data. The individual components of Event Related Potentials (ERPs) represent different aspects of stimulus processing on a millisecond time scale, a temporal resolution that is not available in fMRI data. Thus, using relevant ERP parameters in the analysis of fMRI

Corresponding author. Neuropsychiatric Research Laboratory, Department of Psychiatry, Heinrich-Heine University, Bergische Landstr. 2, 40629 Duesseldorf, Germany. Fax: +49 211 922 2759. E-mail address: t.warbrick@fz-juelich.de (T. Warbrick). 1053-8119/$ see front matter 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2009.05.082

data can identify a task/process related fMRI activation by exploiting the temporal resolution of EEG (Debener et al., 2005; Eichele et al., 2005; Mulert et al., 2008). One particular advantage of the single-trial approach is that it can utilise variability in responses that is lost during standard averaging. This variability is often viewed as noise but may reect differences in the response to individual trials (Bagshaw and Warbrick, 2007). Using single-trial ERP parameters in the fMRI model is a way of utilising the information provided by this single-trial variability to understand cognitive processes. This is particularly important in studies where variability in responses may be indicative of modulation of stimulus processing or task performance and not simply irrelevant noise (Debener et al., 2005; Eichele et al., 2005). Using single-trial ERP amplitude as a regressor when modelling the Blood Oxygen Level Dependant (BOLD) response has yielded different activation patterns for different ERP components (Eichele et al., 2005). Compared to conventional BOLD analysis, more specic activation patterns (Debener et al., 2005) as well as additional activation in task related areas (Mulert et al., 2008) have been reported. However, the parameters used to characterise or differentiate brain regions involved in the response to a particular task vary across studies. For example, ERP amplitude (Bnar et al., 2007; Debener et al., 2005; Eichele et al., 2005; Mobascher et al., 2009b;

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

1533

Mulert et al., 2008), the latency of the ERP component of interest (Bnar et al., 2007), the ERP time course (Mantini et al., 2009), reaction time (Hagenbeek et al., 2007; Grinband et al., 2008), and electrodermal activity (Mobascher et al., 2009a) have all been used to provide additional information to include in the GLM. Naturally, the choice of task related parameter should be guided by the research question, task and available data. An a priori selection of the appropriate parameter, however, requires specic knowledge of the activation patterns obtained as a result of those different parameters. In the case of detecting infrequent target stimuli, the amplitude and latency of the ERP P3 component, the P3 time course and reaction time may all be considered as possible task related parameters that could characterise or differentiate brain regions involved in target detection and decision making. The way in which the single-trial information is used in the fMRI analysis is also important. For example, an increasingly common method is to use the single-trial amplitude, or in some cases latency, as an additional regressor in the GLM. This is desirable when one is interested in brain activation that is sensitive to the single-trial variability of response intensity. However, it has been suggested that when using parameters related to the timing or duration of cognitive processes, such as the latency of ERP components or reaction times, using the single-trial information to modify the duration of the haemodynamic response function, i.e., the timing aspects of the model is more powerful (Grinband et al., 2008). To illustrate, the most common way to analyse event-related designs is to use an impulse function where the duration of the event is zero. When investigating cognitive processes, such as target discrimination, the decision process of interest extends some hundreds of milliseconds after stimulus onset. So, by using an event duration that captures this cognitive processing rather than just the instant of event onset, a more accurate representation of task related BOLD activation might be achieved. For example, Grinband et al. (2008) showed that using reaction time as the duration of an event resulted in more powerful and more reliable models. This not only makes sense from a cognitive perspective but also from an fMRI data analysis standpoint. The haemodynamic response to a decision making process has been shown to vary with the time taken for the subject to make a response (Connolly et al., 2005; Menon et al., 1998). This is a critical point for tasks where subjects are involved in a decision making process and provide a response. However, surveying recent fMRI studies published in a ve month period (JanMay 2007) Grinband et al. (2008) found that 84% of eventrelated studies with a decision component assumed that the time necessary to process a stimulus or generate a response was constant for all trials and all subjects. The study presented here focuses on using the single-trial ERP parameters and reaction time to model the BOLD response to a visual two choice reaction time task with infrequent target stimuli similar to that of an oddball task. The P3 component of the ERP represents target detection/event categorisation (Halgren et al., 1998; Picton, 1992), or more broadly, is considered to reect selective attention and working memory processes (Gur et al., 2007; Javitt et al., 2008). In oddball tasks, working memory is fundamental to top-down attention in the sense that whatever requires attention (for example, a visual target stimulus) has to be maintained in working memory. The working memory then biases competition between the multiple bottom-up items in the stimulus input; as a result, in the neuronal competition between the multiple inputs, the item that receives top-down bias from the working memory has an advantage (Rolls et al., 2008). Attention to task related stimuli will vary over the course of an experiment thus inuencing the competition between the multiple inputs and which items are successfully held in working memory. This variability in attention and working memory processing across experimental trials can be modelled by analysing data at the singletrial level.

The fMRI response to the same tasks that also evoke the ERP P3 in electrophysiological experiments involves a large distributed network including the supramarginal gyrus, frontal, insula, thalamus, cerebellum, occipitaltemporal, superior temporal and cingulate regions (Bledowski et al., 2004; Kiehl et al., 2005; Gur et al., 2007; Musso et al., 2006; Winterer et al., 2007; Strobel et al., 2008). Single-trial ERP informed fMRI analysis of responses to target detection tasks has predominantly involved using the ERP parameters as a regressor in the fMRI model (Bnar et al., 2007; Eichele et al., 2005; Mulert et al., 2008), with the exception of Mantini et al. (2009) who used P3 timecourse to inform their analysis. Thus, they obtained a P3 response time course for each subject which was then convolved with a canonical haemodynamic response function and used as a predictor in the GLM fMRI analysis. Using this method, they were able to differentiate two functionally distinct networks, i.e., the ventral (temporo-parietal junction, inferior and middle frontal gyri, anterior cingulate cortex) and dorsal (intraparietal sulcus, frontal eye eld, middle frontal gyrus) attention systems, suggesting that using singletrial ERP timing parameters can provide valuable information for modelling the BOLD response. However, what remains unclear from the study of Mantini et al. is what specic information sub-processes are related to the ventral and dorsal attentional network. The aim of this study is to use simultaneously acquired EEG, fMRI and behavioural data to assess whether different information subprocesses as reected by single-trial ERP amplitude and latency parameters can be used to differentiate brain regions of the ventral and dorsal attentional network involved in target detection processes. Specically, the amplitude of the P3, the latency of the P3 and the reaction time to each target stimulus will be used to inform the fMRI analysis. Methods Subjects Twenty one healthy subjects participated in the study (10 male, mean age 35.1 years (SD = 13.3)). Subjects were recruited from a large population-based database in Germany with no history of medical, neurological or psychiatric illness or alcohol and drug abuse as assessed by a full medical interview and examination, routine laboratory tests, a drug screening test, an electrocardiogram and a standardized psychiatric interview (SCID) (First et al., 1995). Eleven subjects were smokers and 10 had never smoked. Written informed consent was obtained from all subjects. The study was conducted in compliance with the declaration of Helsinki and was approved by the ethics committee of the Heinrich-Heine University Dsseldorf. The data for one subject were discarded due to excessive motion artefact in the fMRI, one subject was excluded because of a lack of signicant BOLD response to the task, and four further subjects' data were discarded due to poor EEG quality (inadequate ballistocardiogram artefact correction for three subjects, and a generally noisy signal for one subject due to the inability to reduce impedance at recording electrodes to b10 k), resulting in the data from fteen subjects (8 male, with a mean age of 33.7 years (SD = 14.2), 7 subjects were smokers) being included in the analysis. Behavioural task Subjects performed a hybrid two choice reaction timeoddball task (black and white checkerboard reversal) consisting of 256 frequent stimuli and 64 infrequent stimuli that is, the reversal of the checkerboard from the frequent pattern was the target. The ratio of frequent to infrequent stimuli is similar to that of an oddball task and requires the subject to identify the target stimuli and make a different response to these stimuli. The frequent stimuli will be referred to as frequent targets and the infrequent stimuli as infrequent targets.

1534

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

Stimuli were presented using Presentation version 11.3 (Neurobehavioural systems, Albany, CA, USA) via a screen situated behind the scanner. Subjects were able to view the screen via a mirror mounted on the head coil. Subjects' responses were recorded using Lumitouch key pads (Photon Control Inc, Burnaby, BC, Canada). For infrequent target stimuli subjects responded with their right index nger and for frequent targets they responded with their left index nger; they were asked to respond quickly and accurately to each stimulus and reaction time was recorded for each response. Stimuli were presented with a duration of 1000 ms and a pseudorandomised interstimulus interval of 4000 (2500) ms. fMRI data acquisition Functional MR-images were acquired using a 3 T scanner (Trio, Siemens, Erlangen, Germany). In order to avoid head movements, the head of each subject was secured using sponge pads. Using echo planar imaging (EPI), 630 volumes were obtained applying the following EPI parameters: 33 slices, slice thickness 3 mm, FOV 200 200 mm, 64 64 matrix, repetition time 2000 ms, echo time 30 ms, and ip angle 90. To facilitate localisation and co-registration of functional data, structural scans were acquired using T1-weighted MRI sequences (Magnetization prepared rapid gradient echo (MPRAGE): TR/TE = 2250/3.03 ms, ip angle = 9, 176 sagittal slices, FOV 200 200 mm, 64 64 matrix, and voxel size 1 1 1 mm. fMRI analysis fMRI analysis was performed with FSL (FMRIB's Software Library, www.fmrib.ox.ac.uk/fsl), employing different modules of the FSLsoftware package, non-brain removal using BET (Smith, 2002), spatial smoothing using a Gaussian kernel of FWHM = 8 mm, mean-based intensity normalization of all volumes by the same factor, and highpass temporal ltering (sigma = 125 s). General linear model (GLM) time-series statistical analysis of individual data sets was carried out using FILM (FMRIB's Improved Linear Model) with local autocorrelation correction (Woolrich et al., 2001). Registration of functional images to high resolution structural images was done with FLIRT (Forman et al., 1995; Jenkinson et al., 2002). Explanatory variables were convolved with a gamma haemodynamic response function. A description of the explanatory variables constructed and the different models used is provided below. Group level mixed effect analyses were conducted using FLAME (FMRIB's Local Analysis of Mixed Effects) (Behrens et al., 2003) with spatial normalization to MNI (Montreal Neurological Institute) space and applying a cluster signicance threshold of Z N 2.3 (Forman et al., 1995; Friston et al., 1994; Worsley et al., 1992) with the exception of the models including single-trial parameters as an additional regressor where a cluster signicance threshold of Z N 2.0 was applied. Functional data were imported to MRIcron (Rorden et al., 2007) for visual display purposes. EEG data acquisition EEG data were recorded using a 32-channel MR compatible EEG system (Brain Products, Gilching, Germany). The EEG cap (BrainCap MR, EasyCap GmbH, Breitbrunn, Germany) consisted of 30 scalp electrodes distributed according to the 1020 system and two additional electrodes, one of which was attached to the subjects' back for recording the electrocardiogram (ECG), while the other was attached on the outer canthus of the left eye for detection of ocular artefacts. Data were recorded relative to an FCz reference and a ground electrode was located at Iz (105 electrode system, Oostenveld and Praamstra, 2001). Data were sampled at 5000 Hz, with a bandpass of 0.016250 Hz. Impedance at all recording electrodes was less than 10 k.

EEG data analysis Raw EEG data were processed ofine using BrainVision Analyzer 2 (Brain Products, Gilching, Germany). Gradient artefact correction was performed using modied versions of the algorithms proposed by Allen et al. (2000), where a gradient artefact template is subtracted from the EEG using a baseline corrected sliding average of 20 MRvolumes. Data were then down-sampled to 250 Hz. Following gradient artefact correction, the data were corrected for cardioballistic artefacts. An average artefact subtraction method (Allen et al., 1998) was implemented in Brain Vision Analyzer. This method involves subtracting the artefact on a second by second basis using heartbeat events (R peaks) detected in the previous 10 s. As such it requires accurate detection of R peaks which is aided by the employment of a moving average low pass lter and a nite impulse response high pass lter (for details, see Allen et al., 1998). In this study, the R peaks were detected semi-automatically, with manual adjustment for peaks misidentied by the software. To average the artefact in the EEG channels, the R peaks are transferred from the ECG to the EEG over a selectable time delay. The average artefact was then subtracted from the EEG. Once corrected for gradient and cardioballistic artefacts, the data were segmented into 1000 ms epochs (200 ms to 800 ms) for the purposes of ERP (event-related potential) analysis. Data were then inspected for artefacts resulting from an eye blink or other muscular sources, and any epoch containing a voltage change of more than 150 V was rejected. The remaining epochs were then averaged separately for infrequent and frequent targets (N50 for infrequent targets and N220 for frequent targets for all subjects). Data from Cz were further processed for ST analysis using wavelet denoising. Wavelet denoising was accomplished using EP_den_v2, a freely available tool (http:// www.vis.caltech.edu/~rodri/EP_den/EP_den_home.htm) that runs in MATLAB (The Mathworks Inc, Natick, MA, USA). Following the method of Quian Quiroga and Garcia (2003) the data were decomposed into ve wavelet scales, wavelet coefcients were then chosen to optimise the average signal up to and including the P3. This optimised denoising scheme was then applied to the single-trial data. Single-trial P3 parameters were extracted from the data for each subject by identifying the peak positive amplitude in a window from 280550 ms post stimulus. The single-trial latencies and amplitudes were then screened for outliers using Z scores; any trials with a value more than 3 standard deviations away from the mean were discarded; a mean of 89% (SD = 6.7) of trials were kept for further analysis. Combined fMRI/ERP analysis First we performed a conventional analysis of the data to assess group level activation for the task by using one explanatory variable with a constant impulse function (same height and duration) positioned at stimulus onset (Fig. 1a). It is worth noting at this point that in all models the input functions (the onset time, event duration, and event intensity) are convolved with the gamma haemodynamic response function which blurs and delays the waveform to match the difference between the input function and the measured haemodynamic response. To test the effects of using ERP parameters to modify the basis function we used a number of models. First we included P3 amplitude as a regressor to provide a basis function with variable height (Fig. 1b). This results in the height of the basis function being higher when P3 amplitude is larger. This reects differences in the intensity of responses, as measured by P3 amplitude. We hypothesised that this would show activation in areas sensitive to single-trial variability in P3 amplitude. For P3 latency we used the single-trial latency value in two ways. First, we used single-trial P3 peak amplitude latencies as an additional regressor (to modify the height of the basis function, Fig. 1b). So, the height of the basis function increases as P3 latency increases. The idea here is that a longer response would reect longer processing time and thus a

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

1535

Fig. 1. Summary of how the basis function is modied by including single-trial parameters in the GLM. Part A shows the conventional analysis where a constant impulse (same height and duration) is positioned at stimulus onset. Part B shows a variable impulse model where the height of the basis function is modied by single-trial parameters. The curves to the right of the model show how the HRF can be modied by including the parameters, the height changes but the overall shape remains the same. When P3 amplitude was included in the model this resulted in the height of the basis function being higher when P3 amplitude was larger. This reects differences in the intensity of responses, as reected in P3 amplitude. When P3 latency and reaction time values were used in this model the height of the basis function increased as P3 latency and reaction time increased. The idea here is that a longer response would reect longer processing time and thus a stronger BOLD response. Part C shows a variable epoch model where the duration of the boxcar in the basis function is modied by single-trial parameters. The curves to the right of the model show how the HRF can be modied by including the parameters, both the height and the overall shape change. When P3 latency and reaction time are included in this model the length of the boxcar increases with longer P3 latency and reaction times. So, when we have a longer P3 latency or reaction time we expect the decision making process to be longer in those trials, by using these single-trial parameters to determine the epoch length we aim to capture neural activity in an appropriate time window for that response. For all models the input functions (the onset time, event duration, and event intensity) were convolved with the gamma haemodynamic response function which blurs and delays the waveform to match the difference between the input function and the measured haemodynamic response.

stronger BOLD response, so this approach should model activation sensitive to P3 latency variability. However, Bnar et al. (2007) used single-trial latency values in this way and found an inconsistent pattern of results across subjects and Grinband et al. (2008) found that including reaction time as an additional regressor results in the least powerful models. We therefore expected this model to be the least informative. Second, we created a variable epoch model where the length of the box car in the basis function was determined at the single-trial level on the basis of the P3 peak amplitude latencies (Fig. 1c). So, the length of the boxcar increases with longer P3 latencies. Increasing the boxcar length results in a change in the height and overall shape of the HRF. The mean and standard deviation of the parameters included can be seen in Table 1. When we have a longer P3 latency we expect the decision making process to be longer in those trials; by using these single-trial parameters to determine the epoch length we aim to capture neural activity relating to the duration of the cognitive process of target detection. Previous studies have shown that considering the time taken to make a decision can be more powerful for detecting activation related to decision making (Menon et al., 1998; Grinband et al., 2008). Consequently, we hypothesise that this model will emphasise target detection related activity, while being aware that time resolution of the BOLD-responses may not be sufcient to disentangle different aspects on the basis of this approach. For comparison with the work of Grinband et al. (2008) we also used single-trial mean centred reaction times in the same way. Firstly, by using them to determine the height of the basis function on

a single-trial level to test whether activation related to variability in response time could be found (Fig. 1b). Secondly, we used reaction time to determine the length of the boxcar to look at the whole duration of the response process (Fig. 1c). For all variations of our models, we ran the analysis in two ways: rstly, with a single explanatory variable (containing the single-trial information) to test the overall effect of these parameters. Secondly, a model containing two explanatory variables, the rst being the conventional analysis (stimulus onset time course with no additional single-trial information) and the second explanatory variable containing the various single-trial parameters. The second EV was orthogonalised to the rst to determine any additional activation explained by the inclusion of single-trial parameters. Orthogonalisation was performed using the SchmidtGram procedure as implemented in FSL's FEAT, the formula is as follows: Aorth = A B (A.B) / (B.B).

Table 1 Mean and standard deviation (SD) for the single-trial parameters included in the GLM: P3 amplitude and P3 latency at Cz and reaction time. Reaction time Mean Target Non-target 647 598 SD 117 128 P3 amplitude Mean 14.3 12.5 SD 9.2 6.99 P3 latency Mean 438 451 SD 92 101

Means and standard deviations were calculated from the single-trial values for each parameter.

1536

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

Fig. 2. (A) BOLD activation for the conventional (uninformed) analysis (second-level mixed-effects FLAME. N = 19, Cluster-corrected threshold Z = 2.3, P = 0.05) (B) Grand average ERP at Fz, Cz and Pz (N = 15) for infrequent target vs frequent target stimuli. (C) Stacked plot of single-trial ERPs at Cz for one representative subject.

Our main aim was to detect activation related to identifying the infrequent target stimuli as targets, this was achieved by including only response to infrequent target stimuli in the model. All analyses described above were conducted for responses to infrequent target stimuli. To detect activation that might be related to the general

processing of the visual stimuli presented during the task we also ran the P3 amplitude informed variable impulse model and P3 latency informed variable epoch model with responses to all stimuli included in the model. This also allowed us to determine activation specically associated with responding to infrequent target stimuli.

Table 2 Mean and standard error of the mean (SEM) for the amplitude and latency of the N2 and P3 components of the ERP at Fz, Cz, and Pz for target and non target stimuli. Fz Target N2 amplitude (V) N2 latency (ms) P3 amplitude (V) P3 latency (ms) Mean SEM Mean SEM Mean SEM Mean SEM 4.01 0.57 119 5.10 5.83 6.67 459 22.4 2.96 0.41 118 5.2 4.58 2.59 444 20.74 Cz 3.6 0.51 120 4.7 6.06 5.78 453 23.4 2.7 0.38 123 4.6 4.68 2.08 450 20.98 Pz 2.8 0.45 118 5.09 5.63 2.59 491 25.7 2.3 0.41 116 5.15 4.98 .88 449 20.57

Table 3 Brain regions and local maxima for the standard analysis of responses to target stimuli. Region (Harvard-Oxford, maximum probability) Left postcentral gyrus MNI coordinates of local maxima (x, y, z) 40, 28, 68 40, 26, 60 40, 22, 60 22, 50, 26 6, 62, 20 42, 8, 28 40, 4, 32 34, 22, 2 42, 2, 2 32, 96, 2 26, 100, 4 26, 100, 4 Maximum Z score 5.74 5.67 5.64 5.24 4.76 4.09 3.94 4.03 4.45 4.61 4.54 4.49

Non-target Target Non-target Target Non-target

Cerebelluma Right precentral gyrus Right insula Left insula Left occipital pole

Peak amplitude and latency values were extracted from the average ERP for each subject for each condition, these values were then used to calculate mean and SEM for the group.

MNI label.

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

1537

Region of interest analysis Region of interest (ROI) analysis was performed for the models using single-trial P3 amplitude as an additional regressor and the model using P3 latency to create variable epochs for the infrequent target stimuli. Masks were created for the regions of interest based on the additional group level activation in each orthogonalised model (added value). For each model one mask was created and included all active voxels in the added value analysis (Z = 2.0 for the model including amplitude as a regressor, Z = 2.3 for the variable epoch model). For the P3 amplitude model the mask included the left central opercular cortex, left postcentral gyrus, left insula, left middle frontal gyrus, and left parietal operculum (see Fig. 3a). For the P3 latency variable epoch model the mask included the left postcentral gyrus and left precentral gyrus (Fig. 3c). Statistical analysis Pearson's correlation coefcient r was used to test the relationships between measures using the SPSS (SPSS Inc, Chicago, IL, USA) statistical analysis programme. Results Behavioural data All subjects performed the task with an average of 98.6% (SD 3.3%) correct responses. No false responses were recorded but an average of 1.4% (SD 3.3%) stimuli was missed. Mean reaction time was 647 ms (SD = 117) for infrequent target stimuli and 598 ms (SD = 128) for frequent target stimuli. EEG data The grand average ERP showed the typical morphology in response to visual stimuli during a decision making task at central electrodes with N1 (104 ms poststimulus) and P2 (164 ms poststimulus) to all stimuli and a P3 (412 ms poststimulus) that was more prominent for the infrequent target stimuli than the infrequent target stimuli (Fig. 2b). A summary of the ERP data can also be seen in Supplementary Figure 1. P3 data for responses to infrequent target stimuli at the single-trial level can be seen in the stacked plot in Fig. 2c and summary statistics of the amplitude and latency of the N2 and P3 peaks can be seen in Table 2. fMRI data Conventional fMRI analysis The conventional BOLD analysis revealed activation in response to infrequent target stimuli in the postcentral gyrus, precentral gyrus, cerebellum, supramarginal gyrus, insula, frontal operculum, inferior frontal gyrus, middle frontal gyrus, and occipital cortex (Fig. 2a; see Table 3 for MNI coordinates and mean Z values). The models including responses to all stimuli also yielded activation in these areas (Supplementary Figure 2). Similarly, all single-trial informed models with a single EV revealed a similar pattern of activation (Fig. 3, left column). P3 amplitude informed fMRI analysis For the infrequent target stimuli the model including single-trial P3 amplitude as a regressor orthogonalised to the conventional analysis, in other words the added value of P3 amplitude as a regressor, yielded a peak cluster in the central opercular cortex, including precentral gyrus, temporal pole and planum polare. Additional activation was observed in the insula, and left parietal operculum (Fig. 3, right column, Fig. 4 and Table 2). No additional activation was

gained for the P3 amplitude informed model when responses to all stimuli were included. P3 latency informed fMRI analysis The model using P3 latency as a second regressor did not result in any additional activation (added value) (Fig. 3, left column). However, when using the single-trial P3 latency to create a variable epoch model, additional activation was found for the analysis involving infrequent targets only and that involving all stimuli. For infrequent target stimuli activation was observed in the left postcentral gyrus, left precentral gyrus, thalamus, parietal operculum and supramarginal gyrus (see Fig. 3, left column, Fig. 4, and Table 2). For all stimuli additional activation was found in the right precentral gyrus, right postcentral gyrus, cerebellum, right lateral occipital cortex and anterior cingulate cortex (ACC) (Fig. 4). The activation observed in the ACC for the analysis including all stimuli was not observed for the infrequent target only analysis. The peak clusters of additional activation gained for the P3 latency informed variable epoch models could reect the lateralised motor response required as part of the task; predominantly left precentral gyrus activation for the right handed response to infrequent targets and right precentral gyrus activation for the left handed response associated with the majority of responses for all stimuli. This would be expected since the variable epoch model determined by P3 latency would include the cognitive processes involved in identifying the stimuli and deciding which response to make and would therefore overlap somewhat with the motor response to these stimuli. However, we also gain additional activation for infrequent target stimuli only in the parietal operculum, supramarginal gyrus and Heschl's gyrus which are regions associated with target detection and not motor responses. To illustrate that this additional activation in these areas is specic to processing the infrequent target stimuli we also performed P3 latency informed analysis for only the frequent target stimuli. This clearly illustrates the lateralised motor response associated with each type of stimulus, but also shows that the activation in the parietal operculum and supramarginal gyrus is unique to the infrequent target stimuli (Fig. 5). While this analysis shows that the lateralised motor response is modelled by our P3 informed variable epoch models, it also shows that the ratio of frequent to infrequent stimuli in our hybrid two choice reaction timeoddball task is analogous to an oddball paradigm and responses to infrequent target stimuli should be treated target detection responses. To test whether the additional activation seen in the variable epoch model was a result of choosing a longer epoch for analysis or the variability of P3 latency at a single-trial level, we ran a constant epoch model for the infrequent target stimuli only using the P3 latency from the grand average ERP (412 ms). We hypothesised that the temporal resolution of our fMRI data would not be sensitive enough to the small variations (10 s of ms) on a trial to trial level and that it was our longer window that was responsible for the added value activation rather than single-trial variability. This constant epoch model also resulted in added value in similar regions to the variable epoch model (the left postcentral gyrus and left precentral gyrus) and an additional cluster in the right occipital cortex (Fig. 3, right column). Reaction time informed fMRI analysis Analyses including reaction time were performed for the infrequent target stimuli only. When using reaction time as an additional regressor, no additional activation in relation to the conventional, uninformed analysis was observed (Fig. 3, right column). However, when using single-trial reaction time to create a variable epoch model additional activation was observed in the left precentral gyrus, left postcentral gyrus, and left and right lateral occipital cortices (Fig. 3, right column, and see Table 2 for details).

1538

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

1539

ROI analysis To assess whether the added value in the EEG informed fMRI models (P3 amplitude and P3 latency variable epoch models) was driven by variability in the single-trial ERP parameters, correlation analyses were performed between the standard deviation of the ERP parameters of interest and the mean BOLD response Z values in each region of interest (see Methods section for the added value regions dened based on the group level activation). We predicted that single-trial ERP variability would be driving the added value activation and so anticipated a positive correlation between the standard deviation of the ERP measures and the mean Z values in each region of interest. For the model using P3 amplitude as a regressor the mean Z value in the region of interest showed a trend for a positive relationship with the standard deviation of P3 amplitude (r(14) = 0.48, P = 0.03, one sided) (Fig. 6). For the model using a variable epoch model based on ST P3 latency the standard deviation of P3 latency did not correlate with the mean Z value in the region of interest (r(14) = 0.15, P = 0.29, one sided) (Fig. 6). Correlations between single-trial measures across subjects Pearson's product moment correlation coefcient r was used to test the relationships between the single-trial measures: P3 amplitude, P3 latency and reaction time. Given the equivocal ndings on the relationship between reaction time and electrophysiological measures generally reported in literature (Musso et al., 2006), we made no prediction about the direction of the relationship. No signicant relationship was found between reaction time and P3 amplitude (r(14) = 0.003, P = 0.92, two sided) or reaction time and P3 latency (r(14) = 0.41, P = 0.13, two sided). Discussion The conventional GLM analysis revealed activation in response to infrequent target stimuli in brain regions consistent with previous fMRI, intracortical electrophysiological recording and EEG source localisation studies. The postcentral gyrus, precentral gyrus, cerebellum, supramarginal gyrus, insula, frontal operculum, inferior frontal gyrus, middle frontal gyrus, and occipital cortex have all been reported in response to tasks involving the detection of infrequent target stimuli (Halgren et al., 1995a,b; Gur et al., 2007; Strobel et al., 2008; Winterer et al., 2007). The activation pattern was similar for all single explanatory variable (EV) models (Fig. 1), with peak clusters being observed in the same regions with similar Z values. However, it is the so called added value models that provide the most information. By orthogonalising the single-trial informed explanatory variable to the conventional uninformed explanatory variable only additional activation explained by including the single-trial parameters is observed, improving the specicity of the model. Added value of single-trial parameter informed models P3 amplitude For infrequent target stimuli including P3 amplitude as a regressor in the GLM yielded additional activation in the central opercular cortex, precentral gyrus, temporal pole, frontal operculum and insula whereby activation seen in these areas is consistent with the detection of infrequent target stimuli in fMRI studies (Bledowski et al., 2004; Gur et al., 2007). The activation pattern is obtained

without observing the occipital activation that is seen in the conventional analysis. This would suggest that the single-trial P3 amplitude informed analysis is more effective for localising activation in areas related to the detection of infrequent targets. Furthermore, the added value in the P3 amplitude informed model (mean Z score in ROI) shows a signicant positive relationship with the standard deviation of the single-trial P3 amplitudes. This suggests that the additional fMRI activation is driven by the variability in the singletrial P3 amplitudes which supports the idea that using single-trial ERP amplitude is a valid way to harness the variability in responses to individual stimuli in order to maximise our analysis of fMRI data. However, while ST P3 amplitude informed analysis yields additional activation in this left fronto-central region, the activation in the more parietal regions also associated with target detection and frequently seen in fMRI studies of target detection (Gur et al., 2007; Winterer et al., 2007) is lost in the added value analysis. The most plausible explanation is that the added value activation reects areas that are more closely associated with variability in the single-trial P3 amplitudes. The resulting activation map is very similar to the ventral attention system activation map obtained by Mantini et al. (2009) when differentiating the ventral attention system from the dorsal attention system. They suggest that the ventral attention system is associated with the detection of behaviourally relevant stimuli, so an emphasis on this region in P3 amplitude informed analysis is plausible. When all trials (frequent and infrequent) are included in the analysis there is no added value for the amplitude informed model. Given that the amplitude informed model (variable impulse) yields BOLD activation that is related to the variability in the ERP amplitude (Grinband et al., 2008) we can assume that the absence of added value reects one of two things: 1. there is less variability in the frequent non targets compared to the infrequent targets, 2. the variability in responses to the frequent stimuli is not related to a specic pattern of BOLD activation. In support of the rst explanation, the variability in the single-trial P3 amplitudes (as measured by standard deviation) is marginally higher for the targets compared to non target (T(14) = 2.01, P = 0.06). In support of the second explanation, the regions in which we get additional activation in the ERP amplitude model for the frequent stimuli only (central opercular cortex, precentral gyrus, temporal pole, frontal operculum and insula) are consistent with the detection of behaviourally relevant stimuli (e.g. Mantini et al., 2009). We therefore conclude that the pattern of additional activation yielded by the inclusion of single-trial P3 amplitude is related to the additional attentional requirements of detecting the infrequent target stimuli and the variability in responses to these stimuli across trials. An alternative explanation of our added value activation could be related to the nature of our task as a two choice response task. As our task is not a true oddball paradigm, in that a response is also given to frequent stimuli, our results could represent the switch cost between the frequent and infrequent stimuli and corresponding responses. Task switching involves shifting attention and retrieving stimulus response rules (Goffaux et al., 2006) and this process could explain the attention related regions activated when including single-trial P3 amplitude as a regressor. However, imaging studies of task switching have identied the lateral prefrontal cortex and superior frontal gyrus (Cutini et al., 2008; Dove et al., 2000) as the key structures involved in switch cost, this does not match our area of activation. This suggests that we are modelling activity related to a different process. This is

Fig. 3. BOLD activation for all analyses (second-level mixed-effects FLAME. N = 19, Cluster-corrected threshold Z = 2.3, P = 0.05 for the single EV models and Z = 2.0, P = 0.05 for the added value models). The left column shows the single explanatory variable analysis (conventional and single-trial informed). The right column shows the results of the model including two explanatory variables, the rst being the conventional analysis (stimulus onset timecourse with no additional single-trial information) and the second explanatory variable containing the various single-trial parameters, the second EV was orthogonalised to the rst to determine any additional activation explained by including single-trial parameters. Part A shows the conventional model where a constant impulse model was used. Parts B, D and F show variable impulse models where the single-trial parameter was used as an additional regressor to modify the height of the basis function. Parts C and E use a variable epoch model based on single-trial P300 latency and reaction time respectively. Part G uses a constant epoch model where the length of the boxcar is determined by the latency of P3 taken from the grand average ERP. See Fig. 1 for more details on the construction of each type of model.

1540

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

Fig. 4. Coronar slices showing the peak activation for the added value models (second-level mixed-effects FLAME. N = 19, Cluster-corrected threshold Z = 2.0, P = 0.05). Part A shows the additional activation found when using P300 amplitude as an additional regressor for infrequent target stimuli. The activation is in the central opercular cortex, precentral gyrus and insula (see Table 4 for details). Part B shows the additional activation when using a variable epoch model informed by single-trial P300 latency for infrequent target stimuli. The activation is posterior to that of the variable impulse model and involves the parietal operculum and supramarginal gyrus (see Table 4 for details). Part C shows the additional activation for the variable epoch model informed by single-trial P3 latency for all stimuli. This highlights the activation obtained in the anterior cingulate cortex. Part D shows the additional activation when using a variable epoch model informed by single-trial reaction time for infrequent target stimuli. The activation is predominantly in the pre and postcentral gyri (see Table 4 for details).

T. Warbrick et al. / NeuroImage 47 (2009) 15321544 Table 4 Brain regions and local maxima for single-trial informed analysis orthogonalised to the standard analysis (added value). Model Region (Harvard-Oxford, maximum probability) MNI coordinates of Maximum local maxima (x, y, z) Z score 52, 4, 2 32, 8, 0 28, 18, 10 16, 2, 20 48, 36, 60 36, 30, 70 50, 28, 60 46, 28, 64 38, 22, 36 36, 28, 64 8, 10, 2 10, 0, 10 54, 24, 16 62, 24,30 44, 24, 12 54, 20, 54 48, 16, 58 28, 10, 70 22, 68, 60 8, 10, 36 8, 8, 38 2, 4, 42 44, 78, 14 46, 26, 54 34, 16, 50 28, 72, 38 20, 70, 46 30, 64, 40 34, 66, 32 4.02 3.75 3.63 3.55 4.21 4.57 3.92 3.84 4.34 4.25 3.28 3.27 2.83 2.60 2.10 3.69 3.62 3.4 3.48 2.4 2.8 2.5 3.07 3.86 3.40 3.24 3.23 3.75 3.42

1541

P300 amplitude Left central (infrequent stimuli) opercular cortex Left insula Right insula Left caudate Left postcentral gyrus P300 variable epoch Postcentral gyrus (infrequent stimuli) Postcentral gyrus, precentral gyrus Left thalamus Right thalamus Parietal operculum Supramarginal gyrus Heschl's gyrus Right postcentral gyrus Right precentral gyrus Right lateral occipital cortex Anterior cingulate cortex

P300 variable epoch (all stimuli)

Left postcentral gyrus Reaction time Left precentral gyrus variable epoch (infrequent stimuli) Left lateral occipital cortex Right lateral occipital cortex

supported by electrophysiological data; for example in uncued task switching P3 amplitude and latency do not differ between repeat and switch trials (Hsieh and Liu, 2005; Hsieh and Yu, 2003) suggesting that the switch cost is after the stimulus identication stage. Differences in P3 are observed in cued switch tasks (Hsieh and Liu, 2005), however, there is no cue in our task so it is likely that our marginal difference in P3 parameters between frequent and infrequent stimuli (Supplementary Table 1) is related to target detection processes involved in our paradigm rather than a result of switch cost. In support of this Dove et al. (2000) reported that a low ratio of switch trials to repeat trials can make the task more like an oddball task. Given that the stimulus ratio in our paradigm is designed for an oddball task we believe that we are indeed looking at target detection networks rather than switch cost. P3 latency and reaction time The model using single-trial P3 latency as a regressor to modify only the height of the basis function, did not result in any additional activation (added value). Our ndings are consistent with those of Grinband et al. (2008) who found that using timing information (in their case reaction time) as a regressor in the model was the least powerful model. However, when using single-trial P3 latency to create a variable epoch model, modifying the duration/overall shape in addition to the height of the basis function, we found additional activation in the left postcentral gyrus, left precentral gyrus, thalamus, parietal operculum, supramarginal gyrus and Heschl's gyrus for infrequent targets only and in the right postcentral gyrus, right precentral gyrus and ACC when all stimuli are included in the analysis. While this additional activation does include some regions that are associated with the motor response to the stimuli, there is additional activation in regions associated with the detection of stimuli and making decisions about those stimuli. This would suggest that by using an appropriate epoch length and orthogonalising the explanatory variable to the conventional timecourse we can determine

activation in brain regions specic to the cognitive processes involved in the task. Furthermore, the non-motor activation detected differs when only infrequent target stimuli and when all stimuli are included in the model. Additional activation was only obtained in the ACC when all stimuli were included in the analysis. The ACC is associated with executive, evaluative and cognitive functions and its involvement in our task is consistent with other fMRI studies of target detection and decision making tasks (Gur et al., 2007; Philiastides and Sajda, 2007). Since the ACC has an important role in processing top-down and bottom-up stimuli and assigning appropriate control to other areas in the brain, and is associated with effort applied to a task (Mulert et al., 2008; Esposito et al., 2009), its involvement in responses to all stimuli in a decision making task such as ours is expected. The fact that we did not obtain additional ACC activation for infrequent targets only is likely to be due to there being too few trials in this condition to reveal this activation, exploiting the P3 information available in all trials yields this additional activation. The parietal regions identied using the P3 latency informed variable epoch model for infrequent target stimuli only are consistent with fMRI target detection studies (Gur et al., 2007; Strobel et al., 2008; Winterer et al., 2007) and may reect working memory processes. This is also consistent with what is known about the ventral and dorsal streams of information processing; the ventral stream is involved in the detection of salient events while the dorsal stream is involved in maintaining visuospatial attention once the salient stimulus has been identied (Corbetta et al., 2002; Corbetta and Shulman, 2002). The dorsal stream projects to the parietal region, in line with our added value activation, and mediates the response to the salient stimulus (Goodale and Milner, 1992). In other words the topdown inuence of the ventral system in voluntarily directing attention to the salient stimulus leads to the maintenance of attention to that stimulus and the updating of working memory (Sridharan et al., 2007). So, in our paradigm attention is directed to the salient infrequent target stimuli and thus an increase in BOLD activation is seen in these brain areas in response to infrequent target stimuli. From the pattern of activation we observed in the added value analysis, it would appear that we can model these working memory processes by using a variable epoch model informed by single-trial P3 latency. Similar results were found when using single-trial reaction time in the GLM; no additional activation was seen when single-trial reaction time was used to create a variable impulse model but additional activation in parietal and occipital regions were observed when using single-trial reaction time to create a variable epoch model. Using single-trial parameters in the GLM Responses to repeated stimuli do vary, not just from subject to subject but on a trial to trial basis. By accounting for this variability in the GLM by using single-trial parameters we can conduct a more appropriate analysis of our data and gain a better understanding of the relationship between event-related BOLD and EEG (Debener et al., 2006; Fell, 2007; Makeig et al., 2005). The questions we aimed to answer in this paper were what aspects of the ERP and behavioural response to a stimulus are most informative when included in the GLM, and what is the most appropriate way to use this single-trial information in the GLM, specically for a choice reaction time/target detection (P3) paradigm. The summary above indicates that including different single-trial parameters in the GLM does provide varied results, what we now need to do is determine which parameter should be used and in what way this information should be included in the GLM. Our rst approach was to include the ST information as a regressor in the GLM to modify the height of the basis function, this serves to modify the height of the basis function at a trial by trial level (see Fig. 1 for how this was achieved in this paper, and Grinband et al. (2008) for a more detailed general discussion). This is analogous to modifying the intensity of each event. This proved to be effective for P3

1542

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

Fig. 5. Coronar slices showing the peak activation for the added value models (second-level mixed-effects FLAME. N = 19, Cluster-corrected threshold Z = 2.0, P = 0.05) Part A shows the additional activation when using a variable epoch model informed by single-trial P300 latency for non target stimuli. Part B shows the additional activation when using a variable epoch model informed by single-trial P300 latency for target stimuli. Part C shows the binarised activation maps for additional activation for both the target (blue) and non target (red) stimuli. The binarised maps show the location of the activation only and give no indication of varying amplitude of activation. This illustrates that while the lateralised motor response is modelled by the P3 latency informed variable epoch model, the target stimuli elicit additional activation in the parietal operculum compared to the non target stimuli.

amplitude but not for P3 latency or reaction time. This suggests that information gained from timing measures are not as informative as amplitude measures when used in this way. This makes sense given that reaction time and P3 latency vary in time, and while variable impulse models changes the height of the basis function the duration of the function remains the same. It is a logical conclusion then, that a variable impulse model is not the most appropriate way to use singletrial latency measures. However, this method does appear to be appropriate for measures of event intensity, in our case P3 amplitude. The amplitude of P3 varies with the intensity/arousal associated with a response and so we would expect that the most appropriate way to model this variability would be to use a variable impulse model. Our results show that variability in responses on a trial by trial basis is reected in the amplitude of the P3 and that the BOLD response associated with this variability can be determined by including these parameters in a variable impulse model. This is in line with previous studies using ERP amplitude as a regressor in the GLM (Debener et al., 2005; Eichele et al., 2005; Mulert et al., 2008). Furthermore, distinct activation patterns were observed for infrequent targets and frequent targets illustrating that this method can differentiate responses to stimuli with differing cognitive demands.

The second approach was to use timing information gained from the ERP or behavioural response to modify the overall shape and height of the basis function in the GLM. The haemodynamic response to a decision making process has been shown to vary with the time taken for the subject to make a response (Menon et al., 1998). This is a critical point for paradigms such as choice reaction time tasks and oddball tasks where subjects are involved in a decision making process and provide a response. This decision making process is likely to persist after stimulus onset, so a zero duration impulse function would not be the most appropriate way to model this process. We would also argue that while reaction time provides some indication of how long this decision making process takes, it also includes the processing involved in producing the voluntary motor response. For this reason we suggest that P3 latency is a more accurate measure of the duration of the target detection/decision making process. Using P3 latency to create a variable epoch model resulted in additional activation in parietal regions associated with target detection. This would suggest that modelling the duration of the decision making process results in a more specic localisation of brain activation related to target detection. However, the added value activation (Z values in ROIs) and the standard deviation of P3 latency

T. Warbrick et al. / NeuroImage 47 (2009) 15321544

1543

Fig. 6. Scatter plots showing the relationship between mean Z values in added value regions of interest (ROI) and single-trial parameter standard deviation for both P3 amplitude and P3 latency.

did not correlate which would suggest that this nding is not driven by a single-trial variability. A plausible alternative is that these methods just capture a more appropriate window for analysis than using an impulse function. To test this we conducted some additional analysis using a constant epoch model using P3 latency from the grand average ERP to determine the epoch duration for each subject. This model resulted in a similar activation pattern suggesting that it is a more appropriate time window that is important rather than latency variability on a single-trial level being the sole driving force behind this nding. The temporal resolution of fMRI BOLD, particularly for whole brain analysis, might not be sensitive enough to modulation by single-trial uctuations in decision making time that will be in the order of 10s of milliseconds. For example, some of the studies that show a detectable BOLD response with stimulus durations and onset asynchronies as low as 50 ms, use a small number of slices covering the area of interest with short TR (Connolly et al., 2005; Menon et al., 1998). On the other hand, some studies using whole brain analysis have also detected short latency differences in the BOLD response (Henson et al., 2002; Formisano et al., 2002). Further investigation of the sensitivity of the BOLD response to small temporal variations on the single-trial level is necessary to determine whether variable epoch models do indeed model single-trial variability or whether the gain is due to an event duration that is more representative of cognitive processing. Using reaction time to create a variable epoch model also yielded added value activity in left parietal regions, however the cluster was smaller than for the model informed by P3 latency. The reduced additional activation for the reaction time variable epoch model compared to the P3 latency variable epoch model could be related to the fact that reaction time itself covers a longer window than the P3. This would include some processes irrelevant to the decision making process resulting in more activation in general, perhaps obscuring the potential added value of this parameter in detecting activity related to decision making. However, we have shown that using reaction time to

create a variable epoch model does localise some activation in target detection areas. This builds on the ndings of Grinband et al. (2008) who showed that using reaction time simulations and varying length visual stimuli as a proxy for varying reaction time could result in a more powerful model. We found that more voxels were activated in the informed models compared to the conventional model and similar Z scores were also observed (with the exception of the P3 amplitude model which had the smallest maximum Z value). We also found added value in these models, showing that this nding holds for real reaction time data. While the variable epoch model informed by reaction time showed less additional activation than the P3 latency informed model, it can still be considered more powerful than the uninformed model, this illustrates that including basic single-trial information is useful even in the absence of single-trial EEG. However, as described above it is still unclear whether single-trial variability is modelled by the variable epoch models or whether the more appropriate HRF shape determined by an event duration extending beyond stimulus onset leads to the additional activation observed. Grinband et al. (2008) observed that while many studies measure reaction time very few actually use this information when analysing their fMRI data. Based on our ndings we would concur with Grinband et al. in that a measure reecting the duration of cognitive processes involved in a task should be considered in order to optimise the analysis of fMRI data. We found different additional activations when including responses to all stimuli or only responses to infrequent target stimuli in the models. For the P3 latency informed variable epoch model the ACC activation was only apparent when all trials were included and the left parietal operculum/supramarginal gyrus activation was only observed for infrequent target stimuli. We only obtained additional activation for the P3 amplitude informed model for infrequent target stimuli. These differences in activation have been explained by the demands of the task and the concordance of our activation patterns with previous research. These distinct patterns of activation show that ERP parameters can be used to differentiate regions involved in different aspects of cognitive processing however, this is dependent on the information that is included in the model. This highlights the fact that while ERP informed fMRI analysis is potentially useful great care needs to be taken in order to ensure that the single-trial information entered into the model reects the cognitive processes of interest. In conclusion it would appear that both single-trial P3 amplitude and P3 latency are informative when included in the GLM when analysing fMRI data. Single-trial amplitude should be used to modify the height of the basis function whereas P3 latency is more informative when used to create a variable epoch model. Given that P3 amplitude and latency yield different patterns of additional task related BOLD activation we would advocate using both of these ST parameters when analysing P3 data. While it is still unclear whether the P3 latency informed variable epoch model accounts for single-trial latency uctuations or reects a more appropriate time window for analysis, it would still appear that this model is more powerful for detecting task related activity than the conventional constant impulse mode. By including parameters representing aspects of cognitive processing BOLD activation in brain areas involved in the selective attention and working memory aspects of target detection can be investigated more specically. Furthermore, given that P3 is a common measure when investigating disorders such as schizophrenia and Alzheimer's, effects of substances such as nicotine and processes such as ageing, understanding how single-trial P3 parameters can be used to inform fMRI analyses in these populations is potentially useful. Of particular relevance to our P3 latency related ndings is that the haemodynamic response can be delayed in specic populations, such as in schizophrenics (Ford et al., 2005). Using indices of cognitive processing to help model these difference in the haemodynamic response could make single-trial informed fMRI analyses a particularly useful tool in this context.

1544

T. Warbrick et al. / NeuroImage 47 (2009) 15321544 Hagenbeek, R.E., Rombouts, S.A., Veltman, D.J., Van Strien, J.W., Witter, M.P., Scheltens, P., Barkhof, F., 2007. Nonlinear changes in brain activity during continuous word repetition: an event-related multiparametric functional MR imaging study. Am. J. Neuroradiol. 28 (9), 17151721. Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Ligeois, C., Chauvel, P., Musolino, A., 1995a. Intracerebral potentials to rare target and distractor auditory and visual stimuli. I. Superior temporal plane and parietal lobe. Electroencephalogr. Clin. Neurophysiol. 94, 191220. Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., Devaux, B., Vignal, J-P., Biraben, A., 1995b. Intracerebral potentials to rare target and distractor auditory and visual stimuli. II. Medial, lateral and posterior temporal lobe. Electroencephalogr. Clin. Neurophysiol. 94, 229250. Halgren, E., Marinkovic, K., Chauvel, K., 1998. Generators of the late cognitive potentials in auditory and visual oddball tasks. Electroencephalogr. Clin. Neurophysiol. 106, 156164. Henson, R.N.A., Price, C.J., Rugg, M.D., Turner, R., Friston, K.J., 2002. Detecting latency differences in event-related BOLD responses: application to words versus nonwords and initial versus repeated face presentations. NeuroImage 15, 8397. Horovitz, S.G., Skudlarski, P., Gore, J.C., 2002. Correlations and dissociations between BOLD signal and P3 amplitude in an auditory oddball task: a parametric approach to combining fMRI and ERP. Magn. Reson. Imaging 20, 319325. Hsieh, S., Liu, L.-C., 2005. The nature of switch cost: task set conguration or carry-over effect. Cogn. Brain Res. 22, 165175. Hsieh, S., Yu, Y.-T., 2003. Exploring the nature of switch cost: inferences from the P300 and the lateralised readiness potentials. Brain Res. Protocols 12, 4959. Javitt, D.C., Spencer, K.M., Thaker, G.K., Winterer, G., Hajs, M., 2008. Neurophysiological biomarkers for drug development in schizophrenia. Nat. Rev. Drug Discov. 7 (1), 6883. Jenkinson, M., Bannister, P., Brady, J., Smith, S., 2002. Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17, 825841. Kiehl, K.A., Stevens, M.C., Laurens, K.R., Pearlson, G., Calhoun, V.D., Liddle, P.F., 2005. An adaptive reexive processing model of neurocognitive function: supporting evidence from a large scale (n = 100) fMRI study of an auditory oddball task. NeuroImage 25, 899915. Makeig, S., Debener, S., Onton, J., Delorme, A., 2005. Mining event-related brain dynamics. Trends Cogn. Sci. 8, 204210. Menon, R.S., Luknowsky, D.C., Gati, J.S., 1998. Mental chronometry using latencyresolved functional MRI. Proc. Natl. Acad Sci. U. S. A. 95, 1090210907. Mantini, D., Corbetta, M., Perucci, M.G., Romani, G.L., Del Gratta, C., 2009. Large-scale brain networks account for sustained and transient activity during target detection. NeuroImage 44 (1), 265274. Mobascher, A., Brinkmeyer, J., Warbrick, T., Musso, F., Wittsack, H.J., Stoermer, R., Saleh, A., Schnitzler, A., Winterer, G., 2009a. Fluctuations in electrodermal activity reveal variations in single trial brain responses to painful laser stimulation a fMRI/EEG study. NeuroImage 44 (3), 10811092. Mobasher, A., Brinkmeyer, J., Warbrick, T., Musso, F., Wittsack, H.J., Stoermer, R., Saleh, A., Schnitzler, A., Winterer, G., 2009b. Laser-evoked potential P2 single-trial amplitudes covary with the fMRI BOLD response in the medial pain system and interconnected subcortical structures. NeuroImage 45 (3), 917926. Mulert, C., Seifert, C., Leicht, G., Kirsch, V., Ertl, M., Karch, S., Moosman, M., Lutz, J., Mller, H.-J., Hegerl, U., Pogarell, O., Jger, L., 2008. Single-trial coupling of EEG and fMRI reveals the involvement of early anterior cingulate cortex activation in effortful decision making. NeuroImage 42, 158168. Musso, F, Konrad, A, Vucurevic, G, Schffner, C, Friedrich, B, Frech, P, Stoeter, P, Winterer, G., 2006. Distributed BOLD-response in association cortex vector state space predicts reaction time during selective attention. NeuroImage 29 (4), 13111318. Oostenveld, R., Praamstra, P., 2001. The ve percent electrode system for highresolution EEG and ERP measurement. Clin. Neurophysiol. 112, 713719. Philiastides, M.G., Sajda, P., 2007. EEG-Informed fMRI Reveals Spatiotemporal Characteristics of Perceptual Decision Making. J. Neurosci. 27 (48), 1308213091. Picton, T.W., 1992. The P300 wave of the human event related potential. J. Clin. Neurophysiol. 9, 456479. Quian Quiroga, R., Garcia, H., 2003. Single-trial event-related potentials with wavelet denoising. Clin. Neurophysiol. 114, 376390. Rolls, E.T., Loh, M., Deco, G., Winterer, G., 2008. Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nat. Rev., Neurosci. 9, 696709. Rorden, C., Karnath, H.O., Bonilha, L., 2007. Improving lesion-symptom mapping. J. Cogn. Neurosci. 19, 10811088. Smith, S., 2002. Fast robust automated brain extraction. Hum. Brain Mapp. 17, 143155. Sridharan, D., Levitin, D.J., Chafe, C.H., Berger, J., Menon, V., 2007. Neural dynamics of event segmentation in music: converging evidence for dissociable ventral and dorsal networks. Neuron 55, 521532. Strobel, A., Debener, S., Sorger, B., Peters, J.C., Kranczioch, C., Hoechstetter, K., Engel, A.K., Brocke, Burkhard, Goebel, R., 2008. Novelty and target processing during an auditory novelty oddball: a simultaneous event-related potential and functional magnetic resonance imaging study. NeuroImage 40, 869883. Winterer, G., Musso, F., Konrad, A., Vucurevic, G., Stoeter, P., Sander, T., Gallinat, J., 2007. 2007. Association of attentional network function with exon 5 variations of the CHRNA4 gene. Hum. Molec. Genet. 16 (18), 21652174. Woolrich, M.W., Ripley, B.D., Brady, M., Smith, S.M., 2001. Temporal autocorrelation in univariate linear modelling of fMRI data. Worsley, K.J., Evans, A.C., Marrett, S., Neelin, P., 1992. A three-dimensional statistical analysis for CBF activation studies in human brain. J. Cereb. Blood Flow Metab. 12, 900918.

Acknowledgment This study was conducted within the framework of the Priority Program SPP1226 of the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft): Nicotine: Molecular and Physiologial Effects in CNS [http://www.nicotine-research.com], grant number: Wi1316/7-1. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.neuroimage.2009.05.082. References
Allen, P., Josephs, O., Turner, R., 2000. A method for removing imaging artifact from continuous EEG recorded during functional MRI. NeuroImage 1, 230239. Allen, P., Polizz, G., Krakow, K., Fish, D.R., Lemieux, L., 1998. Identication of EEG events in the MR scanner: the problem of pulse artifact and a method for its subtraction. NeuroImage 8, 229239. Bagshaw, A.P., Warbrick, T., 2007. Single trial variability of EEG and fMRI responses to visual stimuli. NeuroImage 38, 280292. Behrens, T., Woolrich, M.W., Smith, S., 2003. Multi-Testing Using a Fully Subject Null Hypothesis Bayesian Framework: Theory. Human Brain Mapping Meeting, New York City. Bnar, C-G., Schn, D., Grimault, S., Nazarian, B., Burle, B., Roth, M., Badier, J-M., Marquis, P., Liegeois-Chauvel, C., Anton, J-C., 2007. Single-trial analysis of oddball eventrelated potentials in simultaneous EEG-fMRI. Hum. Bain Mapp. 28, 602613. Bledowski, C., Prvulovic, D., Hoechstetter, K., Scherg, M., Wibral, M., Goebel, R., Linden, D.E.J., 2004. Localizing P300 generators in visual target and distractor processing: a combined event-related potential and functional magnetic resonance imaging study. J. Neurosci. 24 (42), 93539360. Connolly, J.D., Goodal, M.A., Goltz, H.C., Munoz, D.P., 2005. fMRI activation in the human frontal eye eld is correlated with saccadic reaction time. J. Neurophysiol. 94, 605611. Corbetta, M., Kincade, J.M., Shulman, G.L., 2002. Neural systems for visual orienting and their relationships to spatial working memory. J. Cogn. Neurosci. 14 (3), 508523. Corbetta, M., Shulman, G.L., 2002. Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3, 201215. Cutini, S., Scatturin, P., Menon, E., Bisiacchi, P.S., Gamberini, L., Zorzi, M., Dell'Acqua, R., 2008. Selective activation of the superior frontal gyrus in task-switching: an event related fNIRS study. NeuroImage 42, 945955. Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D.Y., Engels, A.K., 2005. Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identies the dynamics of performance monitoring. J. Neurosci. 25, 1173011737. Debener, S., Ullsperger, M., Siegel, M., Engel, A.K., 2006. Single-trial EEG-fMRI reveals the dynamics of cognitive function. Trends Cogn. Sci. 10, 463558. Dove, A., Pollman, S., Schubert, T., Wiggins, C.J., von Crammon, D.Y., 2000. Prefrontal cortex activation in task switching: an event related fMRI study. Cogn. Brain Res. 9, 103109. Eichele, T., Specht, K., Moosman, M., Jongsma, M.L., Quiroga, R.Q., Nordby, H., Hugdahl, K., 2005. Assessing the spatiotemporal evolution of neuronal activation with single trial event-related potentials and functional MRI. Proc. Natl. Acad. Sci. 102, 1779817803. Esposito, F., Mulert, C., Goebel, R., 2009. Combined distributed source and singletrial EEG-fMRI modeling: application to effortful decision making processes. NeuroImage, doi:10.1016/j.neuroimage.2009.03.074. Fell, J., 2007. Cognitive neurophysiology: beyond averaging. NeuroImage 37, 10691072. First, M.B., Spitzer, R.L., Giddon, M., Williams, J.B., 1995. The Structured Clinical Interview for DCM-IV Axis I Disorders Research Version (SCID-1). American Psychiatric Press, New York. Ford, J.M., Johnson, M.B., Whiteld, S.L., Faustman, W.O., Mathalon, D.H., 2005. Delayed hemodynamic response in schizophrenia. NeuroImage 26, 922931. Forman, S.D., Cohen, J.D., Fitzgerald, M., Eddy, W.F., Mintun, M.A., Noll, D.C., 1995. Improved assessment of signicant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 33, 636647. Formisano, E., Linden, D.E., Di Salle, F., Trojano, L., Esposito, F., Sack, A.T., Grossi, D., Zanella, F.E., Goebel, R., 2002. Tracking the mind's image in the brain I: timeresolved fMRI during visuospatial mental imagery. Neuron 35, 185194. Friston, K.J., Worsley, K.J., Frackowiak, R.S.J., Mazziotta, J.C., Evans, A.C., 1994. Assessing the signicance of focal activations using their spatial extent. Hum. Brain Mapp. 1, 214220. Goffaux, P., Phillips, N.A., Sinai, M., Pushkar, D., 2006. Behavioural and electrophysiological measures of task switching during single and mixed-task conditions. Biol. Psychol. 72, 278290. Goodale, M.A., Milner, A.D., 1992. Separate visual pathways for perception and action. Trends Neurosci. 15 (1), 2025. Gur, R.C., Turetsky, B.I., Loughead, J., Waxman, J., Snyder, W., Ragland, J.D., Elliott, M.A., Bilker, W.B., Arnold, S.E., Gur, R.E., 2007. Hemodynamic responses in neural circuitries for detection of visual target and novelty: an event-related fMRI study. Hum. Brain Mapp. 28, 263274. Grinband, J., Wager, T.D., Lindquist, M., Ferrera, V.P., Hirsch, J., 2008. Detection of timevarying signals in event-related fMRI designs. NeuroImage 43 (3), 509520.