Single-Trial Analysis and Its Applications in Eeg and Simultaneous Eeg-Fmri Studies

Material Information

Single-Trial Analysis and Its Applications in Eeg and Simultaneous Eeg-Fmri Studies
Liu, Yuelu
Place of Publication:
[Gainesville, Fla.]
University of Florida
Publication Date:
Physical Description:
1 online resource (111 p.)

Thesis/Dissertation Information

Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Biomedical Engineering
Committee Chair:
Ding, Mingzhou
Committee Members:
Van Oostrom, Johannes H
Wheeler, Bruce
Keil, Andreas
Graduation Date:


Subjects / Keywords:
Amygdala ( jstor )
Anticipation ( jstor )
Electroencephalography ( jstor )
Emotional expression ( jstor )
Experimentation ( jstor )
Hemispheres ( jstor )
Magnetic resonance imaging ( jstor )
Prefrontal cortex ( jstor )
Signals ( jstor )
Visual cortex ( jstor )
Biomedical Engineering -- Dissertations, Academic -- UF
attention -- conditioning -- eeg -- emotion -- fmri
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Biomedical Engineering thesis, Ph.D.


Electrophysiological recordings such as human electroencephalogram (EEG) and monkey local field potentials (LFP) are traditionally thought to be comprised of a deterministic event-related potential (ERP) plus a task-irrelevant random noise. However, evidence has long suggested that the ERP waveform can vary from trial to trial. Such trial-to-trial variability often bears important information about the dynamics of the underlying cognitive processes. With recent advancements in single-trial analysis methods, we seek to explore in this dissertation the dynamics of cognitive processes underlying higher-level cognitive functions including visual attention, emotional processing, and classical conditioning. First, we applied a recently developed single-trial analysis method, Analysis of Single-trial ERP and Ongoing signals (ASEO), to study the trial-to-trial temporal dynamics of sensory facilitation during classical aversive conditioning. Estimating the amplitude of the P1 component elicited by the conditioned stimuli (CSs), we found that P1 amplitude fluctuations within the acquisition session followed three distinct phases. Further, the effects of sensory facilitation toward the CS predicting aversive outcomes, compared to the CS predicting neutral outcomes, were manifested by differences in the rate of P1 amplitude changes within each phase. Second, we investigated brain structures involved in the generation and modulation of the late positive potential (LPP) during emotional processing. Correlating single-trial LPP amplitude with the simultaneously recorded blood-oxygen-level-dependent (BOLD) activity, we found that regions contributing to scalp-recorded LPP included the visual cortices and emotional processing structures. Moreover, the degree of contribution to scalp LPP from these structures was valence specific. Finally, we investigated brain areas contributing to the attentional modulation of alpha (8–12 Hz) in spatial visual attention. Our results suggest that the intraparietal sulcus, a core region within the frontoparietal attention network, modulates the trial-to-trial alpha desynchronization. Further, we found a positive correlation between alpha lateralization and BOLD in dorsal anterior cingulate cortex (dACC), suggesting a role for dACC in facilitating the attentional set via executive influences over attentional control systems. In conclusion, the results suggest that trial-to-trial variability is an important aspect of the cognitive process. Exploring such variability can gain us valuable insights into the neuronal mechanisms of higher-level cognitive processes. ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis (Ph.D.)--University of Florida, 2013.
Adviser: Ding, Mingzhou.
Statement of Responsibility:
by Yuelu Liu.

Record Information

Source Institution:
Rights Management:
Copyright Liu, Yuelu. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
LD1780 2013 ( lcc )


This item has the following downloads:

Full Text




2 2013 Yuelu Liu


3 To my Mom and Dad, and my wife, Bing For their love and support


4 ACKNOWLEDGMENTS First and foremost, I would like to express my d eep est appreciation to my advisor, Dr. Mingzhou Ding, for his continuous support and patience in guiding me through the toughness of my research. His academic excellence wisdom, and sharpness set as an invaluable exemplar for me to emulate. I owe him a tr emendous amount of gratitude for introducing me into the exciting realm of cognitive neuroscience research and providing me with the opportunity to conduct highest quality research. My past five years spent working with him have been exceptionally rewardin g yet at the same time full of fun and excitement. I would like to express my special thanks to Dr. Andreas Keil, for his constant encouragement and guidance and with whom I had the honor and pleasure to collaborate for the most part of my stay at UF. His enthusiasm and professionalism have influenced profoundly my career development. Also, many thanks to my committee members, Drs. Bruce Wheeler and Hans van Oostrom for the ir insightful comments and constructive suggestions throughout various stages of my r esearch and dissertation preparation. I w ould like to thank Dr. Andrew Ahn for introduc ing me to the field of clinical translational research and giving me the opportunity to participate in cutting edge clinical studies. I am also grateful of Dr. Peter Lan g, Dr. Margaret Bradley, Dr. George Mangun at UC Davis, Dr. Robert Knight at UC Berkeley, Dr. Yang Jiang at U of Kentucky, and Dr. On ur Seref at Virginia Tech for the inspiring discussions that facilitated immensely my research and manuscript writing. I am thankful of Vladimir Miskovic, Vincent Costa, Jesse Bengson at UC Davis, and Bradley Voytek at UCSF for


5 their timely assist ance and professional knowledge that helped me overcome obstacles in my research. Special thanks to Haiqing Huang, who has helped m e diligently in carrying out all the lengthy experiments throughout the past few years. Also, special thanks to Rajasimhan Rajagovindan for his insightful comments during the early days of my research and for our fun discussions just about everything. I wa nt to express my big thanks to all former and current lab colleagues: Yan Zhang, Anil Bollimunta, Yonghong Chen, Sahng Min Han, Kristopher Anderson, Xiaotong Wen, Jue Mo, Chao Wang, Amy Trongnetrpunya, Yijun Zhu, Siyang Yin, Felix Bartsch, Abhijit Rajan, a nd Immanuel Babu Henry Samuel. Thank you all for the motivating discussions and also for being my best friends. Thanks to all BME staffs including Tifiny McDonald, Katherine Whitesides, Diana Dampier, Art Bautista Hardman, Ruth McFetridge, Valerie Anderso n, and Todd Andersen Davis. My research and life here in the BME department could not have be en easier all because of your support and assistance in handling the daunting paper works and computer related issues. Above all, I want to express my heartfelt ap preciation to my parents and most importantly, my wife Bing. Your understanding and support have been the everlasting power source that stimulates me to advance forward. My achievement could not have been attained without your unconditional love and sacri fices.


6 TABLE OF CONTENTS Page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURES ................................ ................................ ................................ .......... 9 ABSTRACT ................................ ................................ ................................ ................... 10 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 12 2 EFFECTS OF EM OTIONAL CONDITIONING ON EARLY VISUAL PROCESSING: TEMPORAL DYNAMICS REVEALED BY ERP SINGLE TRIAL ANALYSIS .......... 20 2.1 B ackground and Significance ................................ ................................ ........... 20 2. 2 Materials and Methods ................................ ................................ ...................... 22 2.2.1 Participants ................................ ................................ .............................. 22 2.2.2 Stimuli ................................ ................................ ................................ ...... 23 2 .2.3 Paradigm ................................ ................................ ................................ 24 2.2.4 Data Acquisition ................................ ................................ ...................... 25 2.2.5 ASEO Single trial Analysis ................................ ................................ ...... 26 2.3 Results ................................ ................................ ................................ .............. 29 2.3.1 Average ERP Analysis ................................ ................................ ............ 29 2.3.2 Single trial Estimation of ERPs ................................ ................................ 30 2.3.3 Temporal Dynamics of the P1 Component ................................ .............. 30 2.4 Discussion ................................ ................................ ................................ ........ 32 3 NEURAL SUBSTRATES OF THE LATE PO SITIVE POTENTIAL IN EMOTIONAL PROCESSING ................................ ................................ ................................ ........ 43 3.1 B ackground and Significance ................................ ................................ ........... 43 3.2 Methods ................................ ................................ ................................ ............ 45 3.2.1 Participants ................................ ................................ .............................. 45 3.2.2 Stimuli and Procedure ................................ ................................ ............. 45 3.2.3 Simultaneous EEG fMRI Acquisition ................................ ....................... 47 3.2.4 EEG Data Preprocessing ................................ ................................ ........ 48 3.2.5 Single trial Estimation of LPP ................................ ................................ .. 49 3.2.6 MRI Data An alysis ................................ ................................ ................... 50 3.3 Results ................................ ................................ ................................ .............. 52 3.3.1 ERP Analysis ................................ ................................ ........................... 52 3.3.2 fMRI Analysis ................................ ................................ .......................... 54 3.3.3 Trial by trial Coupling of LPP and BOLD ................................ ................. 55 3.4 Discussion ................................ ................................ ................................ ........ 56


7 3.4.1 Methodological Considerations ................................ ............................... 56 3.4.2 LPP BOLD Coupling and Its Theoretical Significance ............................. 57 3.4.3 Category specific Netw ork Processing ................................ .................... 59 4 MODULATION OF ALPHA OSCILLATIONS IN ANTICIPATORY VISUAL ATTENTION: CONTROL STRUCTURES REVEALED BY SIMULTANEOUS FMRI EEG ................................ ................................ ................................ .............. 69 4.1 B ackground and Significance ................................ ................................ ........... 69 4.2 Materials and Methods ................................ ................................ ...................... 70 4.2.1 Participants ................................ ................................ .............................. 70 4.2.2 Paradigm ................................ ................................ ................................ 71 4.2.3 EEG Data Acquisition and Preprocessing ................................ ............... 72 4.2.4 fMRI Acquisition and Preprocessing ................................ ........................ 74 4.2.5 EEG Spectral Analysis ................................ ................................ ............ 75 4.2.6 EEG informed fMRI Analysis ................................ ................................ ... 76 4. 3 Results ................................ ................................ ................................ .............. 77 4.3.1 Attentional Modulation of Alpha ................................ ............................... 78 4.3.2 BOLD Activations Evoked by the Cue ................................ ..................... 78 4.3.3 BOLD Alpha Coupling: Negative Correlations ................................ ......... 78 4.3.4 BOLD Alpha Coupling: Positive Correlations ................................ .......... 79 4.3.5 Coupling between Alpha Lateralization Index and BOLD ........................ 79 4.4 Discussion ................................ ................................ ................................ ........ 80 4.4.1 Alpha and Dorsal Attention Network ................................ ........................ 80 4.4.2 Alpha and Task Irrelevant Networks ................................ ........................ 82 4.4.3 Differential Roles of IPS and FEF in Controlling Alpha ............................ 83 4.4.4 The Role of dACC in Anticipatory Visual Attention ................................ .. 83 4.4.5 Summary ................................ ................................ ................................ 84 5 CONCLUSIONS ................................ ................................ ................................ ..... 91 LIST OF REFERENCES ................................ ................................ ............................... 95 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 111


8 LIST OF TABLES Table Pa ge 2 1 Results from MARS piecewise linear regression analysis ................................ .. 39 3 1 Regions showing coupling between LPP amplitude and BOLD ......................... 63 4 1 Event related activations following cue onset ................................ ..................... 85 4 2 Coupling between alpha and BOLD with attend left and right combined ............ 86


9 LIST OF FIGURES Figure P age 2 1 Schematic of p aradigm and ERP ................................ ................................ ........ 39 2 2 Temporal distribution o f trials analyzed in this study ................................ .......... 40 2 3 Illustration of ASEO single trial analy sis on a representative subject ................. 41 2 4 Smoothing of single trial ER Ps ................................ ................................ ........... 41 2 5 Te mporal dynamics of P1 amplitude ................................ ................................ .. 42 3 1 ERP analysis ................................ ................................ ................................ ...... 64 3 2 Single trial ERP analysis ................................ ................................ .................... 65 3 3 Single trial LPP dynamics ................................ ................................ ................... 66 3 4 Activation maps based on BOLD contrast ................................ .......................... 67 3 5 LPP BOLD coupling maps ................................ ................................ .................. 68 4 1 An illustration of the se quence of events within a trial ................................ ........ 87 4 2 Attentional modula tion of a lpha power and BOLD ................................ .............. 88 4 3 Negative coupling between BOLD and alpha with Attend Left and Right combined ................................ ................................ ................................ ............ 89 4 4 Positive coupling b etween BOLD and alpha with Attend Left and Right combined. ................................ ................................ ................................ ........... 90 4 5 Coupling between BOLD and alpha lateralization i ndex ................................ ..... 90


10 Abstract of Dissertation Presented t o the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy SINGLE TRIAL ANALYSIS AND ITS APPLICATIONS IN EEG AND SIMULTANEOUS EEG FMRI STUDIES By Yuelu Liu August 2013 Chair: Mingzhou Ding Major: Biomedical Engineering E lectrophysiological recordings such as human electroencephalogram (EEG) and monkey local field potentials (LFP) are traditionally thought to be co mprised o f a deterministic event related potential (ERP) plus a task irrelevant random noise. However, evidence has long suggested that the ERP waveform can vary from trial to trial S uch trial to trial variability often bears important information about the dynamics of the underlying cognitive processes. With r ecent advancements in single trial analysis methods we seek to explore in this dissertation the dynamics of cognitive processes underlying higher level cognitive functions including visual attention, emotional processing, and classical conditioning First, we applied a recently developed single trial analysis method, Analysis of Single trial ERP and Ongoing signals ( ASEO ), to study the trial to trial temporal dynamics of sensory facilitation during classical aversive conditioning. Estimating the am plitude of the P1 component elicited by the conditioned stimuli (CSs), we found that P1 amplitude fluctuations within the acquisition session followed three distinct pha ses. Further, the effects of sensory facilitation toward the CS predicting aversive out comes, compare d to the CS predicting neutral outcomes, were manifested by differences in the


11 rate of P1 amplitude changes within each phase. Second, we investigate d brain structures involved in the generation and modulation of the late positive potential ( LPP) during emotional processing. Correlating single trial LPP amplitude with the simultaneously recorded blood oxygen level dependent (BOLD) activity, we found that regions contributing to scalp recorded LPP included the visual cortices and emotional proc essing structures. Moreover, the degree of contribution to scalp LPP from these structures was valence specific Finally we investigated brain areas contributing to the attentional modulation of alpha (8 12 Hz) in spatial visual attention Our results sug gest that the intraparietal sulcus, a core region within the frontoparietal attention network, modulates the trial to trial alpha desynchronization. Further, we found a positive correlation between alpha lateralization and BOLD in dorsal anterior cingulate cortex (dACC) suggesting a role for dACC in facilitating the attentional set via executive influences over attentional control systems In conclusion, the results suggest that trial to trial variability is an important aspect of the cognitive process E xploring such variability can gain us valuable insights into the neuronal mechanisms of higher level cognitive processes.


12 CHAPTER 1 INTRODUCTION Studies of brain functions by electro en cephalogram (EEG) or local field potentials (LFPs) often rely on averagi ng over a large number of repeated experimental trials to enhance the signal to noise ratio (SNR). This is usually performed by aligning all trial epochs belonging to a certain experimental condition to the onset of an event and then computing an ensemble average across all epochs. The resulting time series, the averaged event related potential (AERP), is an estimate of the true underlying event related potential ( ERP ) and has been the major tool for neuroscientists to explore basic brain functions (Luck, 2 005). The underlying assumption for this ensemble averaging procedure is that the recorded neural signal following experimental events can be modeled as a linear combination of the ERP and an ongoing activity. T he ERP is assumed to be deterministic and doe s not vary across trials, whereas the ongoing activity is treated as task irrelevant brain activities. However, evidence has long suggest ed that the ERP can take distinct waveforms across experimental trials and such trial to trial variability has been dem onstrated to relate to fluctuations in performances as well as cognitive processes (Woody, 1967; Coppola et al., 1978; Truccolo et al., 2002, 2003; Chen et al., 2006; Knuth et al., 2006; Truccolo et al., 2003) Moreover, the ongoing activity is not task ir relevant and has been shown to contain rich information about neuronal oscillations also closely related to cognitive functions (Keil et al., 2007; Klimesch et al., 2007; Anderson and Ding, 2011; Rajagovindan and Ding, 2011; Wang and Ding, 2011) S imply ta k ing the ensemble average as the estimator for the ERP would potentially smooth out such trial to trial information of the underlying cognitive processes and thereby rendering the AERP a suboptimal estimator. Thus, it is


13 important that researchers utilize single trial analysis methods to reduce the level of information deterioration involved in traditional ensemble averaging. The advantage of adopting single trial analysis to examine cognitive functions can be summarized in the following three aspects. In t he first place, single trial ERP estimation allows one to better reconstruct the AERP. In the traditional ensemble averaging approach, trial to trial variability negatively impacts the SNR of the estimated AERP as the latency shift for a given ERP componen t tends to decrease the estimated amplitude and widen the waveform after the averaging procedure. Estimating and accounting for the trial to trial latency shift in single trial ERPs prior to the ensemble averaging can therefore reduce the temporal smearing effect and enhance the SNR of the AERP. Second, single trial analysis allows the study of cognitive functions via trial to trial variability. Fluctuations in single trial ERP amplitudes and latency shifts often contain important information about the cog nitive processes such as attention (Anderson and Ding, 2011; Rajagovindan and Ding, 2011; Zhang and Ding, 2010) working memory (Anderson et al., 2010) conflict monitoring (Debener et al., 2005) and associative learning (Keil et al., 2007) Hence, b y stu dying such task related trial to trial variability, one can gain additional information about the dynamics of neuronal mechanisms that cannot be obtained by the traditional ensemble averaging approach. In fact, single trial ERP analysis has been an indispe nsable component in simultaneous EEG and functional magnetic resonance imaging (fMRI) analysi s (Eichele et al., 2005; Debener et al., 2005, 2007; Bnar et al., 2007)


14 Third, estimating single trial ERP allows better estimation of the ongoing activity (Wan g et al., 2008; Xu et al., 2009) The ongoing activity is usually estimated by subtracting the AERP from the original data. Yet, recent evidence suggests that this simple subtraction scheme might not be sufficient to remove the ERP due to the trial to tria l variability in the ERP time series (Wang et al., 2008) The residual ERP in the estimated ongoing activity can cause spurious results when one attempts to analyze properties of the ongoing activity, leading to erroneous interpretations. S ingle trial anal ysis can reduce the level of residual ERP in the estimated ongoing activity when one adjusts for trial to trial variability in the ERP and subtracts the estimated single trial ERP from each corresponding trial. This alternative approach has been shown to g ive better estimations of the ongoing activity and is effective in reducing spurious functional connectivity ( Wang et al., 2008) A number of single trial analysis methods have emerged in the past decade. These methods involve the use of maximum likelihood estimation (McGillem et al., 1985; Tuan et al., 1987; Lange et al., 1997; Jaskowski and Verleger, 1999) wavelet denoising (Bartnik et al., 1992; Quiroga, 2000; Quiroga and Garcia, 2003) Bayesian estimation (Truccolo et al., 2003) and other iterative me thods (Xu et al., 2009) In particular, an iterative algorithm called trial ERP and Ongoing signal (ASEO; Xu et al., 2009 has been shown to achieve a relative high performance level. The ASEO algorithm assumes that the recorded LFP or EEG activity following event onset can be described by a V ariable S ignal P lus O ngoing A ctivity (VSPOA) model (Chen et al., 2006) This signal generative model assumes that the ERP can be decomposed into a set of time domain components, where each componen t has a fixed waveform but could


15 vary in its amplitude and latency across trials. The ongoing activity is further modeled as oscillatory activities. Based on the VSPOA model, t he ASEO algorithm estimates iteratively the waveforms, amplitude scaling factors, and latency shifts for each ERP components, as well as the AR coefficients for the ongoing activity. It can be shown that with this iterative estimation procedure, the ASEO a lgorithm can outperform other single trial ERP estimation algorithms such as the dVCA method (Truccolo et al., 2003) In this dissertation, the dynamics of neuronal processing underlying three important higher level cognitive functions, i.e., classical con ditioning, emotional processing, and voluntary attention, are studied via the above mentioned ASEO single trial analysis. Specifically, the dissertation examines these questions along the following three specific aims. Aim 1: To investigate the detailed t rial by trial dynamics of sensory facilitation process underlying the classical aversive conditioning. A key aspect of adaptively responding to stimuli that signify threats or rewards is the in depth sensory analysis of such stimuli (Bradley et al., 2003) Simple neutral stimuli, when temporally paired with the emotionally engaging unconditioned stimuli in classical conditioning paradigms, were found to evoke increased sensory responses, compared to responses elicited by the same stimuli but not paired with a noxious stimulus (Stolarova et al., 2006) Such effects of sensory facilitation were further found to increase over two consecutive sessions of conditioning, demonstrating sustained learning effects on the time scale of experimental blocks (Stolarova et al., 2006; Keil et al., 2007) Yet to date, the detailed trial to trial temporal dynamics of the sensory facilitation process is not known.


16 Understanding such detailed temporal dynamics of differential sensory changes during the associative learning proce ss is important for both basic science problems as well as clinical translational research addressing the etiology and treatment of affective disorders (Lissek et al., 2008) For instance, reports on the habituation of the amygdala during classical conditi oning (Breiter et al., 1996; Whalen et al., 1998; Buchel et al., 1999; Zald, 2003) suggest that re entrant modulation of visual areas by amygdalo fugal connections (Amaral and Price, 1984) decreases over time. On the other hand, mapping the trial by trial dynamics of emotional response is essential to characterize the dysfunctional pattern of sensory processing of phobic cues in generalized anxiety disorder ( Mogg et al., 2000; Amir et al., 2009) Traditional ERP measures obtained by means of ensemble averag ing do not allow for detailed information of the learning dynamics in the sensory cortex because such measures require the pooling of a substantial number of trials and thus lack trial specificity. Hence in this aim, the ASEO single trial analysis will be employed to study the emotional experience dependent modulation of the extrastriate cortex during a classical differential conditioning paradigm. The evolving pattern of visual P1 component within a single conditioning block will be examined. I expect to o bserve differential temporal dynamics between the P1 amplitudes elicited by the conditioned stimuli predicting aversive (CS+) vs. neutral (CS ) emotional contents, respectively. Aim 2: To investigate the neural substrates that contribute to the generation and modulation of the late positive potential (LPP) during emotional processing. The LPP is a reliable electrophysiological index of emotional perception in humans. During viewing of affective pictures, LPP is characterized as an amplitude enhancement in E RP starting


17 around 300 400 ms after picture onset for both pleasant and unpleasant stimuli, relative to the neutral stimuli. It has been shown that LPP amplitude var ies systematically with the experienced intensity of the affective picture content ( Schupp et al., 2000; Keil et al., 2002) and exhibit s abnormal patterns in mood disorders and other psychiatric conditions (Foti et al., 2010; Leutgeb et al., 2011; Weinberg and Hajcak, 2011; Weymar et al., 2011; Jaworska et al., 2012) Despite the importance of L PP, brain structures that contribute to the generation and modulation of LPP are not well understood. Studies employing ERP source localization techniques are only partly successful because of the relatively low spatial resolution and the difficulty in res olving deep subcortical structures ( Sabatinelli et al., 2007b; Keil et al., 2002) In parallel, functional magnetic resonance imaging (fMRI) studies have found that viewing of affective pictures is associated with increased blood oxygen level dependent (B OLD) activity in widespread brain regions, including occipital, parietal, inferotemporal cortices, and other higher level emotion processing structures such as amygdala ( Breiter et al., 1996; Lang et al., 1998; Bradley et al., 2003; Norris et al., 2004; Sa batinelli et al., 2005 2009 ) Such extensive network activation suggests that emotionally salient contents enhance visual stimulus processing by attracting attentional and sensory processing resources (Lang et al., 1998a; Lang and Bradley, 2010 ) Hence, I hypothesize that if the enhanced LPP and BOLD activity during affective processing reflect a common underlying mechanism, one might expect a coupling between the LPP amplitude and BOLD activities in regions that contribute to the LPP. To test our hypothes is and identify regions that give rise to the scalp recorded LPP we will record simultaneous EEG fMRI while subjects passively view emotional ly arousing and neutral


18 pictures. Single trial LPP amplitudes, estimated via the ASEO algorithm, will be correlate d with single trial evoked BOLD responses across the entire brain to identify brain structures contributing to the scalp recorded LPP. Through this technique, we wish to identif y structures that contribute to the LPP during processing of both pleasant and unpleasant emotions, respectively. Aim 3: To investigate brain areas and mechanisms underlying top down modulation of alpha oscillations (8 12 Hz) during spatial visual attention. A well established phenomenon during spatial visual attention is the desyn chronization of posterior alpha oscillation on the hemisphere contralateral to the direction of covert spatial attention ( Worden et al., 2000; Sauseng et al., 2005; Thut et al., 2006; Rajagovindan and Ding, 2011 ). Such modulation of posterior alpha is thou ght to reflect an increase in cortical excitability among task relevant cortices through top down attentional mechanisms (Klimesch et al. 2007; Romei et al. 2008, 2010 ). Although putative sources of alpha attentional modulation have been attributed to th e dorsal frontoparietal attention network as well as other higher level executive regions ( Kastner et al., 1999; Shulman et al., 1999a; Corbetta et al., 2000; Hopfinger et al., 2000; Corbetta and Shulman, 2002; Astafiev et al., 2003; Giesbrecht et al., 200 3 ) direct evidence showing modulation of posterior alpha from these brain regions has been scarce. Further, the mechanism through which higher order brain areas control sensory cortices to selectively enhance the processing of behaviorally relevant inform ation and at the same time suppress the processing of behaviorally irrelevant distractors remains largely unknown In this aim we will address the above research questions by correlating trial to trial fluctuations in the posterior alpha power during the anticipatory


19 period with simultaneous ly acquired BOLD activities throughout the brain. We wish to identif y separately regions associated with two forms of alpha attentional modulation, i.e., desynchronization and hemispheric lateralization. In addition, re gions associated with alpha power increases will also be identified to examine potential mechanism of active inhibition over task irrelevant networks


20 CHAPTER 2 EFFECTS OF EMOTIONAL CONDITIONING ON EARLY VISUAL PROCESSING: TEMPORAL DYNAMICS REVEALED BY E RP SINGLE TRIAL ANALYSIS 2.1 Background and Significance Emotions, viewed in an evolutionary context, represent phylogenetically old action dispositions that facilitate survival vival of the organism and the species ( Lang et al., 1998 ) A key aspect of ada ptively responding to stimuli that signify threats or rewards is the in depth sensory analysis of such stimuli ( Bradley et al., 2003 ) In line with this view, neuroscience studies of emotional perception have suggested that processing of emotionally salien t stimuli is facilitated at multiple stages of the visual pathway ( Pizzagalli et al., 1999, 2003; Vuilleumier et al., 2001; Schupp et al., 2003, 2004 ) In situations that require spatial orienting to affectively arousing stimuli, studies using event relate d potentials (ERPs) have shown that early visual components C1 and P1 are enhanced by affective cues ( Pourtois et al., 2004 ) Specifically, experiments using differential classical conditioning have demonstrated increased early cortical responses over lear ning trials for the conditioned stimulus that predicts an unpleasant event (i.e., the CS + ) across the conditioning blocks ( Stolarova et al., 2006 ) As suggested in Keil et al. ( 2007 ) a possible explanation for such early sensory enhancement are changes in the depth of process ing needed for discrimination, as learning progresses: Experience with a given set of stimuli and spatial locations allows observers to predict certain aspects of the stimulus array, which then no longer require in depth processing in each subsequent trial. In terms of neural mass activity, this predictability may lead to constantly increasing cortical facilitation for a specific set of features (e.g., the orientation of the CS + grating), at increasingly early stages of visual analysis ( Stolarova et al., 2006 ) One prediction of this notion is that early trials should be characterized by


21 slower in depth processing, followed by a successive shift of discriminant neural engagement to fewer and earlier levels of visual cortex. Changes in se nsory cortices as a function of learning have been observed in experimental animals ( Lee et al., 2002; Schwabe and Obermayer, 2005; Tolias et al., 2005 ) partic ularly when using intense condi tioning regimens ( Elbert and Heim, 2001 ) Although previous work has identified the effect of learning on the time scale of experimental trial blocks, it is important for both basic science and applied questions to obtain information on the time course of differential changes during emotional learning on a trial to tria l basis. For instance, reports on the habituation of the amygdala during classical conditioning ( Breiter et al., 1996; B chel et al., 1998, 1999 b ; Whalen et al., 1998; Zald, 2003 ) suggest that re entrant modulation of visual areas by amygdalo fugal connect ions ( Amaral and Price, 1984 ) decreases over time. Another question of interest is the time course and duration of early visual e nhancement during the first tri als, as it may represent a measure of the speed of acquisi tion of contingencies ( Moratti and Kei l, 2009 ) Trial by trial dynamics of emoti onal learning are also an impor tant aspect in translationa l research addressing the etiol ogy, maintenance, and treatment of anxiety disorders ( Lissek et al., 2008 ) For instance, initial hyper vigilance and subsequ ent perceptual avoidance of fear related cues may represent a dysfuncti onal pattern of sensory process ing of phobic cues in generalized anxiety disorder, or in depression ( Mogg et al., 2000 ) and this pattern may change over trials, e.g., with attention tr aining ( Amir et al., 2009 ) ERP measures obtained by means of ensemble averaging do not allow for a detaile d picture of the learning dynam ics in visual cortex because they require the


22 pooling of a substantial number of trials and thus lack trial specifici ty. For instance, using ensemble averaging, Stolarova et al. ( 2006 ) examined electrocortical changes on the time scale of trial blocks, which obscures putative condition differen ces in terms of trial to trial dynamics. In this article, a novel single trial ERP analysis method called Analysis of Single trial ERP and Ongoing activity (ASEO) ( Xu et al., 2009 ) is employed to study the emotional experience dependent modulation of the extrastriate cortex during a classical differential conditioning paradigm. Trea ting the visual P1 component elicited by the conditioned stimuli (CS + and CS ) as an index of early sensory processing, we estimated the amplitude of P1 on a single trial basis over conditioning and control blocks and compared the results across the two CS s. In contrast to previous reports by Keil et al. ( 2007 ) and Stolarova et al. ( 2006 ) where block based averaging was used, the current work aimed at examining the evolving pattern of visual P1 amplitude within a single conditioning block. Given that the P 1 has been shown to be modulated by classically conditioned cues in previous studies ( Pizzagalli et al., 2003; Stolarova et al., 2006 ) we expected to observe differential temporal dynamics between the P1 amplitudes elicited by the conditioned stimuli pred icting aversive (CS + ) vs. neutral (CS ) pictures, respectively. 2.2 Materials and Methods 2 .2.1 Participants Datasets from five participants (two males and three females, mean age: 22.4, SD = 1.8) were selected from an earlier study of classical emotional conditioning (Keil et al., 2007), in which 14 undergraduate students participated in exchange of class credit or a financial bonus of ~30 USD. To be selected for the single trial analysis in the present


23 study, datasets had to meet the following three crite ria: (a) Reliable enhancement of the eye blink startle response measured when viewing the CS+, compared to the CS which was taken to index successful acquisition of contingencies (see below), (b) reliable P1 ERP scalp topography among parietal occipital sites, and (c) excellent signal to noise ratio of the recorded EEG signal as indexed by a 3 dB gain between the alpha band power and power in beta and gamma band. Eight subjects showing strong contingency acquisition were included according to the first cr iterion. Among the eight subjects, datasets from two subjects showing abnormal P1 topography during at least one experimental condition were excluded according to criterion b). We further excluded gh frequency artifact from our study. The goal of the above subject inclusion criteria was to make sure that: (1) reliable P1 temporal dynamics could be extracted from each individual subject; and (2) the P1 effects seen in the present participants would b e consistent with successful conditioning as indexed by startle reflex modulation. The data from five participants meeting all three criteria were processed using the single trial analysis technique described below. 2 .2.2 Stimuli One hundred twenty picture s from the International Affective Picture System (IAPS; Lang et al., 1997) were chosen based on their normative ratings of hedonic valence and emotional arousal as listed in the IAPS manual. The 60 unpleasant pictures contained mutilated bodies, attack sc enes, and disgusting objects (mean valence: 2.2, SD = 0.6; mean arousal: 6.1, SD = 0.8). They served as the Unconditioned Stimuli (USs). The 60 neutral pictures served as control stimuli and contained landscapes, people, objects,


24 and abstract patterns (mea n valence: 5.9, SD = 0.7; mean arousal: 3.8, SD = 0.9). The differences between these two picture categories was maximized in terms of both emotional valence and their level of emotional arousal to facilitate differential conditioning despite the relative weakness of picture USs compared to sound or electric shock stimuli used in other works. Picture USs enabled us to use rapid presentation rates, resulting in high trial counts. The affective pictures were presented centrally on a 19 in. computer monitor wi th a refresh rate of 70 Hz and subtended a visual angle of by 8 checkerboards with different colors (black and bright red, black and dark red, black and bright green, and black and dark green) were designed s uch that they matched the size of affective pictures. These checkerboard patterns were used to replace the affective pictures during the control block with no contingencies (see below). ffering only in respectively. They had a spatial frequency of 2.3 cycles per degree with 100% in contrast. The gratings were presented in either the upper left or right visual field (eccentricity of the inner border: 2 .2.3 Paradigm Two experimental blocks, a conditioning block and a control block, conducted in that order on the same day during the original experiment were analyzed in this study. The experimen tal timeline for the conditioning block is shown in Figure 2 1A. For each CS+ trial, the grating pattern designated as the CS+ appeared on the screen for a total of 600 ms. About 200 ms following the onset of the CS+, an affective picture was


25 presented cen trally for a duration of 400 ms. The inter trial interval (ITI) varied randomly between 400 and 1,400 ms (Fig. 2 1A). For each neutral trial, the grating pattern designated as the CS was paired with the neutral and low arousing pictures in the same manner as described for the CS + Both trial types were presented in a randomized fashion within the block. The experimental block contained a total of 480 standard trials (grating followed by an IAPS stimulus), out of which four conditions were formed, by crossi ng the grating orientation (CS + versus CS ) and visual hemifield (left versus right). There were 120 trials for each condition. The experimental time line for the control block was kept the same as the conditioning block except that the affective pictures were replaced with the checkerboards mentioned above. The experiment was designed such that there was no systematic association between the grating orientation and color of checkerboard to minimize contingency reinforcement. During the experiment, particip ants sat at a distance of 80 cm from the computer screen. They were asked to maintain fixation of a white cross in the middle of the screen present at all times throughout recording. A chin rest was used to ensure consistency of head position and to minimi ze head movements. 2.2.4 Data Acquisition The EEG was recorded using an EGI 128 channel system. The vertex (Cz) was the recording reference. The sampling rate was 250 Hz and impedances were kept were sub jected to 0.1 Hz high pass and 100 Hz low pass online filtering. Artifact free epochs (196 ms pre and 600 ms post stimulus interval) were obtained using the SCADS procedure suggested by ( Jungh fer et al., 2000) This procedure creates distributions of sta tistical indices of data


26 quality and lets researchers identify bad channels and trials, with the latter being discarded and the former being interpolated from the full channel set. In a subsequent step, data were re referenced to average reference. The mea n number of artifact free trials per condition was 82. 2.2.5 ASEO Single trial Analysis A recently developed single trial analysis method, called Analysis of Single trial ERP and Ongoing activity (ASEO) ( Xu et al., 2009 ) was adopted to extract the visual single trial ERP from the raw single trial EEG data. A detailed description of the method can be found in the above reference. Briefly, the recorded th trial ( ) of the EEG data is modeled by the variable signal plus ongoing activity (VSPOA) model ( Che n et al., 2006; Xu et al., 2009) : (2 1) where ( ) is the unknown ERP component waveform with being the total number of such components, and are the unknown amplitude scaling facto r and the latency shift for the th ERP component, and is the ongoing acti vity which is further modeled as an AR random process. With proper initial conditions, the ASEO algorithm estimates the ERP component waveforms the corresponding single trial amplitude scaling factors and latency shifts and the AR coefficients of the ongoing activity in an iterative fashion. From these estimated quantities, the single trial event related potentials were reconstituted, forming the basis for estimating the ampl itude of the P1 component. While implementing ASEO single trial


27 analysis, the number of components and the waveform for each component required by the algorithm were selected according to visual inspection of the ERP waveform for each subject. An example i s illustrated in Figure 2 3C. The number of components used in this study varied from three to six. These initial components were selected according to computational considerations, and might not effectively represent the true underlying neural generating processes. Comparison of average ERPs from the raw single trial EEG time series and that from the single trial ERPs estimated from the same raw single trial EEG time series was used to evaluate the effectiveness of the ASEO procedure (Wang and Ding, 2012 ; Wang et al., 2008). The same analysis was applied to both the conditioning block and the control block. Because of the lateralized presentation of the CSs, the single trial analysis was performed on the hemisphere contralateral to CS presentation at two la teral posterior electrodes (channel 66 and 85 of the EGI 128 channel system) for every subject. These two electrodes, selected for exhibiting maximum ERP peak P1 amplitude, were located near sites P3 and P4 in the standard 10 20 system. Single trial ERPs f rom these electrodes were estimated and reconstructed on a trial by trial basis for all four experimental conditions. The estimated single trial ERPs were further subjected to a 10 trial moving averaging procedure to smooth the data and enhance signal to n oise ratio, i.e., the 1st trial to the 10th trial were pooled together to form smoothing Bin 1 and the singletrial ERPs averaged to produce smoothed Trial 1, and the 2nd trial to the 11th trial were pooled together to form smoothing Bin 2 and the single tr ial ERPs averaged to produce smoothed Trial 2, and so on. The window length for the trial to trial moving averaging was selected such that reliable P1 amplitude could be obtained without


28 compromising much trial to trial specificity. The time interval of th e contralateral P1 component was then defined for each condition and subject, separately, and the peak voltage within the time interval was selected as the P1 amplitude measurement. Hemispheric conditions were further combined by averaging the results over the two electrodes, leaving for comparison two experimental conditions: CS+ and CS To examine the effect of emotional conditioning on the CS evoked P1 component, P1 amplitudes for CS+ and CS conditions were averaged across five subjects, and plotted ag ainst the smoothing bin index to reveal the temporal dynamics. The temporal functions of P1 amplitude data for CS+ and CS were described by Multivariate Adaptive Regression Splines (MARS; Friedman, 1991), a segmented linear regression method. The paramete rs of these segmented linear functions were then subjected to statistical analysis. Given the emphasis of our study on the P1 temporal dynamics, we took the following steps to ensure that the time across subjects and conditions were comparable when we perf ormed the above hemispheric and cross subject averaging. First, we ensured that among the subjects included in our analyses, the trials remained after artifact rejection were approximately evenly distributed throughout the block without leaving any major d iscontinuity in time (Fig. 2 2A,B). Second, since the number of remaining trials tended to vary within different experimental conditions across subjects after artifact rejection, we further equalized the number of trials prior to the 10 trial moving averag e by deleting trials randomly from subjects with more remaining trials. The trial deletion indexes were sampled without replacement from a uniform distribution spanning the entire block. To further ensure that deleted trials covered the block evenly,


29 we im posed an additional constraint such that no two consecutive trials were deleted. To demonstrate that the physical time remained comparable after the above procedure, we conducted for every condition within each subject a 10 trial moving average on the rema ining within condition trial indexes after trial equalization to estimate the approximate physical time associated with each smoothing bin used to reveal P1 temporal dynamics. We combined the hemispheric results and further performed a grand average across five subjects leaving for comparison the averaged physical time indexes between CS+ and CS Here we used the within condition trial indexes (i.e., the trial order within its own presentation stream for each condition) as a proxy of the actual physical ti me due to the fact that all four types of trials were evenly intermixed in our experiment design. Figure 2 2C showed the averaged trial index plotted against smoothing bin index for CS+ and CS respectively. The close similarity between CS+ and CS as wel l as the quasi linear pattern observed for both conditions suggest that the P1 dynamics between the two conditions can be compared to assess the effect of emotional conditioning on early visual processing. 2.3 Results 2.3.1 Average ERP Analysis The grand a veraged ERPs time locked to the onset of CS+ and CS are shown in Figure 2 1B. A robust P1 response is seen over the parietal occipital regions contralateral to the stimulated hemifield. Consistent with the previous report (Stolarova et al., 2006), no stat istically significant difference between conditions was observed in grand averaged P1 amplitude (one sided paired t test; left hemisphere: p = 0.423; right hemisphere: p = 0.412).


30 2.3.2 Single trial Estimation of ERPs ASEO single trial analysis was applied to extract the ERP on a trial by trial basis. The process is illustrated in Figure 2 3 for a representative subject. The raw single trial EEG time series from the right parietal occipital area are shown in Figure 2 3A. The ASEO estimated single trial ERPs from the data in Figure 2 3A are shown in Figure 2 3B. Ensemble averages of the data in Figure 2 3A, B are compared in Figure 2 3C. The close similarity between the two averages is taken as evidence that the ASEO method adequately estimated the single tri al ERPs and did not introduce spurious artifacts. The ongoing activities, obtained by subtracting the single trial ERPs in Figure 2 3B from the raw single trial data in Figure 2 3A, are shown in Figure 2 3D. The relatively constant variance of the ongoing activity for the entire epoch (mean 54.921, SD = 11) is seen as further support for the validity of the single trial estimation procedure (Truccolo et al., 2002; Wang and Ding, 2012 ; Xu et al., 2009). 2.3.3 Temporal Dynamics of the P1 Component To examine how the history of emotional conditioning affects early sensory processing, we first analyzed the temporal dynamics of the CS evoked P1 amplitude for the conditioning block measured at sites contralateral to CS presentation across trials. The single trial ERP data was smoothed by using a 10 trial moving average and each 10 trial ensemble was referred to a smoothing bin. Figure 2 4 demonstrates the result for a typical subject for both CS+ and CS conditions. The P1 amplitudes, averaged across two hemisphere s and across all subjects, were plotted as a function of the bin index for both CS+ and CS in Figure 2 5. Although the amplitude of P1 elicited by CS+ is generally larger than that elicited by CS the difference failed to reach significance at


31 the level However, as time progressed along the CS+ stream or the CS stream, the temporal dynamics of the P1 amplitude revealed three distinct phases for both CS+ and CS : (a) an initial decr ease phase, (b) a subsequent increase phase, and (c) a final habituating phase. The data in Figure 2 5A were fit by piecewise linear functions using the MARS algorithm and the result is shown in Figure 2 5B. The durations of the initial decrease phase as m easured in unit of bin indexes for CS+ and CS were both brief and covered eight and six bins, respectively. According to the relationship between smoothing bin index and trial index shown in Figure 2 2C, this decrease phase covered about 26 CS+ trials and 18 CS trials. The slopes for CS+ and CS conditions during this phase were 0.028 and 0.16. A t test on the slopes (Kleinbaum et al., 1988) revealed a significant condition related difference (t(12) = 3.3174, p = 0.0061) (Table 2 1 ). For the subsequent increase phase, the CS+ trials spanned a shorter interval (from 9 to 17 as measured by the bin index), before reaching the maximum value than the CS trials (7 to 24). The estimated corresponding number of trials within each condition covered by this phase was 28 and 64 for CS+ and CS The slopes for the CS+ and CS trials within thi s phase were 0.27 and 0.15. A t test on the slopes within this phase between the two conditions revealed that the slope for CS+ trials was significantly larger than the slope f or CS trials (t(20) = 3.1450, p = 0.0051). The habituating phase for the CS+ trials covered a relatively longer period starting at about the 88th trial compared with the CS trials which started around the 112th trial. The slopes for the CS+ and CS trial s within this phase were 0.064 and 0.0067, respectively. As characterized by the slopes within


32 this phase, the rate of habituation for CS+ trials was significantly larger than that for CS trials (a slightly positive slope) (t(64) = 8.3364, p < 10 4 ). To demonstrate that the P1 amplitude dynamics is specific for the conditioning block, we also applied the same analysis to data from the control block and the result is shown in Figure 2 5C. In contrast to the conditioning block, no systematic changes in pro cessing phase over trials were found during the control block. The average P1 amplitude is also significantly smaller than that of the conditioning block (one sided t test, p < 10 10 ). In addition, the relatively small variation of P1 amplitude across the entire block suggests that participants maintained a steady level of early electrocortical processing during the block. Fitting the P1 amplitude data by the MARS algorithm described above using three segments of linear functions revealed unsystematic effec ts between the CSs. A linear regression on P1 amplitude data across the block showed that the slopes of the linear functions were 0.0064 for CS + and 0.0080 for CS respectively (Fig. 2 5D). Both slopes were considered to be close to zero. 2.4 Discussion Emotionally salient stimuli can attract attentional resources involuntarily to affect visual processing (Mller et al., 2008). Pourtois et al. (2004), incorporating an emotional cue within a spatial orienting task, found enhanced P1 time locked to a bar th at replaced a fearful face, compared to a bar that replaced a happy face. In addition, the C1 component time locked to the face presentation had a greater amplitude for fearful faces as compared to happy faces. In a classical conditioning paradigm, Keil et al. (2007) found gradual increase of early (60 90 ms) gamma band (>30 Hz) oscillations evoked by the CS+, across two consecutive experimental blocks. This was taken to


33 f sensory information (Moratti and Keil, 2009). In the current study, we examined this stimulus history dependent modulation in a classical delayed conditioning experiment on a more detailed time scale by means of single trial ERP analysis (Xu et al., 2009 ). By extracting the P1 component evoked by the conditioning stimuli on a trial by trial basis, we reported three results. First, three distinct phases of P1 amplitude as a function of time specific to emotional conditioning were found: (1) a short initial decrease phase, (2) a subsequent increase phase, and (3) a final habituating phase for both the CS+ and CS trials within the block. Second, the P1 response to CS+ stimuli exhibited slower rate of decrease over the first phase, faster rate of increase ove r the second phase, and again faster rate of decrease over the third phase relative to that evoked by the CS stimuli. Third, for the control block where the same grating patterns used as CS+ and CS stimuli in the condition block were paired with checkerb oards, no systematic temporal effects were found for the P1 amplitude. Previous work has yielded mixed results with respect to P1 differences during classical conditioning ( see Stolarova et al., 2006 ) The present data are informative by revealing the intr a block P1 dynamics associated with learning of contingencies. One possible explanation for the initial P1 decline could be that during the early trials, the contingencies are not yet established and thus neural responses to both CSs may be subject to repe tition suppression. Previous studies in human electrophysiology, neuroimaging, and primate single cell recordings have shown that repeated stimulus presentation tends to elicit decreased neural responses over time in the visual cortex ( Grill Spector et al. 2006; Guo et al., 2007, 2008; Jiang et al., 2000, 2009; Liu et al.,


34 2009; McMahon and Olson, 2007 ) Research in classical conditioning has established that it frequently takes up to 5 trials of both the CS + and the CS for differential conditioning to ta ke place, not only as reflected in psychophysiological measures, but also associated with selective enhancement in human visual cortex for the CS + ( e.g., Moratti and Keil, 2005 ) Considering the fact that our experiment paradigm further added a potential c onfounding factor by adopting a lateralized CS presenting scheme, the number of trials within the initial decrease phase (26 for CS + ; 18 for CS ) might indicate the average amount of trials required to obtain conditioning effect in early visual cortex duri ng our experiment. The smaller reduction rate for CS + as compared to that for CS may be reflective of the fact that any suppression was accompanied by differential enhancement for the CS + It is worth noting that repetition suppression was not apparent in the control block, which may reflect a floor effect, given that the control block was conducted subsequent to the conditioning block, and the overall P1 amplitude was small across the control block. After the initial decrease phase, the P1 response to bot h CS+ and CS underwent a subsequent increase phase, which is consistent with an increase in the predictive value of the CSs across these trials. At this stage, increased salience for both CSs appears to lead to increased attention bias and ultimately faci litated perceptual processing in extrastriate cortex. The fact that the rate of P1 increase was greater for CS+ than for CS suggests that the motivationally relevant predictive stimulus (i.e., the CS+) benefits more strongly from such facilitation. Hence, the difference between the slopes for CS+ and CS during the increase phase further supports the view that


35 experience established during classical conditioning affects the neural network organization underlying early visual processing (Keil et al., 2007). Functional imaging studies of visual aversive conditioning ( LaBar et al., 1998; Morris et al., 1998 a ; Bchel and Dolan, 2000; Adolphs, 2002; LeDoux, 2003 ) have highlighted the amygdala, higher order sensory areas, and frontal cortices as key structures fo r providing re entrant input into visual areas ( Zald, 2003; Sabatinelli et al., 2009). In addition, the amygdala has been hypothesized to mediate cortical plasticity during emotional learning (Hendler et al., 2003). Given that the visual P1 component of th e ERP originates in the extrastriate cortex (DiRusso et al., 2001), and that the amygdala projects to the extrastriate cortex in primates (Amaral and Price, 1984), the differential modulatory effects of the CSs on P1 may reflect a stronger involvement of t he amygdala in re entrant sensory modulation of the CS+ as opposed to the CS The final phase for both CS+ and CS after the increase phase was a habituating phase toward the end of the conditioning block. The P1 response to the CS+ declined gradually as time progressed whereas the P1 response to CS ceased to further increase and remained at a constant level. A widely observed activation pattern of amygdala during emotional perception and conditioning is its initial activation followed by rapid habituatio n or even deactivation as experiment progresses (Breiter et al., 1996; Bchel et al., 1998, 199 9 b ; Phillips et al., 2001; Zald, 2003). Although no measurement from subcortical structures was available in our study, the habituation of the P1 amplitude acros s trials might reflect habituation of deep structures such as the amygdala, which may eventually cease to provide re entry input into visual cortex.


36 From a network point of view, changes in connectivity as a function of learning and experience should be co nsidered. In the domain of auditory classical conditioning, research has shown learning induced plasticity in the receptive fields of the primary auditory areas in animals (Diamond and Weinberger, 1984; Quirk et al., 1997; Weinberger, 2004) and humans (Mor ris et al., 1998 b ). Such changes are accompanied by an increase in dopamine or acetylcholine release (Weinberger, 1995), possibly leading to long term potentiation and the strengthening of neural connectivity (Fox and Wong, 2005). Heightened neural connecti vity has been shown during viewing of emotionally salient cues (Keil et al., 2003), and directional analyses of electrocortical data in humans have suggested re entry can be observed from higher to lower tiers of visual cortex ( Keil et al., 2009 ) Because heightened connectivity is often associated with amplitude reduction of neural mass activity ( B chel et al., 1999 b ; Gruber et al., 2001 ) the response habituation to the CSs in our experiment might thus reflect the strengthening of visual networks represen repeated contingency reinforcement. This is also in line with the notion that cortical modulations during learning occur at increasingly earlier stages as learning progresses, with early trials characterized by la te modulation and late trials characterized by early modulation ( Stolarova et al., 2006 ) Regarding the observed slope difference between the CSs, our results agree with a previous study on the impact of emotion on repetition suppression (Ishai et al., 200 4). Using event related fMRI to measure the rate of repetition suppression, Ishai et al. found that fearful faces which initially elicited stronger activation had a higher rate of repetition suppression compared with neutral faces. Although in our experime nt the CSs


37 acquired emotional salience through conditioning, the higher P1 increase rate during the second stage coupled with subsequent stronger suppression effects for CS+ compared to CS during the third stage suggests that processing of emotional stimu li may become more shifted toward a highly efficient neural network. In terms of attention mechanisms during conditioning, our P1 temporal dynamics during the conditioning block support the prediction error theory of attention during conditioning (Pearce a to decrease if a stimulus fully predicted an event during previous trials and vice versa. measurements ( W ills et al., 2007; Wills, 2009 ) have lent support to this theory by providing evidence that increased attention is correlated with prediction error. In this regard, the increase phase and the habituating phase for the CSs in our study might reflect two dis tinct learning stages with the increase phase indicating high uncertainty level and habituating phase indicating the opposite case according to this theory. On a neurophysiological level, these theoretical notions are consistent with the perspective that p rocessing simple, important, and predictable information may be accomplished by increasingly early stages of visual analysis as learning progresses (Keil, 2004). Thus, initial P1 enhancement should be followed by P1 amplitude reduction (as found here), cou pled with enhancement of the C1 component (60 90 ms post stimulus). Such a pattern was observed for the CS+ when comparing blocks of an identical conditioning protocol (Stolarova et al., 2006). In summary, by adopting a single trial analysis method, our st udy revealed distinct phases of temporal dynamics of the visual P1 component within a classical emotional


38 conditioning paradigm. The fact that the three distinct phases of P1 amplitude dynamics were only found in the conditioning block as opposed to the co ntrol block supports the visual processing through conditioning. By analyzing the differential effects within each processing phase between CS+ and CS trials we were a ble to examine possible neuronal mechanisms of conditioned response. Despite these positives, the present study also has limitations that should be addressed in future studies. First, owing to the stringent selection criteria (see Methods), the number of s ubjects is small (n = 5). Data from both hemispheres had to be combined to create a pool of sufficient trials. As a consequence, we were not able to test the hypotheses in regard to hemispheric lateralization of emotional processing (Alves et al., 2008). S econd, due to the limitation of scalp EEG, we were not able to concurrently record activities from important subcortical structures such as amygdala. Thus, the linkage between the effects observed at the cortical level and the previously established subcor tical conditioning effects remains to be further substantiated. Simultaneous EEG and fMRI recording is a promising technique to overcome this weakness.


39 Figure 2 1. Schematic of Paradigm and ERP. A ) The timeline of experiment. B ) Grand average ERPs for both hemispheres and for both conditions. Condition induced differences in P1 are not statistically significan t (one sided paired t test; left hemisphere: p = 0.423; right hemisphere: p = 0.412).


40 Figure 2 2. Temporal distribution of trials analyzed in this study. A ) Trial raster plot for the conditioning block showing the remaining aversive (CS+) trials (vertica l bars) after artifact rejection. LH: Left Hemifield and RH: Right Hemifield. The horizontal axis denotes the sequential trial index within the two experimental conditions examined in this plot: right hemifield and left hemifield CS+ trials. B ) Trial raste r plot showing the remaining neutral (CS ) tria ls for the conditioning block. C ) Plot showing the relation between the smoothing bin index and the averaged within condition trial index for the conditioning block. Note that a similar relation was observed a lso for the control block (not shown here).


41 Figure 2 3. Illustration of ASEO single trial analysis on a representative subject. A ) Raw single trial EEG data from the right parietal occipital channel under contralateral CS+ condition. B ) ASEO estimated si ngle trial ERP time series. C ) Average of ASEO single trial ERPs is compared with the AERP from raw data. The four components used for initiating the ASEO algorithm were indicated. D ) Single trial ongoing activities obtained by subtracting single trial ERP s in C) from th e raw single trial EEG data in A). Figure 2 4. Smoothing of single trial ERPs. Depicted are averaged single trial ERP time series from a smoothing bin of 10 consecutive trials of the same condition. The P1 amplitude is estimated from the smoothed waveform and plotted as a function of time in Figure 2 5.


42 Figure 2 5. Temporal dynamics of P1 amplitude. A) The P1 amplitude as a function of respectively. Three distinct an initial decrease phase, a subsequent increase phase, and a final habituating phase. B) Piecewise linear regression using MARS. s and s ) P1


43 CHAPTER 3 NEURAL SUBSTRATES OF THE LATE POSITIVE POTENTIAL IN EMOTIONAL PROCESSING 3.1 Background and Signi ficance The event related potential (ERP) method is used extensively in affective neuroscience. A key feature observed in ERPs evoked by emotionally engaging stimuli is the late positive potential (LPP), which is characterized by an amplitude enhancement f or pleasant and unpleasant stimuli, relative to neutral stimuli, and has a centroparietal maximum topography. For affective picture viewing, LPP starts around 300 400 ms after picture onset, and is often sustained throughout the duration of picture prese ntation (Cuthbert et al., 2000) LPP amplitude has been shown to vary systematically with the experienced intensity of the affective picture content (Schupp et al., 2000; Keil et al., 2002) and exhibit abnormal patterns in mood disorders and other psychiat ric conditions (Leutgeb et al., 2011; Weinberg and Hajcak, 2011; Jaworska et al., 2012). In parallel, functional magnetic resonance imaging (fMRI) have found that viewing of affective pictures is associated with increased blood oxygen level dependent (BOL D) activity in widespread brain regions, including occipital, parietal, inferotemporal cortices, and amygdala (Breiter et al., 1996; Lang et al., 1998a; Bradley et al., 2003; Norris et al., 2004; Sabatinelli et al., 2005; Sabatinelli et al., 2009) suggest ing that emotionally salient content enhances visual stimulus processing by attracting attentional resources (Lang et al., 1998b; Lang and Bradley, 2010) Taken together, if enhanced LPP and BOLD reflect a common underlying mechanism, one might expect a co upling between LPP amplitude and BOLD activity in the above reported regions. A prior study recording EEG and fMRI from the same subjects but in separate sessions has found that LPP amplitude was positively correlated with BOLD responses


44 in lateral occipit al, parietal, and inferotemporal cortices (Sabatinelli et al., 2007a) This study did not examine LPP BOLD coupling in other higher order emotional processing areas such as prefrontal cortex and deep subcortical structures known to be involved in emotional perception (Sabatinelli et al., 2009). A more recent study using a between subjects design observed coupling between LPP amplitude and BOLD activity in both deep and anterior structures (Sabatinelli et al., 2012 ), but it is still unclear whether these str uctures are engaged with the LPP in a category specific way based on trial by trial information within each picture category. The advent of the simultaneous EEG fMRI recording technique, together with reliable estimation of single trial ERPs, opens new ave nues to address this problem. We recorded simultaneous EEG fMRI while subjects passively viewed emotionally arousing and neutral pictures. The single trial LPP amplitudes were estimated using a recently proposed method (Xu et al., 2009) and then correlated with the single trial evoked BOLD responses across the entire brain to identify brain structures whose activity is linearly related to the trial by trial variation of the scalp recorded LPP. In addition, in light of a host of prior studies reporting diffe rential engagement of cortical and sub cortical structures in appetitive versus aversive processing (e.g., Sabatinelli et al., 2007a) we investigated whether trial by trial LPP amplitude fluctuations are mediated by different neural generators during diff erent affective states by examining the coupling between LPP amplitude and BOLD within each picture category (pleasant, neutral, unpleasant).


45 3.2 Methods 3.2.1 Participants Fifteen healthy volunteers participated in the experiment in exchange of either co urse credits or a financial incentive of $30. One participant withdrew from the experiment. In addition, data from 3 participants were discarded due to artifacts generated by excessive movement inside the scanner. The remaining 11 participants (7 females, mean age: 20, standard deviation: 2.65) had normal or corrected to normal vision. The experimental protocol was approved by the Institutional Review Board of the University of Florida. Informed consent was obtained from all participants prior to the experi ment. 3.2.2 Stimuli and Procedure The stimuli consisted of 20 pleasant, 20 neutral, and 20 unpleasant pictures selected from the International Affective Picture System (IAPS, Lang et al., 2008) based on their normative valence and arousal levels. The IAPS picture numbers used in this study are: Pleasant: 4311, 4599, 4610, 4624, 4626, 4641, 4658, 4680, 4694, 4695, 2057, 2332, 2345, 8186, 8250, 2655, 4597, 4668, 4693, 8030. Neutral: 2398, 2032, 2036, 2037, 2102, 2191, 2305, 2374, 2377, 2411, 2499, 2635, 2347, 5600, 5700, 5781, 5814, 5900, 8034, 2387. Unpleasant: 1114, 1120, 1205, 1220, 1271, 1300, 1302, 1931, 3030, 3051, 3150, 6230, 6550, 9008, 9181, 9253, 9420, 9571, 3000, 3069. The selected pictures cover a wide range of contents and normative ratings. The p leasant pictures in general included sport scenes, romance, and erotic couples, whereas the unpleasant pictures incorporated threat, attack scenes, and bodily mutilations. The neutral pictures included landscapes and neutral human beings. The mean pleasure


46 (valence) rating for pleasant, neutral, and unpleasant pictures was 7.0, 6.3, and 2.8, respectively. The pleasant and unpleasant pictures had similar mean arousal levels (pleasant: 5.8, unpleasant: 5.9), both being higher than neutral pictures (4.2). Pict ures were chosen to be similar overall in composition, matched in jpeg size across categories, and comparable in rated complexity, to minimize confounds. The experimental paradigm was implemented in an event related fMRI design. Each IAPS picture was cent rally displayed on a monitor for 3 seconds followed by a variable (2800 or 4300 ms) interstimulus interval (ITI). All participants completed 5 experimental sessions in which the pictures were repeated in different random orders. The order of picture presen tation was also randomized across different participants. A cross was displayed at the center of the screen during the entire experiment to aid fixation. Stimuli were presented on an MR compatible monitor using E Prime software (Psychology Software Tools). The monitor was placed outside the scanner bore over the head of the subject. Participants viewed the task presentation in the scanner via a reflective mirror. Before the start of the first experimental session, participants were instructed to maintain ey e fixation whenever the fixation cross is present and viewed the pictures without moving their e yes. After the experiment, as a validation, participants were asked to provide their ratings of 12 representative pictures (4 pictures within each category) the y had not seen during the experiment along the scales of valence and arousal using a paper and pencil version of the self assessment manikin (Bradley and Lang, 1994) The entire experiment lasted about 40 minutes.


47 3.2.3 Simultaneous EEG fMRI Acquisition MR I data were collected on a 3 T Philips Achieva scanner (Philips Medical Systems, the Netherlands). Two hundred and twelve (212) volumes of functional images were acquired using a gradient echo echo planar imaging (EPI) sequence during each session (echo ti slice number: 36, field of view (FOV): 224 mm, voxel size: mm, matrix size: 6464). The slices were acquired in ascending order and oriented parallel to the plane connecting the ant erior and posterior commissure. Slice acquisition was performed within an interval of 1850 ms during each TR, leaving an interval of 130 ms toward the end of the TR where no image acquisition was performed. This image acquisition approach allowed us to vis ually monitor the EEG recording within each volume during the no scan period where EEG was not contaminated by gradient switching. A T1 weighted high resolution structural image was also obtained. EEG data were recorded during the experiment using a 32 ch annel MR compatible EEG system (Brain Products GmbH). Thirty one sintered Ag/AgCl electrodes were placed according to the 10 20 system, and one additional electrode ECG wi ll be used to detect heartbeat events to be used for the removal of the cardioballistic artifact. The EEG channels were referenced to site FCz during recording. suggested by th e manufacturer. EEG signal was recorded with a built in 0.1~250 Hz band pass filter and digitized to 16 bit at a sampling rate of 5 kHz. The digitized EEG signal was then transferred to the recording computer via a fiber optic cable. The EEG


48 recording syst recording session to ensure the successful removal of the gradient artifact in subsequent analyses. 3.2.4 EEG Data Preprocessing Brain Vision Analyzer 2.0 (Brain Products GmbH) was used for data preprocessing. Gradient artifacts in the EEG data were removed using a modified version of the original algorithms proposed by Allen et al. (2000). Briefly, an artifact template was created by segmenting and averaging the data according to the ons et of each volume within a sliding window consisting of 41 consecutive volumes, and subtracted from the raw EEG data. To remove the cardioballistic artifact, an average artifact subtraction method (Allen et al., 1998) was used, in which R peaks were detect ed in the low pass filtered ECG signal and used to construct a delayed average artifact template over 21 consecutive heartbeat events in a sliding window approach, which was subtracted from the original EEG signal. The resulting EEG data were then low pass filtered with the cutoff set at 50 Hz, down sampled to 250 Hz, and re referenced to the average reference. These data were then exported to the EEGLAB (Delorme and Makeig, 2004). SOBI (Second Order Blind Identification) (Belouchrani et al., 1993) was perf ormed to further correct for eye blinking, residual cardioballistic, and movement related artifacts. Recent work has shown that SOBI is effective in removing the residual cardioballistic artifact (Vanderperren et al., 2010), as well as in separating EEG da ta into physiologically interpretable components (Tang et al., 2005; Klemm et al., 2009). The artifacts corrected data were then epoched from 300 ms to 2000 ms with 0 ms being the onset of affective pictures. The prestimulus baseline was defined as 300 t o 0 ms. The EEG


49 epochs were averaged within each condition separately to produce the average ERP (AERP). The AERP coming from each subject was further averaged across subjects to produce the grand average ERP. 3.2.5 Single trial Estimation of LPP Channel P z was chosen to guide our subsequent EEG informed fMRI analysis as it showed strong LPP difference between both emotional conditions and the neutral condition (Figure 3 1B and 3 1C). The ERP of each trial at Pz was estimated using the Analysis of Single tr ial ERP and Ongoing activity (ASEO) method (Xu et al., 2009). ASEO has the following basic steps. First, according to the variable signal plus ongoing activity (VSPOA) generative model (Chen et al., 2006), the recorded EEG data for the r th trial ( ) are expressed as: (3 1) where ( ) is the n th ERP component and is an autoregressive (AR) process modelin g the ongoing activity. Within each individual trial, the th ERP component is characterized by an amplitude scaling factor and a latency shift to account for trial to trial var iability. Second, using a proper initial condition, t he ASEO algorithm estimates the waveforms of the ERP components and their associated amplitude scaling factors and latency shifts in an iterative fashion. Third, from scaled and latency adjusted ERP comp onent estimates the single trial ERP was reconstructed on a trial by trial basis Fourth, the LPP amplitude on each trial w as obtained by averaging the single trial ERP amplitude within a time interval around the peak of LPP (see Figure s 3 2 and 3 3). To date ASEO has been applied to study both monkey local


50 field potential data and human EEG data (Wang et al., 2008; Wang and Ding 2011; Liu et al., 20 12 a, b ) See Xu et al. (2009) for a more detailed description of the ASEO algorithm. 3.2.6 MRI Data Analysis Th e fMRI data were processed using SPM5 ( The first five volumes in an experimental session were discarded to allow the scanner to stabilize. Slice timing was corrected using sinc interpolation to account for differences in acquisition time. The images were then corrected for head movement by spatially realigning the images to the sixth image of each session. Images were further normalized and registered to a standard template within SPM (the Montreal Neurological Institute (MNI) space). The functional volume images were resampled to a spatial resolution of 333 mm. The transformed images were then smoothed by a Gaussian filter with a full width at half maximum (FWHM) of 8 mm. The low frequency temporal drifts were removed f rom the functional images by applying a high pass filter with a cut off frequency of 1/128 Hz, and the global signal was removed by dividing every voxel in a slice by the estimated global signal value. Two separate general linear models (GLMs) with paramet ric modulation were constructed to model the relationship between LPP amplitude and BOLD. In the first model, we were mainly interested in examining the overall LPP BOLD coupling across all three picture categories (i.e. pleasant, neutral, and unpleasant). Therefore, in this model we combined all three picture categories and modeled the resulting single experimental condition with two task related regressors. The first regressor described the combined condition and consisted of a sequence of boxcar function s with unit height


51 synchronized with the onset of pictures. The width of each boxcar function was set to the duration of picture presentation (3 s). For the second regressor, the height of the boxcar functions in the first regressor varied according to the ASEO estimated single trial LPP amplitude, with the mean level of the single trial LPP amplitudes removed (Figure 3 3C). This regressor was intended to account for both the between category and the within category variability in LPP amplitude and its corr elation with BOLD. The two regressors were further convolved with a canonical hemodynamic response function (HRF) before being incorporated into the design matrix. Six regressors further introduced to account for any movement related artifacts during scan. We this model resulting in one statistical map for each subject. In the second model, w e included six task related regressors, with three corresponding to the three picture categories (pleasant, neutral, and unpleasant), and the other three modeling the relationship between LPP amplitudes and BOLD within each picture category. Since this mod el captures trial by trial coupling between LPP category sequences of boxcar functions with unit height placed according to the picture onset within the corresponding categories. The width of the boxcar functions remained the by trial LPP BOLD coupling, we further scaled the height of the boxcar functions with the corresponding mean removed single trial LPP amplitudes within each picture category. Similar to the


52 full model described above, the six task related regressors were convolved with a canonical HRF. Additional six regressors also introduced as covariates in the model. The following contrasts were performed based on this model: pleasant vs. neutral, unpleasant vs. neutral, LPP BOLD coupling within each category (pleasant, neutral, and un pleasant). Second level analyses was performed using random effects models based on the statistical maps obtained from the within subjects analyses to examine reproducible effects across all subjects. For conventional BOLD contrasts, the group level T map s were thresholded at p < 0.05 (FDR corrected). For LPP BOLD coupling, because it is derived from the trial to trial variability signal on top of the large picture evoked response, and this residual variability is generally smaller than the large picture e voked response and may contain other ongoing brain processes that are not related to the experimental task, the correlation was generally smaller and required a more relaxed statistical threshold. In line with recent studies using EEG informed fMRI analysi s (see, e.g., Debener et al., 2005; Eichele et al., 2005; Bnar et al., 2007; Scheeringa et al., 2011), for LPP BOLD coupling effects, the group level T maps were thresholded at p < 0.003 (uncorrected). A cluster level threshold of k = 5 voxels was further imposed. 3.3 Results 3.3.1 ERP Analysis Post experiment ratings of 12 representative pictures indicate that the subjects correctly distinguished the three categories of pictures (valence: pleasant = 6.5; neutral = 5.3; unpleasant = 2.6; arousal: pleasant = 4.7; neutral = 2.9; unpleasant = 4.0). Figure 3 1A shows enhanced positivity for both pleasant and unpleasant pictures, relative to


53 neutral pictures, in the grand average ERP at Pz, starting from about 300 ms after picture onset. Since the time interval during which LPP reached a maximum was relatively broad, the LPP amplitude was measured by taking the mean within 300 to 600 ms. A one way analysis of variance on LPP amplitudes with repeated measures identified a significant picture category related diffe rence in LPP amplitudes (F(2,20) = 23.11, p < 0.05). As further indicated by the results of post hoc tests with Bonferroni adjusted significance level, the mean LPP amplitudes for both the pleasant (M = 3.153, SD = 1.733) and unpleasant (M = 3.090, SD = 2. 048) pictures were significantly larger than that for the neutral pictures (M = 1.523, SD = 1.684; pleasant vs. neutral: t(10) = 6.26, p < 0.001; unpleasant vs. neutral: t(10) = 5.65, p < 0.001). However, no significant difference was found in LPP amplitud es between the pleasant and unpleasant categories (t(10) = 0.227, p = 0.825). The ERP difference topography further confirmed that the positivity is strongest among parietal channels for both pleasant and unpleasant conditions (Figure 3 1B and 3 1C), agree ing with prior ERP studies of emotion and motivation (e.g. Lang and Bradley, 2010). The enhanced positivity was sustained throughout the duration of picture presentation for both pleasant and unpleasant pictures. Artifact removed raw EEG data and ASEO est imated single trial ERPs at Pz are shown in Figure 3 2A and 2B for a representative subject. Displayed as raster plots in Figure 3 2C and Figure 3 2D, the ASEO estimated single trial ERPs improved signal to noise ratio, and preserved the trial by trial dyn amics of the LPP amplitude, which is important because the single trial LPP amplitudes were used to correlate with BOLD response in subsequent analyses. The validity of the single trial ERPs can be further


54 supported by averaging the data in Figure 3 2A and 3 2B (Xu et al., 2009; Wang et al., 2008; Wang and Ding, 2011). The similarity between ASEO AERP and the original AERP indicated that the algorithm accurately estimated the single trial ERPs from the raw data (Figure 3 2E). Figure 3 3A and 3 3B display si ngle trial LPP amplitudes as functions of trial index. From these two figures, one can see that, on average, the single trial LPP amplitudes for both pleasant and unpleasant pictures are higher than those for the neutral condition, yielding further support for the grand average ERP result in Figure 3 1. The estimated single trial LPP amplitudes were used to scale the boxcar functions to examine the relationship between LPP amplitude and BOLD (Figure 3 3C). 3.3.2 fMRI Analysis The traditional fMRI group leve l activation maps contrasting 1) pleasant against neutral and 2) unpleasant against neutral picture categories are shown in Figure 3 4A and 3 4B. Both pleasant and unpleasant pictures activated the emotion processing network, encompassing the visual cortic es and deep structures. Specifically, relative to neutral pictures, pleasant pictures mainly activated areas in bilateral occipito temporal junctions, bilateral posterior parietal cortices, medial prefrontal cortex, and left orbital frontal cortex. Other a ctivated areas included fusiform gyrus, lingual gyrus, middle frontal gyrus, supramarginal gyrus, parahippocampal gyrus, and temporal pole (Figure 3 4A). Unpleasant pictures mainly activated areas such as the bilateral occipito temporal junctions, bilatera l posterior parietal cortices, bilateral ventral lateral prefrontal cortices, left orbital frontal cortex, bilateral amygdalae/hippocampi, insula, and supplementary motor area. Other activated areas included fusiform gyrus, lingual gyrus, supramarginal gyr us, temporal pole, and postcentral cortex (Figure 3 4B). In general,


55 the activation results agree with a previous report employing a similar experimental protocol (Sabatinelli et al., 2007a), and serve to demonstrate that the quality of the fMRI data is pr eserved despite the presence of the EEG recording system in the scanner. 3.3.3 Trial by trial Coupling of LPP and BOLD To assess the coupling between the LPP amplitude and BOLD, the coefficients for regressors associated with LPP amplitude variations were model, which combined the neutral, pleasant, and unpleasant picture categories as a single regressor to describe the effect of both between and within category LPP amplitude variations on BOLD, the single trial LPP amplitude was positively correlated with evoked BOLD responses in bilateral occipito temporal junctions, insula, amygdala/hippocampus, temporal poles, and left orbital frontal cortex (Figure 3 5A). pling between LPP and BOLD within each picture category, it was found that for the neutral condition, no significant coupling between single trial LPP amplitude and BOLD existed among subjects. For the pleasant condition, the single trial LPP amplitude was positively correlated with BOLD responses in bilateral occipito temporal junctions, amygdala, temporal poles, precuneus, right nucleus accumbens (NAcc), medial prefrontal cortex (MPFC), and cerebellum (Figure 3 5B). For the unpleasant condition, the singl e trial LPP amplitude was positively correlated with BOLD responses in bilateral ventral lateral prefrontal cortices, bilateral insula, temporal poles, precuneus, left middle temporal cortex, and left postcentral cortex (Figure 3 5C). Table 3 1 listed the MNI coordinates of these regions. It is worth noting that we did not find any structures in


56 which BOLD was negatively correlated with LPP amplitude under the same significance level. 3.4 Discussion Emotional stimuli evoke a late positive potential (LPP) wh ich is interpreted to signify enhanced attention and visual processing (Bradley, 2009). This signature ERP response is known to be altered in mood disorders and other related psychiatric illnesses (Foti et al., 2010; Leutgeb et al., 2011; Weinberg and Hajc ak, 2011; Jaworska et al., 2012; Weymar et al., 2012). Despite the importance of LPP its neural substrate is not clear. ERP source localization is only partly successful (Keil et al., 2002; Sabatinelli et al., 2007a). This problem is addressed here by reco rding simultaneous EEG and fMRI while subjects viewed IAPS affective pictures. Extracting LPP on a trial by trial basis, the overall LPP amplitude variability across three picture categories (pleasant, neutral and unpleasant) was found to be correlated wit h BOLD responses in an extensive cortical and subcortical network, including visual cortices and deep emotion processing structures. In addition, consistent with the notion that appetitive and aversive information may engage different neural substrates, th e brain areas where BOLD activity was correlated with LPP amplitude during pleasant picture viewing were not the same as those during unpleasant picture viewing. 3.4.1 Methodological Considerations Prior investigation of the association between LPP and evo ked BOLD responses relied on recording EEG and fMRI over separate sessions and correlating averaged responses across subjects (Sabatinelli et al., 2007a; Sabatinelli et al., 2012 ). One potential drawback of such an approach is that it is difficult to keep the psychological


57 and biological conditions exactly the same in different recording sessions, and moreover, the correlation between average LPP and BOLD does not reflect their trial by trial co variations and coupling towards individual pictures within eac h subject. A new technology, simultaneous EEG fMRI, has become available over the past few years. As has been recently demonstrated (Nagai et al., 2004; Eichele et al., 2005; Debener et al., 2005; Scheeringa et al., 2011), simultaneous EEG fMRI is capable of overcoming these limitations, and has the potential to allow the interrogation of trial by trial associations between the two recording modalities. The present study benefits from another methodological development. Applying a recently proposed single t rial analysis algorithm called ASEO we were able to estimate the LPP amplitude on a trial by trial basis. The improved signal to noise ratio helps to more fully reveal the brain areas whose BOLD responses are correlated with LPP fluctuations. The ASEO algo rithm has been tested in both monkey local field potential data (Wang et al., 2008; Xu et al., 2009) and human scalp EEG data (Fogelson et al., 2008; Wang and Ding, 2011; Liu et al., 2012 a, b ), and proven useful to address questions arising in a number of cont exts, ranging from the proper preprocessing of event related data for functional connectivity analysis to the temporal dynamics of emotional conditioning. 3.4.2 LPP BOLD Coupling and Its Theoretical Significance The brain regions where BOLD activities co vary with single trial LPP amplitude across three picture categories reflected a joint involvement of the visual system and a network of structures known to be associated with emotional processing. Past source space modeling of LPP has only been able to id entify generators in the visual system, including occipito temporal, parietal, and inferior temporal cortices (Keil et al., 2002;


58 Sabatinelli et al., 2007a), despite the fact that the amplitude of LPP is closely related to the rated intensity of emotion (S chupp et al., 2000; Keil et al., 2002). In line with a recent study (Sabatinelli et al., 2012 ), the present study extends the prior findings by showing that deep structures such as the insula and the amygdala, along with visual structures, contribute to th e generation of LPP and its amplitude modulation. These results provide further evidence supporting the view that emotional pictures naturally attract attentional resources as a result of the engagement of the fundamental motivational system (Cacioppo et a l., 1993, 1994; Palomba et al., 1997; Lang et al., 1997, 1998b; Schupp et al., 2000; Pastor et al., 2008; Lang and Bradley, 2010). The contribution to scalp recorded potentials by the emotional processing areas may be modulatory and mediated by the visual cortex. It has been hypothesized that when observers view emotionally engaging scenes cortical and deep subcortical structures modulate visual cortex in a re entrant fashion (Keil et al 2009; Pessoa and Adolphs, 2010). These structures include the amygd ala, insula, and prefrontal cortex (Rotshtein et al., 2001; Phan et al., 2002; Adolphs, 2002; LeDoux, 2003; Zald, 2003; Luo et al., 2007). As evidenced by recent human intracranial studies, amygdala and orbitofrontal cortex show fast responses to the emoti onal content of stimuli, which would enable them to provide re entrant feedback to the visual cortices (Oya et al., 2002; Krolak Salmon et al., 2004), to potentially affect the gain of visual neurons. In addition, the activation of emotion related BOLD mod ulations in the amygdala is found to precede that in the fusiform gyrus, the medial occipital gyrus, and the calcarine fissure, entrant interaction (Sabatinelli et al., 2009). Taken together the positive correlation found in the current


59 study between single trial LPP amplitude and BOLD activity in the amygdala, insula and areas in prefrontal cortex is supportive of the re entrant hypothesis of emotional perception. 3.4.3 Category specific Ne twork Processing Restricting to pictures within each valence category revealed category dependent differences in regions showing LPP and BOLD coupling. For pleasant pictures, LPP amplitude variability was found to be linearly related to BOLD activity in bi lateral amygdalae, whereas for unpleasant pictures, this correlation was absent. This finding was further corroborated by a contrast of LPP BOLD coupling maps between pleasant and unpleasant conditions showing that the amygdala was preferentially engaged i n LPP modulation in the pleasant condition. For the unpleasant pictures, the amygdala was found to be activated by a traditional fMRI contrast between unpleasant and neutral conditions, suggesting a rather constant level of amygdala activation on a trial b y trial basis. Specifically, while the amygdala activity is clearly enhanced by unpleasant pictures as a whole, it does not co vary with trial by trial LPP changes for different pictures within this category. This may reflect a limited response variability of the amygdala for unpleasant scenes or the presence of additional sources of variance that govern the modulation of LPP. A wider selection of unpleasant scenes may help to identify the potential sources of co variation between electrophysiological measu res and amygdala BOLD activity during aversive/defensive engagement. Whether amygdala responds to pleasant stimuli is debated, although several studies reported amygdala activation during processing of pleasant stimuli, especially when these stimuli have


60 al., 2001; Hamann et al., 2002; Zald, 2003). In the present study, the fact that the amygdala was not activated in a traditional fMRI analysis by contrasting the pleasant with the neutral cat egory may indicate that the overall mean amygdala BOLD response to pleasant pictures was small. Yet, the positive correlation between LPP amplitude and BOLD activities in bilateral amygdalae suggests that the BOLD fluctuations in the amygdala is parametric ally related to intensity variations of pleasant emotional content, as measured by the LPP. This finding further demonstrates that combining electrophysiological recordings and functional imaging can yield information not possible with either modality alon e. The LPP BOLD coupling was found in MPFC and NAcc only for pleasant pictures. Contrasting LPP BOLD coupling maps between pleasant and unpleasant conditions showed that the MPFC is preferentially coupled with LPP in the pleasant condition. In addition, t he MPFC was activated by contrasting pleasant with neutral pictures, but the same region was not activated when comparing unpleasant and neutral pictures. NAcc and MPFC are densely interconnected (Ferry et al., 2000; Roberts et al., 2007) and often show co rrelated activities in human reward studies (Knutson et al., 2003; Rogers et al., 2004), leading to the view that both NAcc and MPFC are part of the human reward system mediating appetitive behaviors. Several studies have reported involvement of both struc tures in the perception of pleasant emotional stimuli including attractive faces, romance, and erotica (Aharon et al., 2001; Karama et al., 2002; as well as in viv id imagery of pleasant scenes (Costa et al., 2010). Hence, the observed


61 positive correlation between LPP amplitude and key structures in the reward system may reflect the contribution to the cortical potential by the appetitive system. For unpleasant pictu res, the LPP was correlated with BOLD in insula and adjacent temporal and ventrolateral prefrontal cortices (VLPFC). The same regions were found to be active when contrasting unpleasant with neutral pictures. It has been shown repeatedly that the human ins ula is involved in tasks that challenge the representation of bodily states as well as processing of emotions (Craig, 2009; Gu et al., 2010; Fan et al., 2011), especially for aversive emotions such as disgust and threat (Phillips et al., 1997, 1998; Adolph s, 2002; Straube and Miltner, 2011). Reliable co variation of the insula and peri insula with the LPP during aversive perception demonstrates that these structures contribute to the modulation of cortical potential during aversive events. It also suggests that the aversive/defensive circuitry involved in processing unpleasant pictures is not engaged in an all or none fashion, but varies parametrically as a function of aversive motivation, indexed by the LPP amplitude. It is worth noting that insula was not activated when LPP BOLD coupling maps were contrasted between pleasant and unpleasant conditions. In light of the finding that insula is activated during viewing of highly arousing pleasant stimuli (e.g. erotica; Karama et al., 2002), this may suggest tha t for pleasant pictures, insula is engaged in LPP modulation but the degree of modulation did not reach the level of statistical significance. Finally, for both pleasant and unpleasant pictures, BOLD activity in regions within midline parietal cortex is l inearly correlated with the LPP. For pleasant pictures, the region that was most correlated with LPP amplitude was within the precuneus, whereas for unpleasant pictures, such correlation occurred in more ventral regions, particularly


62 the posterior cingulat e cortex and precuneus. The involvement of these parietal regions processing resources (K eil et al., 2002; Keil et al., in press).


63 Anatomical Regions Side MNI Coordinates (x,y,z) Z score Three picture categories combined: Occipital Cortex Left 27, 96, 3 3.33 Right 27, 99, 15 3.62 Superior Temporal Cortex Left 45, 3, 6 3.51 Right 45, 0, 9 3.19 Middle Temporal Cortex Left 45, 63, 3 3.00 Inferior Temporal Cortex Right 54, 63, 12 3.18 Insula Left 45, 9, 9 4.30 Right 42, 6, 6 3.83 Orbitofrontal Cortex Left 30, 12, 15 3.33 Amygdala/Hippocampus Left 24, 3, 24 3.07 Right 21, 6, 19 3.18 Temporal Pole Left 33, 3, 42 3.10 Right 27, 3, 42 3.62 Pleasant pictures: Occipital Cortex Left 33, 90, 9 3.83 Right 33, 87, 6 4.81 Middle Temporal Cortex Left 53, 72, 12 3.63 Right 60, 60, 9 4.49 Inferior Temporal Cortex Right 54, 63, 6 3.41 Amygdala Left 21, 0, 18 4.28 Right 21, 0, 18 3.67 Temporal Pole Left 27, 6, 21 3.94 Right 42, 21, 27 4.27 Precuneus 3, 48, 57 3.7 5 Medial Prefrontal Cortex Right 9, 63, 15 3.47 Cerebellum Left 45, 63, 24 3.63 Nucleus Accumbens Right 6,12, 9 3.36


64 Figure 3 1. ERP analysis. A) Grand average (n = 11 subjects) ERP showing the LPP at Pz with time zero set to the onset of pictures. B) The scalp topography showing the ERP difference between pleasant and neutral conditions. Here ERP was averaged within the time interval from 300 to 600 ms. C) The scal p topography showing the ERP difference between unpleasant and neutral conditions. Here ERP was averaged within the same interval.


65 Figure 3 2. Single trial ERP analysis. A) Epoched EEG data at Pz from a representative subject after artifact removal. B ) Single trial ERPs estimated using ASEO from data shown in A C) Raster plot of the EEG data in A (smoothed with a moving average across 5 trials for visualization purpose). D) Raster plot of single trial ERP data in B smoothed the same way as in C E) Co mparison between averaged ERP (AERP) using data in A and averaged ASEO single trial ERPs (ASEO AERP) using data in B


66 Figure 3 3. Single trial LPP dynamics. A) Trial by trial LPP amplitude for the pleasant and neutral conditions. B) Trial by trial LPP amplitude for the unpleasant and neutral conditions. The horizontal axes in A and B represent the sequential index of picture presentation within each picture category. C) Schematic illustrating the use of single trial LPP amplitude as a parametric modula tion in GLM. The height of each boxcar function is scaled by the mean removed LPP amplitude.


67 Figure 3 4. Activation maps based on BOLD contrast. A) Pleasant vs. Neutral (P vs. N) condition. B) Unpleasant vs. Neutral (U vs. N) condition. Activations ar e thresholded at p = 0.05 FDR corrected. PPC: posterior parietal cortex, OFC: orbital frontal cortex, MPFC: medial prefrontal cortex, OTJ: occipitotemporal junction, AMYG: amygdala, HIPP: hippocampus, TP: temporal pole, SMA: supplementary motor area, VLPFC : ventral lateral prefrontal cortex, INS: insula.


68 Figure 3 5. LPP BOLD coupling maps where highlighted regions indicate significant correlation between trial by trial fMRI response and the corresponding single trial LPP amplitude. A) Pleasant, Neutra l, and Unpleasant combined. B) Pleasant. C) Unpleasant. All maps are thresholded at p = 0.003. A cluster threshold k = 5 is further applied. OTJ: occipitotemporal junction, INS: insula, AMYG: amygdala, HIPP: hippocampus, TP: temporal pole, PCu: precuneus, PCC: posterior cingulate cortex, MPFC: medial prefrontal cortex, VLPFC: ventrolateral prefrontal cortex, NAcc: nucleus accumbens.


69 CHAPTER 4 MODULATION OF ALPHA OSCILLATIONS IN ANTICIPATORY VISUAL ATTENTION: CONTROL STRUCTURES REVEALED BY SIMULTANEOUS FMRI EEG 4.1 Background and Significance Our ability to focus attention is a core cognitive faculty. Uncovering the neuronal mechanisms of attention remains a key challenge for neuroscience and represents an essential goal in the translational efforts to mitiga te attention deficits in a variety of psychiatric and neurological disorders. Extensive research on selective attention has focused on stimulus evoked responses and their attentional modulation. It is becoming increasingly clear that a complete understandi ng of attention mechanisms requires improved knowledge of the processes that underlie the deployment and control of attention in advance of sensory stimulation. In particular, how higher order brain areas control sensory cortices to prospectively enhance t he processing of behaviorally relevant signals and suppress the processing of behaviorally irrelevant distractors remains largely unknown. The modulation of posterior alpha oscillation (8 12 Hz) following an attentional cue is a robust neural marker sign ifying selective sensory biasing by covert attention via top down mechanisms. When covert attention is directed to one side of the visual field, alpha oscillation is more strongly suppressed over the hemisphere contralateral to the attended hemifield ( Word en et al., 2000; Sauseng et al., 2005; Thut et al., 2006; Rajagovindan and Ding, 2011). Such desynchronization and hemispheric lateralization of alpha are thought to reflect an increase in cortical excitability among task relevant sensory cortices to facil itate upcoming input processing ( Sauseng et al., 2005; Thut et al., 2006; Romei et al., 2008 ). Putative sources of the top down control signals include the dorsal frontoparietal attention network and other higher level executive regions


70 known to mediate go al directed behaviors ( Kastner et al., 1999; Shulman et al., 1999 ; Corbetta et al., 2000; Hopfinger et al., 2000; Corbetta and Shulman, 2002; Astafiev et al., 2003; Giesbrecht et al., 2003 ). Two recent studies employing repetitive transcranial magnetic sti mulation (rTMS) showed disrupted alpha lateralization after selectively disturbing the activities in FEF and IPS (Capotosto et al., 2009, 2012), providing evidence of top down modulation of alpha by the dorsal attention network. Evidence from non perturbat ive testing, however, has remained scarce. The simultaneous EEG fMRI technique opens avenues to address this problem. Resting state studies have established the feasibility of the technique by revealing correlations between alpha amplitude and blood oxygen level dependent (BOLD) activities in both the frontoparietal and default mode networks ( Laufs et al., 2003 a 2003 b 2006; Moosmann et al., 2003; Mo et al., 2013). In the present study we recorded simultaneous EEG fMRI from subjects performing a cued spati al visual attention task. Correlating single trial alpha power and alpha lateralization with BOLD activity across the entire brain, we wish to identify sources responsible for different aspects of alpha attentional modulation. Besides the dorsal attention network, we further hypothesized down attentional modulation, in light of the directed behaviors (Dosenbach et al., 2006, 2007, 2008 ). 4.2 Materials and Methods 4.2.1 Partic ipants The experimental protocol was approved by the Institutional Review Board at the University of Florida. Eighteen right handed college students with normal or corrected


71 to normal vision and no history of mental disorders gave written informed consent and participated in the study in exchange for course credits. Data from five participants were excluded due to one of the following three reasons: 1) poor behavioral performance (1 participant), 2) difficulties in following task instructions (1 participant ), and 3) excessive body or eye movement (3 participants). The remaining thirteen participants (5 females) have a mean age of 19 (SD = 1.34). 4.2.2 Paradigm Stimuli were displayed on a 30 inch MR compatible LCD monitor with 60 Hz refresh rate which was pla ced outside of the scanner bore over the head of the subject. Participants viewed the stimulus presentation via a reflective mirror system with a viewing distance of approximately 230 cm. Within an experimental session, participants were instructed to main tain constant fixation on a white point positioned at the center of a gray background. Two additional points were placed at the lower left and lower right meridian) to mark the locations where target stimuli would appear. As illustrated in Figure 4 1, in the paradigm, each trial began with a cue presented slightly above the central fixation point briefly for 200 ms, which instructed the participant to covertly direct their attention to either the lower left or lower right visual field. Left directing cue and right directing cue has equal probability. Following a variable cue target interval randomized between 2000 and 8000 ms, target stimuli comprised of vertical black and were flashed briefly for 100 ms at one of the marked peripheral spatial locations with equal probability (50% target validity). When target stimuli appeared at the attended


72 spatial location, partic ipants were required to discriminate the spatial frequency (5.5 vs. 5.0 cycles per degree) of the gratings and make a speedy 2 button choice using their right index or middle fingers without sacrificing accuracy. Stimuli occurred at the unattended location were ignored. The inter trial interval (ITI) was randomized between 2000 ~ 8000 ms. In addition to the two instructional cues (left or right), there was a third type of cue in the experiment which instructed the participants to freely select the visual fi eld to attend. The data from the choice cue were not analyzed here. The symmetric All participants went through a training session before the actual experiment to ensure that proper ey e fixation as well as a reasonable level of performance could be maintained (above 70%). The task was divided into multiple sessions with the length of each session kept around six minutes to help participants maintain a constant level of attention within the session. The participants completed between eight and twelve sessions for the experiment. A short break was administered between two adjacent sessions. 4.2.3 EEG Data Acquisition and Preprocessing Continuous EEG data was collected during the experiment with a 32 channel MR compatible EEG recording system (Brain Products). Thirty one sintered Ag/AgCl electrodes were placed on the scalp according to the 10 20 system. One additional am (ECG), which was subsequently used to remove the cardioballistic artifact during EEG the experiment per recommendation of the manufacturer. The EEG signal was


73 referenced to site FCz during recording and filtered online with a built in 0.1 ~ 250 Hz bandpass internal clock, a step important for the proper removal of the gradient artifacts in subsequent preprocessing. The initial EEG preprocessing was performed in BrainVision Analyzer 2.0 (Bra in Products). Gradient artifact and ballistocardiogram (BCG) were corrected according to a modified version of the average artifact subtraction (AAS) method proposed in Allen et al. ( 1998, 2000) Specifically, the gradient artifact was corrected by first c onstructing an average artifact template over 41 consecutive volumes in a sliding window fashion and then subtracting it from the raw EEG data. The BCG was removed using a similar approach in which R waves were first detected and 21 consecutive ECG segment s defined around the R waves were averaged to produce a BCG artifact template. The resulting artifact templates were then subtracted from EEG data to correct for BCG contamination. The MR corrected EEG data were bandpass filtered from 0.1 to 50 Hz and down sampled to 250 Hz before being exported to EEGLAB (Delorme and Makeig, 2004) for further analyses. The continuous EEG data were epoched 500 ms before to 1500 ms after cue onset, according to two experimental conditions, i.e., attend left and attend right. Only trials with correct responses were included. Epochs were visually inspected for artifact contamination and trials containing excessive body motion or eye movement related artifacts were rejected. The mean trial rejection rate was 9.5%. The average num ber of artifact free trials was 67 and 66 for attend left and attend right conditions, respectively, for each subject. Further EEG preprocessing using Second Order Blind Identification


74 (SOBI) (Belouchrani et al., 1993) was applied to correct for any residu al BCG, eye blinking, and movement related artifacts. To sharpen spatial localization, the artifact removed scalp voltage data was converted into reference free current source density (CSD) data by calculating a 2 D spatial Laplacian (Mitzdorf 1985; Chen e t al., 2011). 4.2.4 fMRI Acquisition and Preprocessing MR images were acquired using a 3T Philips Achieva scanner (Philips Medical Systems) equipped with a 32 channel head coil. Functional images were collected during the experimental sessions using an ech o planar imaging (EPI) sequence with the following scanning parameters: repetition time (TR), 1.98 s; echo time (TE), 30 ms; flip matrix size, 64 64. The slices were ori ented parallel to the plane connecting the anterior and posterior commissures. Image acquisition was performed during the initial 1.85 s within each EPI volume, leaving an interval of 130 ms towards the end of each TR where no image acquisition was perform ed. This acquisition approach enables online visual monitoring of EEG acquisition during the period not contaminated by gradient artifacts. MRI data were processed in SPM5 ( ). Preprocessing steps included slice timing correction, realignment, spatial co registration, normalization, and smoothing. Slice timing correction was carried out using sinc interpolation to correct for differences in slice acquisition time within an EP I volume. The images were then spatially realigned to the first image of each session by a 6 parameter rigid body spatial transformation to account for head movement during acquisition. Each eal Neurological


75 Institute (MNI) space. All images were further resampled to a voxel size of 3 3 3 mm and spatially smoothed using a Gaussian kernel with 8 mm full width at half maximum (FWHM). Slow temporal drifts in baseline were removed by applying a high pass filter with cutoff frequency set at 1/128 Hz. Global effects were accounted for by using the proportional scaling approach which divides each voxel by the spatial average of signals from all cerebral voxels (Fox et al., 2009). 4.2.5 EEG Spectra l Analysis EEG power spectral density (PSD) was calculated on CSD data from 500 ms 1000 ms after cue onset (Figure 4 2A) using the FFT based periodogram approach. To average the power spectrum across subjects, the power spectrum from each subject was norma attend left condition. Such normalization was not done when correlating alpha power with the BOLD signal. For each subject, trial by trial spectral power in the alpha band was calculated by integrating the unnormalized single trial power spectrum within the range of 8 to 12 Hz in regions of interests (ROIs) over occipitoparietal sites where alpha showed the strongest attentional effects (Figure 4 2B and 4 2C; left hemisphere: O1 and P3; r ight hemisphere: O2 and P8). For trials rejected during preprocessing because of EEG artifacts, we used the mean alpha power calculated within the same condition as substitutes, to allow for regression with trial by trial BOLD activity in subsequent EEG in formed fMRI analysis (Novitskiy et al., 2011). To measure alpha asymmetry, a single trial alpha hemispheric lateralization index was defined as (Thut et al., 2006) :


76 (4 1) The above lateralization index measures the percentage difference between single trial alpha amplitudes from ROIs ipsilateral and contralateral to the attended visual field. The index is positive in general with larger values indicating stronger lateralization of alpha due to attention. 4.2.6 EEG informed fMR I Analysis The coupling between alpha and BOLD was examined using general linear models (GLMs). In total, seven task related regressors were included in the GLM: Two regressors separately modeled BOLD activities related to leftward and rightward cues with correct responses, two additional regressors modeled BOLD responses to target stimuli appearing on the left and right visual fields, a fifth regressor was added to model lso modeled but not analyzed in the current study. Coupling effects between alpha attentional modulation and BOLD was introduced into the GLM by adding parametric modulations on regressors modeling the cues. Three separate GLMs were constructed for this pu rpose. The first two GLMs each included a single set of parametric modulations on the cue regressors to model the coupling effects from contralateral and ipsilateral alpha, respectively. In this case, new regressors were generated in the attend left and ri ght conditions with the height of the stick functions modeling each trial scaled by mean removed single trial alpha power sampled from the contralateral or ipsilateral ROIs. To model the interaction between BOLD and the degree of alpha hemispheric laterali zation, a single parametric modulation from the trial


77 to trial alpha lateralization index was introduced to the third GLM on regressors modeling the cues. All task relevant regressors were convolved with a canonical hemodynamic response function (HRF) to a llow for comparisons with the recorded BOLD signal. Six movement related regressors were further incorporated into the design matrix to regress out residual signal variance from head movement. At the individual subject level, the coupling between BOLD and alpha modulation was assessed by examining, via t contrasts, the significance of the coefficients related to the regressors with alpha power modulations. At the group level systematic alpha BOLD coupling was assessed via a second level random effects anal ysis using a one sample t test. The group level cue evoked fMRI activations were thresholded at p < 0.05 corrected for multiple comparisons by controlling the false discovery rate (FDR). For alpha BOLD coupling analysis, as it detects linear dependency bet ween alpha and the residual fluctuation in BOLD on top of the task evoked effects, the group level statistical parametric maps were thresholded at p < 0.001, uncorrected. A cluster level threshold of k = 5 voxels was further imposed. The selection of the s tatistical threshold for group level random effects analysis was in line with recent EEG informed fMRI analysis ( Laufs et al., 2003; Debener et al. 2005; Eichele et al., 2005; Scheeringa et al., 2011; Liu et al., 2012 a ). 4.3 Results Thirteen subjects perfo rmed the task according to instructions. The mean accuracy rates, defined as the ratio between the number of correctly performed trials and the total number of trials, were 85.4% and 86.0% for attend left and attend right conditions, respectively.


78 4.3.1 A ttentional Modulation of Alpha For the analysis window chosen (500 1000 ms), posterior alpha on both hemispheres was lower than the pre cue baseline (500 ms) (Figure 4 2B), with stronger decrease in alpha power over the scalp region contralateral to the attended hemifield (Figure 4 2A; left hemisphere: t(12) = 2.1462, p < 0.05; right hemisphere: t(12) = 1.9145, p < 0.05). Topographically, the difference in the degree of alpha suppression between two hemispheres gave rise to the hemispheric lateralizatio n pattern seen in Figure 4 2C, in which alpha power from the attend right condition was subtracted from the attend left condition. 4.3.2 BOLD Activations Evoked by the Cue The cue evoked significant BOLD activations within the dorsal attention network, inc luding bilateral frontal eye fields (FEF), intraparietal sulci (IPS), and regions within the superior parietal lobule (SPL) (Figure 4 2D). This activation pattern, in conjunction with the post cue lateralization of alpha, indicated that participants proper ly allocated their covert attention according to instructions. Other regions activated during the anticipatory period included the supplementary motor area (SMA), precuneus, and regions in the occipital lobe near area MT+ (Table 4 1). 4.3.3 BOLD Alpha Coup ling: Negative Correlations Combining attend left and attend right conditions, the contralateral alpha was negatively correlated with BOLD in bilateral IPS, left middle frontal gyrus (MFG), left ventral occipital cortex (VO), and right inferior and middle temporal gyrus (IT/MTG). Relaxing the statistical threshold to p < 0.005, uncorrected, regions in the calcarine sulcus (CaS), bilateral VO, and bilateral crus II regions of the cerebellum further showed


79 negative correlation with contralateral alpha (Figure 4 3A; Table 4 2). For ipsilateral alpha fewer regions showed negative BOLD alpha correlation (Figure 4 3B; Table 4 2). In addition, the activated voxels in the left and right IPS clusters were 42 and 4 for ipsilateral alpha, compared to 80 and 106 for con tralateral alpha, suggesting that the coupling between IPS and ipsilateral alpha is weaker. It is worth noting that no coupling was observed between alpha and FEF. 4.3.4 BOLD Alpha Coupling: Positive Correlations For contralateral alpha, areas showing po sitive BOLD alpha correlation included regions within the post central gyrus (postCG) and anterior middle temporal gyrus (MTG), whereas for ipsilateral alpha, BOLD in medial prefrontal cortex (MPFC) and adjacent cortices in the superior frontal gyrus (SFG) showed positive correlation (Figure 4 4A, B; Table 4 2). These regions were located primarily in the sensorimotor cortices or in the default mode network. 4.3.5 Coupling between Alpha Lateralization Index and BOLD Analysis above focuses on alpha in indivi dual hemispheres. The degree of alpha lateralization, defined as ipsilateral alpha minus contralateral alpha, assesses the alpha difference between the two hemispheres and is an important indicator of attention control with greater alpha lateralization sig nifying more efficient attention control (Thut et by trial alpha lateralization index with BOLD activity, we found positive correlation between alpha lateralization index and BOLD activity in regions w ithin the dorsal anterior cingulate cortex (dACC) as well as adjacent areas of MPFC and superior frontal gyrus (SFG) (Figure 4 5; Table 4 2). These regions, especially dACC have been hypothesized to be part of a core or task


80 control network responsible for maintaining executive control over the ongoing task (Dosenbach et al., 2006) No region was found to be negatively coupled with the alpha lateralization index. 4.4 Discussion Top down attentional control enhances the processing of attended stimuli by bias ing the sensory cortices before stimulus onset. The lateralization of alpha oscillations is a manifestation of this biasing action in both visual (Worden et al., 2000; Sauseng et al., 2005; Thut et al., 2006) and somatosensory domains (Anderson and Ding, 2 011; Haegens et al., 2011). The present work examined the brain structures that contribute to the modulation of visual alpha oscillations during anticipatory attention by recording simultaneous EEG fMRI from human subjects performing a cued spatial visual attention task. Correlating hemispheric alpha power and cross hemisphere alpha lateralization with concurrently recorded BOLD, we showed that (1) alpha decrease was mainly associated with BOLD increases in bilateral IPS and visual areas, (2) alpha decrease was also associated with BOLD decreases in the sensorimotor cortices and the default mode network, and (3) the degree of alpha lateralization was positively coupled with BOLD in dACC. 4.4.1 Alpha and Dorsal Attention Network During spatial visual attent ion, alpha is generally decreased over posterior scalp regions contralateral to the direction of attention, reflecting increased visual cortical excitability and a readiness to process sensory input ( Sauseng et al., 2005; Thut et al., 2006; Grent Jong e t al., 20 11; Rajagovindan and Ding, 2011 ). One putative source of attentional modulation of alpha is the dorsal attention network, which is hypothesized to


81 generate and maintain a top down expectation signal to selectively bias visual cortical activity (Co rbetta and Shulman, 2002) Here, we provided further evidence showing that, along with visual cortices, increased BOLD in bilateral IPS was found to be coupled with desynchronized alpha on both ipsilateral and contralateral hemispheres during the anticipat ory period. Our result, obtained nonperturbatively, is consistent with two recent studies employing rTMS showing disrupted posterior alpha desynchronization following interference of preparatory activities in FEF and IPS, core regions within the dorsal att ention network (Capotosto et al., 2009, 2012) Whether successful attention allocation is largely achieved by an enhancement of task relevant visual cortices or a suppression of task irrelevant areas is still debated. Evidence in terms of anticipatory al pha is mixed with some studies observing decreased alpha contralateral to the attended location (Sauseng et al., 2005; Thut et al., 2006) while others document primarily an alpha increase contralateral to the unattended location (Worden et al., 2000; Yamag ishi et al., 2003; Kelly et al., 2006). In the present study, we observed alpha decreases on both hemispheres with respect to the baseline, with stronger decrease in the contralateral hemisphere. The stronger negative coupling between contralateral alpha a nd BOLD in IPS, compared to ipsilateral alpha, appears to suggest that top down attentional mechanisms operated mainly by enhancing neuronal activities within task relevant visual cortices (Corbetta and Shulman, 2002). However, unlike the paradigm used in the present study, alpha increases over areas contralateral to the unattended locations are often observed in tasks demanding active suppression of distractors at unattended locations (Yamagishi et al., 2003; Kelly et al., 2006). Therefore, a plausible exp lanation for the discrepancy is that the control of alpha is


82 mediated by the task demand and stimulus content at the to be ignored location with different degrees of task difficulty engaging different levels of alpha modulation on each hemisphere (Kelly et al., 2006). 4.4.2 Alpha and Task Irrelevant Networks Posterior alpha in the current study is found to be positively correlated with BOLD in sensorimotor cortices as well as regions within the default mode network, meaning that elevated visual cortical exc itability, as indicated by decreased posterior alpha, is accompanied by decreased activities within task irrelevant cortices in other sensory or cognitive modalities. This positive coupling suggests that attention to the visual domain disengages networks i n other task irrelevant domains to protect the ongoing visual task. sensory modalities (Klimesch, 2007; Jensen and Mazaheri, 2010). For example, visual alpha is found to b e increased when attention is directed to the somatosensory ( Haegens et al., 2010; Anderson and Ding, 2011) or the auditory domains (Foxe et al., 1998; Fu et al., 2001; Bollimunta et al., 2008) Higher levels of alpha activity within visual cortices were a ssociated with enhanced performance toward auditory stimuli (Bollimunta et al., 2008). The current study extends this mechanism to include the default mode network which is known to mediate non sensory self referential processes (Buckner et al., 2008). Not e that even in resting state, during which external task level is kept at the minimum, a positive coupling between spontaneous visual alpha fluctuations and BOLD in the default network was reported, demonstrating an intrinsic dynamic interaction between di fferential cortical systems ( Mayhew et al., 2013; Mo et al., 2013).


83 4.4.3 Differential Roles of IPS and FEF in Controlling Alpha A somewhat surprising finding is that alpha power was not coupled with BOLD in FEF, suggesting differential roles of FEF and I PS in modulating visual alpha. This finding is consistent with a recent dynamic causal modeling study showing direct modulation o f the visual cortex by IPS instead of FEF ( Vossel et al., 2012) Further, rTMS on IPS, but not FEF, has been shown to induce a paradoxical increase in alpha during anticipatory attention (Capotosto et al., 2009) Taken together, the coupling between IPS an d visual alpha found in our study might suggest that IPS, in contrast to FEF, engages in directly modulating visual cortical excitability. Although a host of studies have reported involvement of FEF in modulating visual cortices during attention ( Ruff et a l., 2006, 2008; Bressler et al ., 2008; Capotosto et al., 2009 ) it is possible that such involvement reflects an indirect engagement of FEF in modulating visual cortices, through changes in inter regional EEG synchrony rather than regional alpha desynchron ization (Sauseng et al., 2005, 2011). 4.4.4 The Role of dACC in Anticipatory Visual Attention When comparing two hemispheres, the degree of alpha lateralization is positively coupled with BOLD activity in dACC, and the coupling is not spatially selective, in that higher activity in dACC is associated with increased alpha lateralization regardless of the direction of attention. To date, the exact role of dACC in visual anticipatory attention is largely unknown, despite prior studies documenting its involveme nt in tasks requiring voluntary attentional orienting (Shulman et al., 2003; Fan et al., 2007; Aarts et al., 2008;


84 maintaining a global task set to mediate goal directed behavior ( Dosenbach et al., 2006, 2007, 2008 ; Corbetta et al., 2008; Sakai, 2008) This network, containing dACC and bilateral anterior insula (aI), is hypothesized to inde pendently send out top down signals to other domain specific executive areas to ensure the proper allocation of resources to support various task specific behaviors ( Shulman et al., 2003; Crottaz Herbette and Menon, 2006; Dosenbach et al., 2006; Walsh et a l., 2010; Wen e t al., 2012 ) Within the domain of attentional control, although alpha on each individual hemisphere might show divergent patterns of synchronization and desynchronization, studies have proposed that the global allocation of attentional reso urces is reflected in the lateralization of correlation between BOLD in dACC and alpha lateralization found in the current study is key evidence indicating that dACC m anticipation and mediates attentional deployment to facilitate overall task performance. 4.4.5 Summary The current study contributes to our understanding of the top down mechanisms of attention by providing nonp erturbative evidence demonstrating the involvement of frontoparietal attention and executive regions in modulating posterior alpha oscillations during anticipatory attention. It also distinguishes the differential roles of parietal and frontal regions in m odulating posterior alpha and the topological organization of the top down mechanisms. By identifying regions positively coupled with posterior alpha, this study further provides evidence suggesting a mechanism of active inhibition over task irrelevant sen sory and cognitive modalities (Klimesch et al., 2007) Finally, by combining imaging modalities, a role of dorsal ACC in anticipatory attentional control is suggested.


85 Anatomical Regions Hemisphere M NI Coordinates (x,y,z) Z score Frontal Eye Field Left 24, 3, 57 3.56 Right 39, 0, 57 4.15 Intraparietal Sulcus Left 33, 48, 42 4.10 Right 27, 57, 51 5.13 MT+ Left 48, 69, 15 5.25 Right 51, 72, 12 5.24 Precuneus Left 6, 57, 57 3.78 R ight 6, 54, 54 3.98 Supplementary Motor Area Left 9, 12, 51 3.16


86 Anatomical Regions Hemisphere MNI Coordinates (x,y,z) Z score Negative coupling between BOLD and contral ateral alpha Intraparietal Sulcus Left 33, 69, 57 4.81 Right 39, 51, 54 4.33 Inferior and Middle Temporal Gyrus Right 60, 33, 15 4.21 Middle Frontal Gyrus Left 39, 42, 33 3.71 Ventral Occipital Cortex Left 35, 81, 17 4.17 Right 42, 84, 15 3.25* Calcarine Sulcus 9, 87, 0 3.65* Crus II of Cerebellum Left 12, 81, 39 4.35 Right 3, 81, 33 2.88* Negative coupling between BOLD and ipsilateral alpha Intraparietal Sulcus Left 30, 63, 39 4.01 Right 36, 63, 51 3.42 Crus II o f Cerebellum Left 21, 75, 42 3.41 Positive coupling between BOLD and contralateral alpha Post central Gyrus Left 48, 12, 24 4.59 Right 60, 12, 39 4.48 Middle Temporal Gyrus Left 57, 9, 18 3.67 Positive coupling between BOLD and ips ilateral alpha Medial Prefrontal Cortex Right 15, 48, 33 4.47 Positive coupling between BOLD and alpha lateralization index Dorsal Anterior Cingulate Cortex Right 6, 24, 39 3.45 Medial Prefrontal Cortex Right 15, 51, 27 3.92 Superior Frontal Gyru s Left 21, 36, 39 3.69 *: p < 0.005.


87 Figure 4 1. An illustration of the sequence of events within a trial. Following cue onset, participants covertly directed their attention toward either left of right hemifield while maintaining eye fixation on th e central point. Targets were flashed briefly at one of the marked locations after a variable inter stimulus interval. Participants were required to discriminate the thickness of the stripes and make a forced 2 button choice only when targets appeared on t he attended location. At the end of each trial, participants were also prompted to report the hemifield they attended to.


88 Figure 4 2. Attentional modulation of alpha power and BOLD. A) Grand average power spectral density from occipital channels (O1 an d O2) showing modulation in the alpha frequency band (8 12 Hz) during 500 1000 ms after cue onset. B) Scalp topography showing alpha desynchronization during Attend Left and Attend Right, respectively. C) Difference topography between Attend Left and A ttend Right showing alpha asymmetry during covert shifting of attention. D) BOLD activation showing the engagement of dorsal attention network during the post cue anticipatory period. Activation map is plotted according to the neurological convention with left shown on the left side. FEF: frontal eye field; IPS: intraparietal sulcus; SPL: superior parietal lobule; SMA: supplementary motor area.


89 Figure 4 3. Negative coupling between BOLD and alpha with Attend Left and Right combined A) Regions exhibit ing negative coupling with contralateral alpha during voluntary attention (top row and middle tow correspond to p < 0.001 and p<0.005, uncorrected, respectively). B) Regions showing negative coupling between BOLD and ipsilateral alpha (p < 0.001, uncorrect ed). IPS: intraparietal sulcus; MFG: middle frontal gyrus; IT: inferotemporal gyrus; MTG: middle temporal gyrus; VO: ventral occipital cortex; CaS: calcarine sulcus; Crus II: crus II of cerebellum.


90 Figure 4 4. Positive coupling between BOLD and A) con tralateral alpha and B) ipsilateral alpha with Attend Left and Right combined. The statistical parametric maps are thresholded at p < 0.001, uncorrected. MPFC: medial prefrontal cortex; MTG: middle temporal gyrus; postCG: post central gyrus; SFG: superior frontal gyrus. Figure 4 5. Coupling between BOLD and alpha lateralization index. A) Sagittal slices showing a region in dorsal anterior cingulate cortex (dACC) which is positively correlated with alpha lateralization. B) Coronal slices showing the s ame region in dACC along with adjacent regions in superior frontal sulcus (SFS) and medial prefrontal cortex (MPFC).


91 CHAPTER 5 CONCLUSION S Neurophysiological signals contain a substantial amount of variability across multiple repeated experimental trials Traditionally, researchers treat such trial to trial variability as task irrelevant noises and the corresponding ensemble averaging procedure tends to average out such variability. However, r ecent studies have shown that the trial to trial variability be ars important information about the dynamics of the underlying cognitive processes (Chen et al. 2006) The simple ensemble averaging procedure cannot resolve such detailed information Therefore, s ingle trial estimation techniques provide a promising tool to analyzing the trial to trial variability inherent in neurophysiological signals. This dissertation employed a recently developed single trial analysis algorithm, ASEO (Xu et al. 2009), to study the functional implications associated with the trial to trial variability in both the EEG and BOLD domain along three studies. In the first study, we sought to study the trial by trial dynamics of the senso ry facilitation process during classical aversive conditioning We for the first time characterized the d etailed temporal dynamics of the learning process within a conditioning block by estimating and comparing the trial by trial evolving patterns of single trial P1 amplitudes between CS+ and CS Specifically, three distinct phases of P1 modulation as a func tion of experimental trials were identified as conditioning progresses, with P1 amplitude showing different ial modulating patterns between CS+ and CS within each phase. In contrast to a prior study employing traditional ERP measures that reported no condi tioning related modulation of P1 (Stolarova et al., 2006) t he differ ence in temporal dynamics of P1 amplitudes found in this study between the


92 CSs was a strong indication that conditioning modulates neuronal activities in the extrastriate cortex, albeit i n a complex way. Moreover, the fact that P1 amplitudes showed a complex trial to trial temporal dynamics in the conditioning block suggests that prior emotional experience acts to increase both the reactivity and efficiency of sensory cortices, possibly in a sequential way with the increase phase reflecting increased reactivity and the final habituating phase reflecting enhanced network efficiency. Such temporal dynamics of sensory modulation is further in line with the prediction error theory of attention during conditioning (Pearce and Hall, 1980) which the level of uncertainty decreases leading to facilitation of increasingly early stages of visual processing as learning progresses (Keil, 2 004). Given that the extrastriate cortex receives re entrant projections from the amygdala (Amaral and Price, 1984), it is likely that the preferential sensory sensitization toward CS+ originated from such re entrant modulations from the amygdala, which ha s also been found to exhibit habituation after repeated CS presentations ( Bchel and Dolan, 2000 ). In the second study, we incorporated ASEO single trial ERP estimation with EEG informed fMRI analysis to investigate brain structures contributing to the sca lp recorded LPP. Areas contributing to either the generation or modulation of LPP were identified by testing for significant correlations between single trial LPP amplitudes with concurrently measured single trial BOLD activities throughout the entire brai n. We found that areas positively correlated with LPP amplitude consisted of both the visual cortex and other areas known to be involved in emotional processing, such as the orbitofrontal cortex, insula, and amygdala. This suggests that emotional processin g naturally attracts


93 processing resources and activates an extensive network reflecting the engagement of the fundamental motivation system. Restricting our analyses into positive and negative categories of emotions, we found categorical specific differenc es in regions showing LPP and BOLD coupling, which indicates that the underlying structures giving rise to the LPP differ across emotion categories despite they all elicit the LPP with similar temporal dynamics and scalp topography. It is worth noting tha t prior studies have attempted to locate the potential generators of the LPP using EEG source localizing techniques (Keil et al., 2002; Sabatinelli et al., 2007b). Yet these studies were only partly successful because EEG source localizing technique s in ge neral have relatively low spatial resolution and more importantly, it could not resolve information coming from distant subcortical structures which are usually critical in emotional processing. Recent studies have also examined this problem by correlating LPP with fMRI acquired from a separate experimental session in a cross subjects fashion (Sabatinelli et al., 2007b, 2013). One potential drawback of such an approach is that it is difficult to control the psychological and biological conditions to be exac tly the same in different recording session. In addition, the correlation between the average LPP across subjects and BOLD does not reflect their trial by trial co variations in a within subjects fashion. Given the above, the single trial estimation combin ed with simultaneous EEG fMRI employed in this study was capable of overcoming these limitations and was proven useful in revealing brain areas contributing to scalp recorded EEG features. Finally in the third study, we identified cortical areas contribut ing to two aspects of the trial to trial attentional modulation of posterior alpha desynchronization and


94 hemispheric lateralization, via the EEG informed fMRI analysis similar to that employed in the second study. Main regions modulating posterior alpha d esynchronization during attention was found to include bilateral intraparietal sulci (IPS) and the left middle frontal gyrus (MFG) core regions within the attention system. Yet interestingly, the frontal eye field, also being part of the dorsal attention network, was not found to contribute to alpha desynchronization suggesting differential roles of frontal and parietal regions in modulating posterior alpha. The fact that a stronger negative coupling was found between BOLD and contralateral alpha, compare d to ipsilateral alpha, suggests that top down attentional mechanisms operated mainly by enhancing neuronal activities within task relevant visual cortices. On the other hand, regions within the sensorimotor cortices and the default mode network showed pos itive coupling with alpha, suggesting a mechanism of active inhibition over task irrelevant networks. Last but not least, the alpha hemispheric lateralization was correlated with BOLD activity in the dorsal anterior cingulate cortex (dACC) for both attend left and right conditions, providing key evidence indicating attentional set to facilitate overall deployment of attentional resources among task relevant and irrelevant cortices


95 LIST OF REFERENCES Aarts E, Roel ofs A, van Turennout M (2008) Anticipatory activity in anterior cingulate cortex can be independent of conflict and error likelihood. J Neurosci, 28:4671 4678. Adolphs R (2002) Neural systems for recognizing emotion. Curr Opin Neurobiol 12:169 177. Aharon have variable reward value: fMRI and behavioral evidence. Neuron 32:537 551. Allen PJ, Josephs O, Turner R (2000) A method for removing imaging artifact from continuous EEG recorded during functional MRI. Neuroimage 12:230 239. Allen PJ, Polizzi G, Krakow K, Fish DR, Lemieux L (1998) Identification of EEG events in the MR sc anner: the problem of pulse artifact and a method for its subtraction. Neuroimage 8:229 239. Alves NT, Fu kusima SS, Aznar Casanova A (2008) Models of brain asymmetry in emotional processing. Psychol Neurosci 1:63 66. Amaral DG, Price JL (1984) Amygdalo cortical projections in the monkey (macaca fascicularis). J Comp Neurol 230:465 496. Amir N, Beard C, Cobb M Bomyea J (2009) Attention modification program in individuals with generalized anxiety disorder. J Abnorm Psychol 118:28 33. Anderson KL, Ding M (2011) Attentional modulation of the somatosensory mu rhythm. Neuroscience 180:165 180. Anderson KL, Rajagovi ndan R, Ghacibeh GA, Meador KJ, Ding M (2010) Theta oscillations mediate interaction between prefrontal cortex and medial temporal lobe in human memory. Cereb Cortex 20:1604 1612. Astafiev SV, Shulman GL, Stanley CM, Snyder AZ, Essen DCV, Corbetta M (2003) Functional organization of human intraparietal and frontal cortex for attending, looking, and pointing. J Neurosci 23:4689 4699. Bartnik EA, Blinowska KJ, Durka PJ (1992) Single evoked potential reconstruction by means of wavelet transform. Biol Cybern 67 :175 181. Belouchrani A, Abed Meraim K, Cardoso JF, Moulines E (1993) Second order blind separation of temporally correlated sources. Proc Int Conf on Digital Sig Proc, pp. 346 351, Cyprus.


96 Bnar C, Schn D, Grimault S, Nazarian B, Burle B, Roth M, Badier J, Marquis P, Liegeois Chauvel C, Anton J (2007) Single trial analysis of oddball event related potentials in simultaneous EEG fMRI. Hum Brain Mapp 28:602 613. Bollimunta A, Chen Y, Schroeder CE, Ding M (2008) Neuronal mechanisms of cortical alpha oscillat ions in awake behaving macaques. J Neurosci 28:9976 9988. Bradley MM (2009) Natural selective attention: orienting and emotion. Psychophysiology 46:1 11. Bradley MM, Lang PJ (1994) Measuring emotion: the self assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25:49 59. Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, Kin g W, Desai P (2003) Activation of the visual cortex in motivated attention. Behav Neurosci 117:369 380. Breiter H, Etcoff N, Whalen P, Kennedy W, Rauch S, Buckn er R, Strauss M, Hyman S, Rosen B (1996) Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17:875 887. Bressler SL, Tang W, Sylvester CM, Shulman GL, Corbetta M (2008) Top down control of human visual cort ex by frontal and parietal cortex in anticipatory visual spatial attention. J Neurosci 28:10056 10061. Buckner RL, Andrews Annals of the New York Academy of Science, 1124:1 38. Bche l C, Coull JT, Friston KJ (1999 a ) The predictive value of changes in effective connectivity for human learning. Science 283:1538 1541. Bchel C, Dolan RJ, Armony JL, Friston KJ ( 1999 b ) Amygdala hippocampal involvement in human aversive trace conditioning revealed thro ugh event related functional magnetic resonance imaging. J Neurosci 19:10869 10876. Bchel C, Dolan RJ (2000) Classi cal fear conditioning in functional neuroimaging. Curr Opin Neurobiol 10:219 223. Bchel C, Morris J, Dolan RJ, Friston KJ (1998) Brain systems mediating aversive conditioning: an event related fMRI study. Neuron 20:947 957. Cacioppo JT, Crites SL Jr, Berntson GG, Coles MGH (1993) If attitudes affect how stimuli are processed, should they not affect the event related brain potential? Psychol Sci 4:108 112. Cacioppo JT, Crites SL Jr, Gardner WL, Bernston GG (1994) Bioelectrical echoes from evaluative categorizations: I. A late positive brain potential that varies as a function of trait negativity and extremity. J Pers Soc Psychol 67:115 125.


97 Capoto sto P, Babiloni C, Romani GL, Corbetta M (2009) Frontoparietal cortex controls spatial attention through modulation of anticipatory alpha rhythms. J Neurosci 29:5863 5872. Capotosto P, Babiloni C, Romani GL, Corbetta M (2012) Differential contribution of r ight and left parietal cortex to the control of spatial attention: a simultaneous EEG rTMS study. Cereb Cortex 22:446 454. Chen Y, Bressler SL, Knuth KH, Truccolo WA, Ding M (2006) Stochastic modeling of neurobiological time series: power, coherence, Grang er causality, and separation of evoked responses from ongoing activity. Chaos 16, 026113. Chen Y, Dhamala M, Bollimunta A, Schroeder CE, Ding M (2011) Current source density analysis of ongoing neural activity: theory and application. In: Electrophysiologi cal recording techniques, neuromethods (Vertes RP and Stackman RW, eds), pp 27 40, Totowa, NJ: Humana. Coppola R, Tabor R, Buchsbaum MS (1978) Signal to noise ratio and response variability measurements in single trial evoked potentials. Electroencephalogr Clin Neurophysiol 44:214 222. Corbetta M, Kincade JM, Ollinger JM, McAvoy MP, Shulman GL (2000) Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nat Neurosci 3:292 297. Corbetta M, Shulman GL (2002) Control of g oal directed and stimulus driven attention in the brain. Nat Rev Neurosci 3:201 215. Corbetta M, Patel G, Shulman GL (2008) The reorienting system of the human brain: from environment to theory of mind. Neuron 58:306 324. Costa VD, Lang PJ, Sabatinelli D, Versace F, Bradley MM (2010) Emotional imagery: 1446 1457. Craig AD (2009) How do you feel now? The anterior insula and human awareness. Nat Rev Neurosci 10:59 70. Crottaz H erbette S, Menon V (2006) Where and when the anterior cingulate cortex modulates attentional response: combined fMRI and ERP evidence. J Cogn Neurosci 18:766 780. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ (2000) Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biol Psychol 52:95 111.


98 David SP, Munaf MR, Johansen Berg H, Smith SM, Rogers RD, Matthews PM, Walton RT (2005) Ventral striatum/nucleus accumbens activation to smoking related pictorial cues in smokers and nonsmokers: a functional magnetic resonance imaging study. Biol Psychiatry 58: 488 494. Debener S, Ullsperger M, Siegel M, Engel AK (2007) Towards single trial analysis in cognitive brain research. Trends Cogn Sci 11:502 503. Debener S, Ullsperger M, Siegel M, Fiehler K, von Cramon DY, Engel AK (2005) Trial by trial coupling of concurrent electroencephalogram an d functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci 25:11730 11737. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single trial EEG dynamics including independent component analysis. J Neurosci Methods 134:9 21. Diamond DM, Weinberger NM (1984): Physiological plasticity of single neurons in audi tory cortex of the cat during acquisition of the pupillary conditioned response: II. Secondary field (AII). Behav Neurosci 98:189 210. DiRusso F, Martnez A, Sereno MI, Pitzalis S, Hillyard SA (2001) Cortical sources of the early components of the visual e voked potential. Hum Brain Mapp 15:95 111. Dosenbach NUF, Fai r DA, Cohen AL, Schlaggar BL, Petersen SE (2008) A dual networks architecture of top down control. Trends Cogn Sci 12:99 105. Dosenbach NUF, Fa ir DA, Miezin FM, Cohen AL, Wenger KK, Dosenbach RAT, Fox MD, Snyder AZ, Vincent JL, Raichle ME, Schlaggar BL, Petersen SE (2007) Distinct brain networks for adaptive and stable task control in humans. Proc Natl Acad Sci 104:11073 11078. Dosenbach NUF, Visscher KM, Palmer ED, Miezin FM, Wenger KK, Kang HC, Burgund ED, Grimes AL, Schlaggar BL, Petersen SE (2006) A core system for the implementation of task sets. Neuron 50:799 812. Eichele T, Specht K, Moosmann M, Jongsma MLA, Quiroga RQ, Nordby H, Hugdahl K (2005) Assessing the spatiotemporal evolution of neu ronal activation with single trial event related potentials and functional MRI. Proc Natl Acad Sci USA 102:17798 17803. Elbert T, Heim S (2001) A light and a dark side. Nature 411:139. Fan J, Gu X, Liu X, Guise KG, Park Y, Martin L, de Marchena A, Tang CY, Minzenberg MJ, Hof PR (2011) I nvolvement of the anterior cingulate and frontoinsular cortices in rapid processing of salient facial emotional information. Neuroimage 54:2539 2546.


99 Fan J, Kolster R, Ghajar J, Suh M, Knight RT, Sarkar R, McCandliss, BD (2007) Response anticipation and response conflict: an event related potential and functional magnetic resonance imaging study. J Neurosci 27:2272 2282. Ferretti A, Caulo M, Del Gratta C, Di Matteo R, Merla A, Montorsi F, Pizzella V, Pompa P, Rigatti P, Ross ini PM, Salonia A, Tartaro A, Romani GL (2005) Dynamics of male sexual arousal: distinct components of brain activation revealed by fMRI. NeuroImage 26:1086 1096. Ferry AT, ngr D, An X, Price JL (2000) Prefrontal cortical projections to the striatum in m acaque monkeys: Evidence for an organization related to prefrontal networks. J Comp Neurol 425:447 470. Fogelson N, Wang X, Lewis JB, Kishiyama MM, Ding M, Knight RT (2009) Multimodal effects of local context on target detection: evidence from P3b. J Cogn Neurosci 21:1680 1692. Foti D, Olvet DM, Klein DN, Hajcak G (2010) Reduced electrocortical response to threatening faces in major depressive disorder. Depress Anxiety 27:813 820. Fox K, Wong RO (2005): A comparison of experience dependent plasticity in the visual and somatosensory systems. Neuron 48:465 477. Fox MD, Zhang D, Snyder AZ, Raichle ME (2009) The global signal and observed anticorrelated resting state brain networks. J Neurophysiol 101:3270 3283. Foxe JJ, Simpson GV, Ahlfors SP (1998) Parieto occipital approximately 10 Hz activity reflects anticipatory state of visual attention mechanisms. Neuroreport 9:3929 3933. Friedman JH (1991) Multivariate adaptive regression splines. Ann Stat 19:1 141. Fu, KM, Foxe JJ, Murray MM, Higgins BA, Javitt DC, S chroeder CE (2001) Attention dependent suppression of distracter visual input can be cross modally cued as indexed by anticipatory parieto occipital alpha band oscillations. Cogn Brain Res 12:145 152. Giesbrecht B, Woldorff MG, Song AW, Mangun GR (2003) Ne ural mechanisms of top down control during spatial and feature attention. NeuroImage 19:496 512. Grent Jong T, Boehler CN, Kenemans JL, Woldorff MG (2011) Differential Functional Roles of Slow Wave and Oscillatory Alpha Activity in Visual Sensory Cortex during Anticipatory Visual Spatial Attention. Cereb Cortex 21:2204 2216. Grill Spector K, Henson R, Martin A (2006) Repetition and the brain: neural models of stimulus specific effects. Trends Cogn Sci 10:14 23.

PAGE 100

100 Gruber T, Keil A, Mller MM (2001) Modulati on of induced gamma band responses and phase synchrony in a paired associate learning task in the human EEG. Neurosci Lett 316:29 32. Gu X, Liu X, Guise KG, Naidich TP, Hof PR, Fan J (2010) Functional dissociation of the frontoinsular and anterior cingulat e cortices in empathy for pain. J Neurosci 30:3739 3744. Guo C, Lawson AL, Jiang Y (2007) Distinct neural mechanisms for repetition effects of visual objects. Neuroscience 149:747 759. Guo C, Lawson AL, Zhang Q, Jiang Y (2008) Brain potentials distinguish new and studied objects during working memory. Hum Brain Mapp 29:441 452. Haegens S, Hndel BF, Jensen O (2011) Top down controlled alpha band activity in somatosensory areas determines behavioral performance in a discrimination task. J Neurosci 31: 5197 5204. Haegens S, Osipova D, Oostenveld R, Jensen O (2010) Somatosensory working memory performance in humans depends on both engagement and disengagement of regions in a distributed netwo rk. Hum Brain Mapp 31:26 35. Hamann SB, Ely TD, Hoffman JM, Kilts CD (2002) Ecstasy and Agony: activation of the human amygdala in positive and negative emotion. Psychol Sci 13:135 141. Hndel BE, Haarmeier T, Jensen O (2011) Alpha oscillations correlate with the successful inhibition of unattended stimuli. J Cogn Neurosci 23:2494 2502. Hendler T, Rotshtein P, Yeshurun Y, Weizmann T, Kahn I, Ben Bashat D, Malach R, Bleich A (2003) Sensing the invisible: differential sensitivity of visual cortex and amygdala to traumatic context. Neuroimage 19:587 600. Hopfinger JB, Buonocor e MH, Mangun GR (2000) The neural mechanisms of top down attentional control. Nat Neurosci 3:284 291. Ishai A, Pessoa L, Bikle PC, Ungerleider LG (2004) Repetition suppression of faces is modulated by emotion. Proc Natl Acad Sci USA 101:9827 9832. Jaskowsk i P, Verleger R (1999) Amplitudes and latencies of single estimated by a maximum likelihood method. IEEE Trans Biomed Eng 46:987 993. Jaworska N, Thompson A, Shah D, Fisher D, Ilivitsky V, Knott V (2012) Acute tryptophan depletion effects on th e vertex and late positive potentials to emotional faces in individuals with a family history of depression. Neuropsychobiology 65:28 40. Jensen O, Mazaheri A (2010) Shaping functional architecture by oscillatory alpha activity: gating by inhibition. Front Hum Neurosci 4:186.

PAGE 101

101 Jiang Y, Haxby JV, Martin A, Ungerleider LG, Parasuraman R (2000) Complementary neural mechanisms for tracking items in human working memory. Science 287:643 646. Jiang Y, Lianekhammy J, Lawson A, Guo C, Lynam D, Joseph JE, Gold BT, Ke lly TH (2009) Brain responses to repeated visual experience among low and high sensation seekers: role of boredom susceptibility. Psychiatry Res 173:100 106. Junghfer M, Elber T, Tucker DM, Rockstroh B (2000) Statistical control of artifacts in dense arra y EEG/MEG studies. Psychophysiology 37:523 532. Karama S, Lecours AR, Leroux J, Bourgouin P, Beaudoin G, Joubert S, Beauregard M (2002) Areas of brain activation in males and females during viewing of erotic film excerpts. Hum Brain Mapp 16:1 13. Kastner S Pinsk MA, Weerd PD, Desimone R, Ungerleider LG (1999) Increased Activity in Human visual cortex during directed attention in the absence of visual stimulation. Neuron 22:751 761. Keil A (2004) The role of human prefrontal cortex in motivated perception and behavior: Evidence from macroscopic electrocortical signals. In: Otani S, editor. Prefrontal Cortex: From Syn aptic Plasticity to Cognition. New York: Kluwer. pp 245 267. Keil A, Bradley MM, Hauk O, Rockstroh B, Elbert T, Lang PJ (2002) Large scale neural correlates of affective picture processing. Psychophysiology 39, 641 649. Keil A, Costa V, Smith JC, Sabatinelli D, McGinnis EM, Bradley MM, Lang PJ (2012) Tagging cortical networks in emotion: a topographical analysis. Hum Brain Mapp 33:2920 2931. Keil A, Gruber T, Mller MM, Moratti S, Stolarova M, Bradley MM, Lang PJ (2003) Early modulation of visual perception by emotional arousal: evidence from steady state v isual evoked brain potentials. Cogn Affect Behav Neurosci 3:195 206. Keil A, Sabatinelli D, Ding M, Lang PJ, Ihssen N, Heim S (2009) Re entrant projections modulate visual cortex in affective perception: directional evidence from Granger causality analysis. Hum Brain Mapp 30:532 540. Keil A, Stolarova M, Moratti S, Ray WJ (2007) Adaptation in human visual cortex as a mechanism for rapid discrimination of aversive stimuli. NeuroImage 36:472 479. Kelly SP, Lalor EC, Reilly RB, Foxe JJ (2006) Increases in alpha oscillatory power reflect an active retinotopic mechanism for distracter suppression du ring sustained visuo spatial attention. J Neurophysiol 95:3844 3851. Kleinbaum DG, Kupper LL, Muller KE (1988) Applied Regression Analysis and Other Multivariable Methods, 2nd ed. Boston: PWS KENT Publishing Company.

PAGE 102

102 Klemm M, Haueisen J, Ivanova G (2009) Independent component analysis: comparison of algorithms for the investigation of surface electrical brain activity. Med Biol Eng Comput 47:413 423. Klimesch W, Sauseng P, Hanslmayr S (2007) EEG alpha oscillations: the inhibition timing hypothesis. Brain R es Rev 53:63 88. Knuth KH, Shah AS, Truccolo WA, Ding M, Bressler SL, Schroeder CE (2006) Differentially variable component analysis: identifying multiple evoked components using trial to trial variability. J Neurophysiol 95:3257 3276. Knutson B, Fong GW, Bennett SM, Adams CM, Hommer D (2003) A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event related fMRI. Neuroimage 18:263 272. Krolak Salmon P, Hnaff MA, Vighetto A, Bertrand O, Mauguire F (2004) E arly amygdala reaction to fear spreading in occipital, temporal, and frontal cortex: a depth electrode ERP study in human. Neuron 42:665 676. LaBar KS, Gatenby C, Gore JC, LeDoux JE (1998) Human amygdala activation during conditioned fear acquisition and e xtinction: a mixed trial fMRI study. Neuron 20:937 945. Lane RD, Reiman EM, Bradley MM, Lang PJ, Ahern GL, Davidson RJ, Schwartz GE (1997) Neuroanatomical correlates of pleasant and unpleasant emotion. Neuropsychologia 35:1437 1444. Lang PJ, Bradley, MM (2010) Emotion and the motivational brain. Bio l Psychol 84:437 450. Lang PJ, Bradley MM, Cu thbert BN (1997a) International affective picture system (IAPS): Technical manual and affective ratings. NIMH Center for the Study of Emotion and Attention, University of Florida. Lang PJ, Bradley MM, Cuthbert BN (1998a) Emotion, motivation, and anxiety: brain mechanisms and psychophysiology Biol Psychiatry 44:1248 1263. Lang PJ, Bradley MM, Cuthbert BN (2008) International affective picture system (IAPS): affective ratings of pictures and instructi on manual. University of Florida, Gainesv ille, FL: Technical Report A 8. Lang PJ, Bradley MM, Fitzsimmons JR, Cuthbert BN, Scott JD, Moul der B, Nangia V (1998b) Emotional arousal and activation of the visual cortex: an fMRI analysi s. Psychophysiology 35:199 210. Lang PJ, Simons RF, Balaban MT (1997b) Attention and orienting: sensory and motivational processes. Hillsdale, NJ: Erlbaum.

PAGE 103

103 Lange DH, Pratt H, Inbar GF (1997) Modeling and estimation of single evoked brain potential co mponents. IEEE Trans Biomed Eng 44:791 799. Laufs H, Holt JL, Elfont R, Krams M, Paul JS, Krakow K, Kleinschmidt A (2006) Where the BOLD signal goes when alpha EEG l eaves. Neuroimage 31:1408 1418. Laufs H, Kleinschmidt A, Beyerle A, Eger E, Salek Haddadi A, Preibisch C, Krakow K (2003a) EEG correlated fMRI of human alpha activity. Neuroimage 19:1463 1476. Laufs H, Krakow K, Sterzer P, Eger E, Beyerle A, Sa lek Haddadi A, Kleinschmidt A (2003b) Electroencephalographic signatures of attentional and cognitive default modes in spontaneous brain activity fluctuations at rest. Proc Natl Acad Sci 100:11053 11058. LeDoux J (2003) The emotional brain, fear, and the amygdala. Cell Mol Neurobiol 23:727 738. Leutgeb V, Schfer A, Schienle A (2011) Late cortical positivit y and cardiac responsitivity in female dental phobics when exposed to phobia relevant pictures. Int J Psychophysiol 79:410 416. Lissek S, Levenson J, Biggs AL, Johnson LL, Ameli R, Pine DS, Grillon C (2008) Elevated fear conditioning to socially relevant u nconditioned stimuli in social anxiety disorder. Am J Psychiatry 165:124 132. Liu Y, Huang H, McGinnis Deweese M, Keil A, Ding M (2012 a ) Neural substrate of the late positive potential in emotional processing. J Neurosci 32:14563 14572. Liu Y, Keil A, Ding M (2012 b ) Effects of emotional conditioning on early visual processing: Temporal dynamics revealed by ERP single trial analysis. Hum Brain Mapp 33:909 919. Liu Y, Murray SO, Jagadeesh B (2009) Time course and stimulus dependence of repetition induced response suppression in inferotemporal cortex. J Neurophysiol 101:418 436. Luck SJ (2005) An introduction to the even t related potential technique. Cambridge, MA: MIT Press. Luo Q, Holroyd T, Jones M, Hendler T, Blair J (2007) Neural dynamics for facial threat processing as revealed by gamma band synchronization using MEG. Neuroimage 34:839 847. Mayhew SD, Ostwald D, Porcaro C, Bagshaw AP (2013) Spontaneous EEG alpha oscillation interacts with positive and negative BOLD responses in the visual auditory cortices and default mode network. Neuroimage 76:362 372.

PAGE 104

104 McGillem CD, Aunon JI, Pomalaza CA (1985) Improved waveform estimation procedures for event related potentials IEEE Trans Biomed Eng BME 32, 371 379. McMahon DBT, Olson CR (2007) Repetition suppression in monkey inferotemporal cortex: relation to behavioral priming. J Neurophysiol 97:3532 3543. Mitzdorf U (1985) Current source density method and application in ca t cerebral cortex: investigation of evoked potentials and EEG phenomena. Physiol Rev 65:37 100. Mo J, Liu Y, Huang H, Ding M (2013) Coupling between visual alpha oscillations and default mode activity. Neuroimage 68:112 118. Mogg K, Millar N, Bradley BP (2 000) Biases in eye movements to threatening facial expressions in generalized anxiety disorder and depressive disorder. J Abnorm Psychol 109:695 704. Moosmann M, Ritter P, Krastel I, Brink A, Thees S, Blankenburg F, Taskin B, Obrig H, Villringer A (2003) C orrelates of alpha rhythm in functional magnetic resonance imaging and near infrared spectroscopy. Neuroimage 20:145 158. Moratti S, Keil A (2005) Cortical activation during Pavlovian fear conditioning depends on heart rate response patterns: an MEG study. Brain Res Cogn Brain Res 25:459 471. Morris JS, Friston KJ, Bchel C, Frith CD, Young AW, Calder AJ, Dolan RJ (1998 a ) A neuromodulatory role for the human amygdala in processing emotional facia l expressions. Brain 121:47 57. Morris JS, Friston KJ, Dolan RJ (1998 b ) Experience dependent modulation of tonotopic neural responses in human auditory cortex. Proc Biol Sci 265:649 657. Moratti S, Keil A (2009) Not what you expect: experience but not expectancy predicts conditioned responses in human visual and supp lementary cortex. Cereb Cortex 19:2803 2809. Mller MM, Andersen S, Keil A (2008) Time course of competition for visual processing resources between emotional pictures and a foreground task. Cereb Cortex 18:1892 1899. Nagai Y, Critchley H, Featherstone E, Fenwick PB, Trimble M, Dolan R (2004) Brain activity relating to the contingent negative variation: an fMRI investigation. Neuroimage 21:1232 1241. Norris CJ, Chen EE, Zhu DC, Small SL, Cacioppo JT (2004) The interaction of social and emotional processes i n the brain. J Cogn Neurosci 16:1818 1829.

PAGE 105

105 Novitskiy N, Ramautar JR, Vanderperren K, De Vos M, Mennes M, Mijovic B, Vanrumste B, Stiers P, Vanden Bergh B, Lagae L, Sunaert S, Van Huffel S, Wagemans J (2011) The BOLD correlates of the visual P1 and N1 in si ngle trial analysis of simultaneous EEG fMRI recordings during a spatial detection task. Neuroimage 54:824 835. pleasant and aversive taste in the human brain. J Neurophysiol 8 5:1315 1321. the role of medial orbitofrontal cortex in facial attractiveness. Neuropsychologia 41:147 155. Olofsson JK, Nordin S, Sequeira H, Polich J (2008) Affecti ve picture processing: an integrative review of ERP findings. Biol Psychol 77:247 265. Oya H, Kawasaki H, Howard MA, Adolphs R (2002) Electrophysiological responses in the human amygdala discriminate emotion categories of complex visual stimuli. J Neurosci 22:9502 9512. Palomba D, Angrilli A, Mini A (1997) Visual evoked potentials, heart rate responses and memory to emotional pictorial stimuli. Int J Psychophysiol 27:55 67. Pastor MC, Bradley MM, Lw A, Versace F, Molt J, Lang PJ (2008) Affective picture p erception: Emotion, context, and the late positive potential. Brain Res 1189:145 151. Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532 552. Pessoa L 783. Phan KL, Wager T, Taylor SF, Liberzon I (2002) Functional neuroanatomy of emotion: a meta analysis of emotion activation studies in PET and fMRI. Neuroimage 16:331 348. Phillips ML, Medford N, Young AW, Williams L, Williams SCR, Bullmore ET, Gray JA, Brammer MJ (2001) Time courses of left and right amygdalar responses to fearful facial expressi ons. Hum Brain Mapp 12: 193 202. Phillips ML, Young AW, Scott SK, Calder AJ, Andrew C, Giampietro V, Williams SCR, Bullmore ET, Brammer M, Gray JA (1998) Neural responses to facial and vocal expressions of fear and disgust. Proc Biol Sci 26 5:1809 1817.

PAGE 106

106 Phillips ML, Young AW, Senior C, Brammer M, Andrew C, Calder AJ, Bullmore ET, Perrett DI, Rowland D, Williams SCR, Gray JA, David AS (1997) A specific neural substrate for perceiving fa cial expressions of disgust. Nature 389:495 498. Pizzagalli DA, Greischar LL, Davidson RJ (2003) Spatio temporal dynamics of brain mechanisms in aversive classical conditioning: high density event related potential and brain electrical tomography analyses. Neuropsychologia 41:184 194. Pizzagalli DA, Regard M, Lehmann D (1999) Rapid emotional face processing in the human right and left brain hemispheres: an ERP study. Neuroreport 10:2691 2698. Pourtois G, Grandjean D, Sander D, Vuilleumier P (2004) Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cereb Cort ex 14:619 633. Quian Quiroga R (2000) Obtaining single stimulus evoked potentials with wavelet denoising. Phys Nonlinear Phenom 145:278 292. Quian Quiroga R, Garcia H (2003) Single trial event related potentials with wavelet denoising. Clin Neurophysiol. 1 14, 376 390. Quirk GJ, Armony JL, LeDoux JE (1997) Fear conditioning enhances different temporal components of tone evoked spike trains in auditory cortex and lateral amygdala. Neuron 19:613 624. Rajagovindan R, Ding M (2011) From prestimulus alpha oscilla tion to visual evoked response: an inverted U function and its attentional modulation. J Cogn Neurosci 23:1379 1394. Roberts AC, Tomic DL, Parkinson CH, Roeling TA, Cutter DJ, Robbins TW, Everitt BJ (2007) Forebrain connectivity of the prefrontal cortex in the marmoset monkey (Callithrix jacchus): an anterograde and retrograde tract tracing study. J Comp Neurol 502:86 112. Rogers RD, Ramnani N, Mackay C, Wilson JL, Jezzard P, Carter CS, Smith SM (2004) Distinct portions of anterior cingulate cortex and medi al prefrontal cortex are activated by reward processing in separable phases of decision making cognition. Biol Psychiatry 55:594 602. Romei V, Brodbeck V, Michel C, Amedi A, Pascual Leone A, Thut G (2008) band EEG ac tivity reflect variability in excitability of human visual areas. Cereb Cortex 18:2010 2018. Romei V, Gross J, Thut G (2010). On the role of prestimulus alpha rhythms over occipito parietal areas in visual input regulation: correlation or causation? J Neur osci 30:8692 8697.

PAGE 107

107 Rotshtein P, Malach R, Hadar U, Graif M, Hendler T (2001) Feeling or features: different sensitivity to emotion in high order visual cortex and amygdala. Neuron 32:747 757. Ruff CC, Bestmann S, Blankenburg F, Bjoertomt O, Josephs O, Weiskopf N, Deich mann R, Driver J (2008). Distinct causal influences of parietal versus frontal areas on human visual cortex: evidence from concurrent TMS fMRI. Cereb Cortex 18:817 827. Ruff CC, Blankenburg F, Bjoertomt O, Bestmann S, Freeman E, Hayn es JD, Rees G, Josephs O, Deichmann R, Driver J (2006) Concurrent TMS fMRI and psychophysics reveal frontal influences on human retinotopic visual cortex. Curr Biol 16:1479 1488. Sabatinelli D, Bradley MM, Fitzsimmons JR, Lang PJ (2005) Parallel amygdala and inferot emporal activation reflect emotional intensity and fear relevance. Neuroimage 24:1265 1270. Sabatinelli D, Bradley MM, Lang PJ, Costa VD, Versace F (2007a) Pleasure rather than salience activates human nucleus accumbens and medial prefrontal cortex. J Neur ophysiol 98:1374 1379. Sabatinelli D, Keil A, Frank DW, Lang PJ ( 2013) Emotional perception: correspondence of early and late event related potentials with cortical and subcortical functional MRI. Biol Psychol 92:513 519. Sabatinelli D, Lang PJ, Bradley MM, Costa VD, Keil A (2009) The timing o f emotional discrimination in human amygdala, inferotemporal, and occipital cortex. J Neurosci 29:14864 14868. Sabatinelli D, Lang PJ, Keil A, Bradley MM (2007b) Emotional perception: correlation of functional MRI and event related potentials. Cereb Cortex 17:1085 1091. Sakai K (2008) Task set and prefrontal cortex. Annu Rev Neurosci 31:219 245. Sauseng P, Feldheim JF, Freunberger R, Hummel FC (201 1) Right prefrontal TMS disrupts interregional anticipatory EEG alpha activity during shifting of visuospatial attention. Front Psychol 2:241. Sauseng P, Klimesch W, Stadler W, Schabus M, Doppelmayr M, Hanslmayr S, Gruber WR, Birbaumer N (2005). A shift of visual spatial attention is selectively associated with human EEG alpha activity. Eur J Neurosci 22:2917 2926. Scheeringa R, Mazaheri A, Bojak I, Norris DG, Kleinschmidt A (2011) Modulation of visually evoked cortical fMRI r esponses by phase of ongoing occipital alpha oscillations. J Neurosci 31:3813 3820.

PAGE 108

108 Schupp HT, Cuthbert BN, Bradley MM, Cacioppo JT, Ito T, Lang PJ (2000) Affective picture processing: the late positive potential is modulated by motivational relevance. Ps ychophysiology 37:257 261. Schupp HT, Cuthbert B, Bradley M, Hillman C, Hamm A, Lang P (2004a) Brain processes in emotion al perception: motivated attention. Cogn Emot 18:593 611. Schupp HT, Junghfer M, Weike AI, Hamm AO (2004b) The selective processing of briefly presented affective pictures: an ERP analysis. Psychophysiology 41:441 449. Schupp HT, Markus J, Weike AI, Hamm AO (2003) Emotional facilitation of sensory processing in the visual cortex. Psychol Sci 14:7 13. Schwabe L, Obermayer K (2005) Adapti vity of tuning functions in a generic recurrent network model of a cortical hypercolumn. J Neurosci 25:3323 3332. (2003) Quantitative analysis of attention and detection signals during visual search. J Neurophysiol 90:3384 3397. Shulman GL, Ollinger JM, Akbudak E, Conturo TE, Snyder AZ, Petersen SE, Corbetta M (1999) Areas involved in encoding and applying directional expectations to moving objects. J Neurosci 19:9480 9496. Stola rova M, Keil A, Moratti S (2006) Modulation of the C1 visual event related component by conditioned stimuli: evidence for sensory plasticity in early affective perception. Cereb Cortex 16:876 887. Straube T, Miltner WHR (2011) Attention to aversive emotio n and specific activation of the right insula and right somatosensory cortex. Neuroimage 54:2534 2538. Tang AC, Sutherland MT, McKinney CJ (2005) Validation of SOBI components from high density EEG. Neuroimage 25:539 553. Thut G, Nietzel A, Brandt SA, Pasc ual band electroencephalographic activity over occipital cortex indexes visuospatial attention bias and predicts visual target detection. J Neurosci 26:9494 9502. Tolias AS, Keliris GA, Smirnakis SM, Logothetis N (2005) Neurons in macaque area V4 acquire directional tuning after adaptation to motion stimuli. Nat Neurosci 8:591 593. Truccolo WA, Ding M, Knuth KH, Nakamura R, Bressler SL (2002) Trial to trial variability of cortical evoked responses: implications for the analysis of function al connectivity. Clin Neurophysiol 113:206 226.

PAGE 109

109 Truccolo W, Knuth KH, Shah A, Bressler SL, Schroeder CE, Ding M (2003) Estimation of single trial multicomponent ERPs: differentially variable component analysis (dVCA). Biol Cybern 89:426 438. Tuan PD, Mcks J, Khler W, Gasser T (1987) Variable latencies of noisy signals: estimation and testing in brain potential data. Biometrika 74:525 533. Vanderperren K, De Vos M, Ramautar JR, Novitskiy N, Mennes M, Assecondi S, Vanrumste B, Stiers P, Van den Bergh BRH, W agemans J, Lagae L, Sunaert S, Van Huffel S (2010) Removal of BCG artifacts from EEG recordings inside the MR scanner: A comparison of methodological and validation related aspects. Neuroimage 50:920 934. Vossel S, Weidner R, Driver J, Friston KJ, Fink GR (2012) Deconstructing the architecture of dorsal and ventral attention systems with dynamic causal modeling. J Neurosci 32:10637 10648. Vuilleumier P, Armony JL, Driver J, Dolan RJ (2001) Effects of attention and emotion on face processing in the human bra in: an event related fMRI study. Neuron 30:829 841. Walsh BJ, Buonocore MH, Carter CS, Mangun GR (2010) Integrating conflict detection and attentional control mechanisms. J Cogn Neurosci 23:2211 2221. Wang X, Chen Y, Ding M (2008) Estimating Granger causal ity after stimulus onset: a cautionary note. Neuroimage 41:767 776. Wang X, Ding M (2011) Relation between P300 and event related theta band synchronization: a single trial analysis. Clin Neurophysiol 122:916 924. Weinberger NM (1995) Dynamic regulation of receptive fields and maps in the adult sensory cortex. Annu Rev Neurosci 18:129 158. Weinberger NM (2004) Specific long term memory traces in primary auditory cortex. Nat Rev Neurosci 5:279 290. Weinberg A, Hajcak G (2011) Electrocortical evidence for vig ilance avoidance in generalized anxiety disorder. Psychophysiology 48:842 851. Wen X, Yao L, Liu Y, Ding M (2012) Causal interactions in attention networks predict behavioral performance. J Neurosci 32:1284 1292. Weymar M, Schwabe L, Lw A, Hamm AO (2012) Stress sensitizes the brain: increased processing of unpleasant pictures after exposure to acute stress. J Cogn Neurosci 24:1511 1518.

PAGE 110

110 Whalen PJ, Rauch SL, Etcoff NL, McInerney SC, Lee MB, Jenike MA (1998) Masked presentations of emotional facial expressio ns modulate amygdala activity without explicit knowledge. J Neurosci 18:411 418. Wills AJ (2009) Prediction errors and attention in the presence and absence of feedback. Curr Dir Psychol Sci 18:95 100. Wills AJ, Lavric A, Croft GS, Hodgson TL (2007) Predictive learning, prediction errors, and attention: Evidence from eventrelated potentials and eye tracking. J Cogn Neurosci 19:843 854. Woody C (1967) Characterization of an adaptive filter for the analysis of variable latency neuroelectric signals. Med Biol Eng 5:539 554. Worden MS, Foxe JJ, Wang N, Simpson GV (2000) Anticipatory biasing of visuospatial band electroencephalography increases over occipital cortex. J Neurosci 20, RC63. Xu L, Stoica P, Li J, Bressl er SL, Shao X, Ding M (2009) ASEO: a method for the simultaneous estimation of single trial eventrelated potentials and ongoing brain activities. IEEE Trans Biomed Eng 56:111 121. Yamagishi N, Callan DE, Goda N, Anderson SJ, Yoshida Y, Kawato M (2003) Atte ntional modulation of oscillatory activity in human visual cortex. Neuroimage 20:98 113. Zald DH (2003) The human amygdala and the emotional evaluation of sensory stimuli. Brain Res Rev 41:88 123. Zhang Y, Ding M (2010) Detection of a weak somatosensory st imulus: role of the prestimulus mu rhythm and its top down modulation. J Cogn Neurosci 22:307 322.

PAGE 111

111 BIOGRAPHICAL SKETCH Yuelu Liu was born in 1985, in city of Changsha, Hunan province, China. He graduated from the m iddle s chool a ttached to the Hunan Normal University in 2003. He did his undergraduate study at Beijing Institute of Technology and earned the B.S. degree in electrical and information engineering in 2007. He then enrolled in the graduate program at the University of Florida and earned his M.S. d egree in electrical engineering in 2009. During his graduate study, Yuelu became interested in human cognitive neuroscience and continued to pursue his doctoral degree in the J Crayton Pruitt Family Department of Biomedical Engineering under the mentorship of Dr. Mingzhou Ding. He received his Ph.D. degree from the University of Florida in the summer of 2013. His current research interests mainly include applying multimodal imaging and advanced engineering methods to understand the neural mechanisms underlying higher level human cognitive functions such as attention and emotion and their impairm ents in neurological and psychiatric disorders