Citation
Episodic Memory, Integrative Processing, and Memory-Contingent Brain Activity during Encoding

Material Information

Title:
Episodic Memory, Integrative Processing, and Memory-Contingent Brain Activity during Encoding
Copyright Date:
2008

Subjects

Subjects / Keywords:
Electrodes ( jstor )
Experimentation ( jstor )
Hemispheres ( jstor )
Magnetic resonance imaging ( jstor )
Memory ( jstor )
Memory encoding ( jstor )
Memory retrieval ( jstor )
Topography ( jstor )
Waveforms ( jstor )
Working memory ( jstor )

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Embargo Date:
7/30/2007

Downloads

This item has the following downloads:


Full Text












EPISODIC MEMORY, INTEGRATIVE PROCESSING, AND MEMORY-
CONTINGENT BRAIN ACTIVITY DURING ENCODING















By

BRIAN G. HOWLAND


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2005

































Copyright 2005

by

Brian G. Howland











ACKNOWLEDGMENTS

Although many have shaped the work presented here, in the interest of space, I

mention but a few. Foremost, I thank Prof Ira Fischler whose scientific curiosity,

creativity and patience enabled me to come full circle from ERPs to fMRI and back

again. I am indebted to Prof. Fischler for his many years of inspiration and tireless effort

on my behalf, as well as on behalf of, literally, the thousands of University of Florida

students whose lives he has enriched with his spirit. The other members of my

committee, Prof. Keith Berg, Dr. Bill Perlstein, especially Dr. Lise Abrams, have

provided valuable feedback and criticism. I give many thanks to Dr. Abrams for her

encouragement when the work was in danger of stalling. I was also inspired by my office

mate, Jesse Itzkowitz, and especially by the words of, and the example set by, Dr.

Michael Membrino. I also thank the undergraduate research assistants, whose enthusiasm

for the project was certainly displayed in their hard work. Among them, but without

ignoring the others not mentioned, I thank especially S. Jones, B. Lawson, K. Tobago, J.

Lapnawan, A. Persons, B. Yocum, A. Schweit, and A. Mejia. Finally, no

acknowledgment of any undertaking this size would be complete without thanking family

and friends. I thank my parents, Lois and Paul Howland, for being supportive of the

middle-aged intellectual wanderings of their son; my father-in-law, Dr. Alan Sheppard

for his scientific curiosity; and, most especially, for their great perseverance, patience,

love and support, my family: Dena, Caley, Jonathan, and Julia. A special note of thanks

to Dena, who selflessly put my interests ahead of her own at a most critical juncture. I'll

always be grateful.















TABLE OF CONTENTS

page

A C K N O W L E D G M E N T S ....................................................................... .....................iii

LIST OF TABLES ........................... ...... ................................ ............ .. vi

LIST OF FIGURES ........................... ...... ................ ............. .......... vii

A B ST R A C T ................................viii............................

1 IN TR O D U C T IO N ............ ............................................. ........ .. ........... .. 1

Events, Context, and Episodic M em ory............................... ................................ 1
Creating Episodic M em ories: Theories and Data ................................. .................... 4
Working Memory Studies of Object-Location Binding .............. .. .......... 5
E R P s and E pisodic E ncoding ..................................................... .................... 8
The Dm/subsequent memory effect............... ...................................... 9
Dm and associative encoding .............. .................................... .......... 10
Dm and elaborative encoding .......................... ................... ............. 12
ERP sum m ary ........... .. ............. .. ............ ...... .. .. .......... 18
Functional Imaging and Associative Encoding ............................................... 20
The Present Investigation...................... ........ ............................. 26

2 E X PE R IM E N T O N E ....................................... ........... ................ ......................... 28

Expected Results................... .......... .............. 29
M e th o d ........................................................................................................ 3 1
Participants ............... ....................... ... .... .......... .......... 31
M materials and A pparatus .................................................................. ....... .. 32
Stimulus display and response recording ........................................ ...... 32
E E G recording ........................................................... ..... ......... 32
Stimulus m materials .............. ..... ............. ....... ............ ............. .. 33
Design .............. ......... ............................ 34
P procedure ........................................ 34
R results .......... ............................................................................................. 37
B behavioral D ata ..................................................... .............. 37
EEG Data Preprocessing....................... ........ ......... 37
ERP waveforms .................................................. ........ 38
Statistical analysis of waveforms .................................. 40
B ound condition ........................................ 44
Separate condition ......................................................... 46


iv









Bound vs. separate conditions ...................................................... 46
D iscu ssion ..................................................................................................... 46

3 EXPERIM ENT TW O .. ...................................................... ...... .............. 52

E expected R results ......................................................................... ......... ...................... 52
M ethod ............. .... ........................................................................................ 54
Participants............................. ............. 54
M materials and Apparatus ................................... ....................... 54

E E G recording ........................................................................ . .......... 54
Stim ulus m materials ............................ ......................... .............. 55
D design ............... ............ ........... ....... .......... .... 55
Procedure .............. ...... ... ........... ............. ......... ... 55
R results .......... ............................................................................................. 57
B behavioral D ata ..................................................... .............. 57
EEG Data ............... .......... ..................... 58
ERP waveforms .................................................. ........ 58
Statistical analysis of waveforms .................................. 60
B ound condition ........................................ 60
Separate condition ......................................................... 60
B found vs. separate analysis ................................................................... 64
D isc u ssio n ...................................................................................................... 6 5

4 GENERAL DISCUSSION ................................................................ 69

Distinctive Aspects of the Present Approach .................................. ..... ................... 69
Memory-Related ERPs and Integrative Episodic Encoding ............... ..................... 71
Com prisons to Previous Findings ....... ............................. ............ 74
Limitations and Future Directions ......... ......................... ............ ........ 78

APPENDIX

A EXPERIMENT 1, STUDY PHASE WORD PAIRS .............................................. 82

B EXPERIMENT 1, PAIRED RECOGNITION TEST WORD PAIRS ...................... 85

C EXPERIMENT 2, STUDY PHASE WORD PAIRS .............................................. 88

D EXPERIMENT 2, OBJECT WORD TEST ITEMS .................... ....................... 90

E EXPERIMENT 2, LOCATION WORD TEST ITEMS ......................................... 93

L IST O F R E FE R E N C E S .............. ................................................. .......................... 96

BIOGRAPHICAL SKETCH ......................................................... .............. 104
















LIST OF TABLES

Table p

2-1. Time Intervals in Experiment 1 during which Amplitude Differences were
Significant .................................. ......... .......... 45

3-1.Time Intervals in Experiment 2 during which Amplitude Differences were
S ig n ific a n t ................................................................................................................6 4
















LIST OF FIGURES


Figure page

2-1. Schematic representation of a single trial during the study phase in
Experiment 1..... .................................................... 36

2-2. ERPs to two words during study phase later shown as intact pairs during
test in Experiment 1, Bound Encoding group.................................. ............ 41

2-3. ERPs to two words during study phase later shown as intact pairs during
test in Experiment 1, Separate Encoding group.. ............... ....................... ..... 42

2-4. ERPs to two words during study phase later shown and correctly
recognized as intact pairs during test in Experiment 1, Bound vs Separate............ 43

3-1. Behavioral performance in Experiment 2............... ................................ ........... 57

3-2. ERPs to two words during study phase during test in Experiment 2, Bound
Encoding group....................................... .............. 61

3-3. ERPs to two words during study phase during test in Experiment 2,
Separate Encoding group ........... .............. ...... .............. 62

3-4. ERPs to two words during study phase later shown and correctly
recognized as intact pairs during test in Experiment 2, Bound vs. Separate......... 63















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

EPISODIC MEMORY, INTEGRATIVE PROCESSING AND MEMORY-
CONTINGENT BRAIN ACTIVITY DURING ENCODING

By

Brian G. Howland

August 2005

Chair: Ira Fischler, Ph.D.
Major Department: Psychology

A fundamental element of encoding an experience is establishing a link between

an object or event and its spatiotemporal context. Current theories posit important roles

for the prefrontal cortex and medial temporal lobe complex in successful episodic

"binding." We conducted two experiments to isolate the timing and scalp topography of

event-context encoding effects using event-related brain potentials (ERPs). Participants

were shown sequential (3 sec. apart) word pairs (e.g., ELEPHANT ... BATHROOM)

while their electroencephalograms (EEG) were recorded. Some participants were

instructed to generate a single, integrated mental image while other participants generated

a pair of separate images. Their ability to recognize intact pairs was then tested. As

expected, recognition was better for pairs studied under Bound than under Separate

instructions. ERP encoding differences between later recognized pairs and later forgotten

pairs (Differences associated with subsequent memory performance or "Dm "), especially

at frontal sites, were found for the Bound, but not the Separate, condition. These slow-









wave differences were seen late following both words; however, the differences were of

opposite polarities and contrasting morphology. An early, first-word difference between

the waveforms for intact pairs that were subsequently recognized in the Bound versus the

Separate conditions suggested different preparatory sets in the two tasks. In the second

experiment, participants were given the same imagery tasks but tested subsequently for

item, rather than pair, recognition. Unlike the first experiment, participants showed no

difference in recognition performance by image generation task. As in the first

experiment, there were ERP differences for correctly recognized vs. unrecognized items

in the Bound condition, but these item-specific Dm's were earlier and of a different

topographic distribution than the Dm's for pair recognition. No Dm effects were noted

for the Separate condition. The contrasting ERPs between the Separate and Bound

conditions, and the contrasting Dm's for ERPs conditionalized on item versus pair

recognition, suggest that relational processing contributing to successful object-location

memory requires effortful processing, and is associated with frontal or prefrontal regions

of the cortex.














CHAPTER 1
INTRODUCTION

This dissertation presents a pair of experiments that explore the cognitive and

neural bases of episodic memory encoding. In particular, the studies examine the

creation of a mental link between events and their spatiotemporal contexts by recording

event-related brain potentials (ERPs) and conditionalizing those electrophysiological

measures on subsequent memory performance in different tasks and under different

instructions.

Events, Context, and Episodic Memory

As Tulving (1984) has noted, the basic unit of an individual's perception of time

is an "event;" that is, some occurrence at a given place at a given time. An ongoing series

of events make up an "episode." Episodic memory enables humans to "time travel;" that

is, to place ourselves in the recent or distant past, or even, the imagined future (Tulving,

1985). Without episodic memory, one lives in a constant, immediate present, like the

well-known amnesic musician, Clive Wearing. Successful episodic memory

performance, therefore, requires that the episode's context be linked with the focal event

itself upon its initial occurrence. It is this linkage of spatiotemporal context and focal

event that enables us to separate personally experienced, geographically distinct, but

close in time, events from one another ("First I was in the kitchen, then I went into the

dining room"). Moreover, we use episodic memory to distinguish identical or similar

events by the order of their temporal occurrence ("I saw a dog run across the street

yesterday. I saw the same dog run across the street this morning").









As important as this spatiotemporal linking of event and context is for memory, it

apparently is not an obligatory or automatic one; indeed, one of the classic "sins of

memory" (Schacter, 2001) is to remember an event but forget the context, or remember it

falsely in the wrong context. A wide variety of experimental protocols have shown that

both healthy participants (Chalfonte & Johnson, 1996; Henkel, Johnson, & DeLeonardis,

1998; Mitchell, Johnson, Raye, Mather, & D'Esposito, 2000; Mitchell, Johnson, Raye, &

D'Esposito, 2000) as well as neurologically impaired patients (Turriziani, Fadda,

Caltagirone, & Carlesimo, 2004) and can retain relatively good levels of item

recognition yet show substantial decrements in the ability to identify either the

spatiotemporal context in which a recognized item was presented or to recognize (an)

additional features) with which the to-be remembered item was to be associated. Thus,

the processes that support item recognition or recall appear to be distinguishable from

those that support contextual memory.

A widely held analogy has been drawn between the ability to remember the

contextual features of an event, and the subjective sense of remembering that has been

termed "recollection." Indeed, since the mid-1980's, the qualitative distinction between

"remembering" and "knowing" (Tulving, 1985) (or "recollection" versus "familiarity" for

others) has been a central topic in the study of episodic memory (e.g., Yonelinas, 2002).

The precise natures of the processes that support recollection are still largely undefined. It

is clear that sensory information must be transformed into internal representations.

However, to permit successful subsequent recollection, the elements of sensory

experience, together with any relevant internally generated cognition and emotional









states, must be combined in such a way that the experience is capable of later being

reinstated (Paller & Wagner, 2002).

Understanding the cognitive and neural processes that support successful episodic

memory performance involves analyzing events both at encoding and at time of retrieval.

While there has been a significant amount of research examining the retrieval-associated

processes in episodic memory, far fewer studies have pursued the encoding processes that

underlie episodic memory formation. For example, the degree to which, and conditions

under which, attention plays a role in the encoding of context is unknown. Given the

continuous stream of information one encounters, it seems likely that some degree of

automaticity is required for everyday episodic memory to function efficiently. However,

while Hasher and Zacks (1979) suggested that fundamental information such as time,

spatial location and frequency of occurrence may be encoded relatively automatically,

Craik (1989) suggested that, in some cases, attention might play an important role in the

integration of an event with its context.

Furthermore, the content and the context may interact to make the context more

memorable (e.g., an elephant on the sidewalk is more memorable than ajogger on the

sidewalk but a jogger in the zoo may be as memorable as an elephant in the zoo). Finally,

it is unclear whether locations and their associated objects are bound together (and

subsequently retrieved) in a single representation in holistic fashion, or if a link or pointer

is formed that connects independently created and maintained episodic representations.

The experiments presented in this dissertation are an attempt to identify some of

the neural and cognitive encoding processes that support successful item + context









retrieval and distinguish them from the encoding processes that support successful item-

only retrieval.

Creating Episodic Memories: Theories and Data

Long-term memory research has identified a variety of encoding factors (e.g.,

organization among items in a list; "depth" or degree of elaborative item processing, item

frequency or familiarity) that are associated with successful long-term memory. The

processes by which items or events and their contexts are bound, however, have been

little explored. While traditional principles of associative learning (e.g., intra- and extra-

item organization) may apply, it is possible (or even likely) that other discoverable,

Gestalt-like principles may be at work (Craik, 1989; Kounios, Smith, Yang, Bachman, &

D'Esposito, 2001).

In a series of unpublished studies, Craik (1989) explored the effects of stimulus

integrability and attention on the degree to which item and context recall were

independent. Overall, he found that context recall declined more rapidly initially than

item recognition as attention was diverted during study, but, as context recall approached

chance levels, item recognition then dropped quite rapidly. Moreover, for items and

contexts that were thought to be more "integrable," memory performance for items and

contexts were more closely bound to one another. An important, yet unanswered question

is what factors might affect integration of item and context. Craik suggested that the

emotional content of the item-context could affect the ease of integration. Nevertheless,

there has been little work, to date, on the cognitive and neural processes that successfully

link events to their contexts in long-term memory.

Three areas of research, reviewed below, may provide some guidance. First, a few

working memory studies (Chalfonte & Johnson, 1996; Luck & Vogel, 1997; Mitchell et









al., 2000a; Mitchell et al., 2000b; Prabhakaran, Narayanan, Zhao, & Gabrieli, 2000; cf,

Bor, Duncan, Wiseman, & Owen, 2003) have examined the binding of an object and

other feature information (e.g., spatial location, color) in working memory. If the

processes engaged during working memory binding are utilized in long-term memory

encoding, these studies are important in revealing the basis of episodic encoding. Second,

use of physiological measures of brain activity such as electroencephalographic (EEG)

recording to identify event-related brain potentials (ERPs) have revealed cognitive and

neural processes that differentiate successful and unsuccessful long-term memory

encoding. Third, event-related functional magnetic resonance imaging (ER-fMRI) studies

have begun to reveal subcortical regions and areas in medial temporal (MTL) and

prefrontal (PFC) cortex that distinguish successful and unsuccessful long-term memory

encoding. Each of these areas of research will be discussed below.

Working Memory Studies of Object-Location Binding

In a number of studies, participants were required to briefly maintain two or more

stimulus features or dimensions either separately, or in an integrated representation.

These studies have shown that the ability to remember an object-location association is

distinguishable from the ability to remember objects or features separately. Thus, for

example, the deficits that older adults show in source memory cannot be attributed

merely to the inability to remember a greater number of objects or features (Chalfonte &

Johnson, 1996; Mitchell et al., 2000a; Mitchell et al., 2000b). The processes that underlie

the short-term maintenance and manipulation of objects in working memory have been

characterized as "reflective processes" (Johnson, 1992), but it is unclear whether these

processes play a role in successful long-term memory for object and context binding. It is

unclear, also, whether working memory and long-term memory encoding share a set of









common cognitive processes. If so, these working memory studies may reveal some of

the principles at work in successful episodic binding.

Recently, there has been some convergence on this issue (Baddeley, 2000;

Fletcher & Henson, 2001; Johnson, 1992; Wagner, 1999). Some investigators have

claimed that working memory and long-term memory encoding share a common set of

processes (Wagner, 1999) while others (Baddeley, 2000) have proposed common

structural components between working memory and long-term memory.

According to one view (Baddeley, 2000), binding of information from verbal and

non-verbal slave systems takes place in an "episodic buffer" that stores that information

in a multimodal code or representation. This bound information can then be passed back

and forth between working memory and long-term "episodic" memory. According to this

view, working memory and long-term episodic memory share an interlocking component

(the episodic buffer) and a set of common processes (binding and maintenance in the

buffer). An alternative view is one that emphasizes the commonality of the cognitive

processes that underlie working memory and long-term memory performance (e.g.,

Fletcher & Henson, 2001; Johnson, 1992; Wagner, 1999) In its most developed form,

this approach is process-specific (versus task-specific) and its goal is to identify and

define the processes that underlie a variety of mnemonic and other cognitive phenomena.

The most developed example of this type of model is Johnson's (1992; Johnson &

Hirst, 1993) multiple-entry, modular memory (MEM) framework. It presupposes that a

common set of cognitive subprocesses act on a variety of cognitive tasks. Thus,

according to the model, the subprocesses used in working memory binding operate in

successful long-term episodic encoding as well. The MEM model includes high-level









subprocesses such as initiating plans; discovering relations among stimuli; rehearsing and

retrieving and lower-level subprocesses such as noting relations; shifting attention;

refreshing currently active representations and reactivating stored representations.

Johnson and colleagues' work has examined the nature of the object-location

binding deficits observed in older (relative to younger) adults (Chalfonte & Johnson,

1996; Mitchell et al., 2000a; Mitchell et al., 2000b). In various studies (Mitchell et al.,

2000a; Mitchell et al., 2000b; Ranganath, Johnson, & D'Esposito, 2003; Raye, Johnson,

Mitchell, Reeder, & Greene, 2002), she and her colleagues have suggested that deficits in

feature binding may be attributable to difficulties in reactivating stored representations

and in failing to refresh currently active representations. Although Johnson and

colleagues' studies provide evidence of a set of cognitive subprocesses that are involved

in binding object and location features in working memory, their findings have not been

extended to long-term memory encoding.

In a recent study using event-related functional imaging, or fMRI (see following),

however, Ranganath and colleagues (2003) have compared the areas of neural activation

associated with working memory and successful long-term memory encoding. They

found that separate face-encoding tasks, with identical stimuli but different encoding

loads and retention intervals, activated similar areas of prefrontal cortex. While this

finding suggests that the same cognitive (or at least neural) processes underlie certain

working memory tasks and long-term memory encoding, methodological and

experimental constraints make such conclusions tentative at best.

Thus, a few studies (Chalfonte & Johnson, 1996; Luck & Vogel, 1997; Mitchell et

al., 2000a; Mitchell et al., 2000b; Prabhakaran et al., 2000) have considered the issue of









how working memory binds separate stimulus dimensions or features into an integrated

whole. To date, however, there has been, to my knowledge, no direct measurement of

long-term memory performance following the systematic manipulation of stimulus

features, dimensions or combinations of stimuli to promote or inhibit working memory

binding.A relatively direct way to study the neural and cognitive bases of long-term

memory encoding is to record some physiological index of cognitive activity during an

encoding event and then sort those records by subsequent memory performance during a

later memory test. Such procedures have yielded reliable differences between later-

remembered and later-forgotten items using both EEG and fMRI measures. Functional

MRI studies suggest that these subsequent memory effects are associated with heightened

medial temporal lobe (MTL) and prefrontal cortex (PFC) activation. However, the

precise role of these structures in long-term memory encoding is still unclear. Findings

from EEG studies, while isolating subsequent memory differences from about 400 ms

onward following stimulus presentation, have varied substantially from one another both

in the locus and timing of subsequent memory effects. As a result, EEG subsequent-

memory studies have done little to identify the important processes that link event and

context in memory encoding.

ERPs and Episodic Encoding

The use of stimulus-locked, event-related brain potentials has been a valuable tool

in identifying a variety of neurocognitive memory processes over the past two decades.

For example, a robust finding is that correctly recognized old items show a greater

positivity than correctly identified new items. These "Old-New" effects have been shown

with a large variety of stimulus materials in many test formats. A less robust, but well-

replicated, finding is that, under certain circumstances, studied stimulus materials that









will be subsequently recognized as "Old" show a greater positivity than items that will

later be classified erroneously as "New." These subsequent memory effects (or "Dm" -

Differences associated with subsequent memory performance) have been used to

examine encoding in long-term memory.

The Dm/subsequent memory effect

Most ERP studies of encoding have used the same basic paradigm. In this

paradigm, electrical activity is recorded while subjects are presented with stimuli that are

subsequently tested under either explicit (e.g., recognition, cued recall) or implicit (e.g.,

stem-completion) conditions. These records are then classified according to subsequent

memory (recalled/recognized vs. unrecalled/unrecognized) performance. Quite a number

of investigators (Besson & Kutas, 1993; Duarte, Ranganath, Winward, Hayward, &

Knight, 2004; Fabiani & Donchin, 1995; Fabiani, Karis, & Donchin, 1986; Fernandez et

al., 1999; Fernandez et al., 1998; Friedman, 1990; Friedman, Ritter, & Snodgrass, 1996;

Friedman & Trott, 2000; Gonsalves & Paller, 2000; Guo, Voss, & Paller, 2005; Guo,

Zhu, Ding, Fan & Paller, 2004; Karis, Fabiani, & Donchin, 1984; Lian, Goldstein,

Donchin, & He, 2002; Mangels, Picton & Craik, 2001; Mecklinger & Muller, 1996;

Munte, Heinze, Scholz, & Kunkel, 1988; Neville, Kutas, Chesney, & Schmidt, 1986;

Sanquist, Rohrbaugh, Syndulko & Lindsley, 1980; Schott, Richardson-Klavehn, Heinze,

& Duzel, 2002; Smith, 1993; Summerfield & Mangels, 2005; Van Petten & Senkfor,

1996; Weyerts, Tendolkar, Smid, & Heinze, 1997; Yovel & Paller, 2004) have shown

differential scalp recorded electrical activity at encoding between subsequently

remembered and unremembered stimuli ("difference associated with memory" (Dm) or

"subsequent memory effects"). These differences usually, but not always, consist of a

greater positivity for remembered items than for unremembered items, although the









timing and topography of these effects vary depending upon the precise experimental

conditions. Some published studies report the effects as containing a frontal maximum.

Others show robust midline effects; and yet other studies have a posterior maximum

(Johnson, 1995; Rugg & Allan, 2000). Some portion of this variance may be due to the

difference in stimulus materials across studies (Johnson, 1995).

Dm and associative encoding

Only a handful of published studies (Guo et al., 2005; Fernandez et al., 1998;

Kounios et al., 2001; Weyerts et al., 1997; Yovel & Paller, 2004) have sorted encoding

ERPs for associative information by memorial success. Kounios and colleagues (2001)

isolated electrical activity associated with faster (better) subsequent memory for

associated words than for more slowly recognized associated words. They explored

whether two proposed processes of cognitive association, juxtaposition and fusion, have

different neural bases. They presented word pairs that either could be fused to create a

novel concept (e.g., computer-virus) or could not easily be combined into a single unique

concept (e.g., salt-pepper). Using a dense electrode array, they measured electrocortical

activity as participants decided whether or not fusion was possible. Subsequently, pairs

were re-presented, one half in the same order (e.g., salt-pepper), one half reordered (e.g.,

virus-computer). Participants judged whether the pairs were as presented previously or

reordered. Faster word pair order judgments were deemed to represent better memory

(and hence better encoding). A median split of the word pair order judgment RTs showed

that fusible pairs that were judged fusible more quickly at study were also responded to

more quickly at test. Conversely, non-fusible (juxtaposed) pairs to which participants

responded more quickly at study were responded to more slowly at test.









As with the behavioral data, ERP data were classified by response speed at test.

Both juxtaposed and fusion pairs showed an effect at study of the speed of the responses

at test. This "subsequent memory effect," however, is different from the standard

subsequent memory effects discussed above. First, it is important to emphasize that the

ERP memory differences are between (ostensibly) better and more poorly remembered

word pair orders, not between remembered and unremembered items. Second, although

the fused pairs showed a greater positivity for faster than for slower judgments, the

juxtaposition pairs showed the opposite pattern, with slower word order judgments being

associated with a greater positivity than faster judgments. These retrieval speed effects

persisted throughout the recording epoch. Kounios and colleagues interpreted the ERP

retrieval differences and the subsequent localization of those differences to the right

prefrontal cortex as indexing processes associated with an attempt to fuse the words of

the pair. Such processes might include maintenance of the pair in working memory,

construction of candidate fusions, and evaluation of these fusions. Implementation of

such processes would explain why the ERP effects would be present in the early epoch

(200 800 ms) of both fusion and juxtaposition pairs, but would persist into the middle

epoch (800 2100 ms) only for juxtaposition pairs in which the search for an

appropriate fusion might be expected to continue.

Weyerts and colleagues (1997) examined the ERP correlates of two semantic

encoding tasks. One task required determining whether either word of a pair was

associated with a given color; thus the task demanded semantic evaluation of both words,

but the associative relationship between the pair was irrelevant (nonassociative task). The

second task required participants to judge whether the words of the pair were









semantically related to one another. Again, semantic analysis was required of each word

of the pair, but the task further required participants to judge the items' interrelatedness

(associative task). Old and new word pairs were presented in a subsequent incidental

recognition memory test. Subsequent memory effects for words encoded in the

associative tasks were found at frontal sites, with right frontal effects greater than left

frontal effects. No subsequent memory effects were found for the nonassociative

encoding task. The authors interpreted the difference in subsequent memory effects

between the associative and nonassociative tasks as reflecting the creation of a more

elaborated memory trace in the associative task than in the nonassociative task.

Dm and elaborative encoding

Another group of studies (Duarte et al., 2004; Fernandez et al., 1998; Friedman &

Trott, 2000; Guo et al., 2004; Mangels et al., 2001; Schott et al., 2002; Smith, 1993) has

observed subsequent memory effects that appear to be associated with elaborative

processing (when the task does not explicitly demand that participants process stimuli

relationally). For example, Mangels, Picton and Craik (2001) had participants memorize

lists of 45 words for subsequent explicit recall and recognition tests. The words were

studied under either full attention or divided-attention (not discussed). Participants were

given no particular instructions for memorizing the words. At test, participants first were

given three minutes to recall as many of the words as possible. Thereafter, they were

shown a series of words (50% old) to which they responded, "remember," "know," or

"new" (following the Remember-Know paradigm of Tulving, 1985). Subsequent memory

effects showed both an anterior positive and a posterior negative sustained potential.

Mangels and colleagues pointed out that the sustained anterior positivity was consistent

with a few earlier findings in which late, sustained anterior subsequent memory effects









were induced when the study task involved biasing participants toward the elaboration of

the stimuli. They pointed out that it was unclear which types of elaborative processes

were involved in their task. They speculated, however, that the anterior positivity may

have consisted of two separate components: (1) an earlier, left-sided positivity

representing activation of processes involved in associational or relational processing

between the stimulus and information in semantic memory and (2) a later, more right-

sided effect representing retrieval of previous list items no longer in current awareness

and comparison processes necessary for strategic organization of list items. Despite their

speculation, the task used by the investigators did not explicitly manipulate any of these

purported processes; so further evidence is necessary to confirm their claims.

Mangels and colleagues attributed the sustained posterior negativity to the

representation of the concrete object represented by the word or the representation of the

word itself. They pointed out, however, that such posterior inferior sustained negativity

had been identified in only one other study. They attributed this lack of similar findings

in the subsequent memory literature to their choice of using an average, rather than a

mastoid or earlobe, reference. They pointed out a similar negativity at the mastoid

electrodes (TP9/10) which would have been subtracted out had they been used as a

reference.

Mangels and colleagues also identified a parietal positivity (P280) and a fronto-

temporal negativity (N340) that separated old words that were subsequently missed from

recognized words (but which did not distinguish between R and K items). They

concluded that processing up to about 340 ms consisted of the perceptual analysis and

selection of the item as task relevant followed by item-specific semantic processing









(N340). Once this processing is completed, the information is made available to the MTL

system for long-term storage (P550). Thereafter, relational and elaborative processing

takes place via the sustained interaction between frontal and inferior temporal regions

beginning at about 1000 ms.

In an aging study (Friedman & Trott, 2000), participants studied sentences

containing two unrelated nouns (e.g., "The dragon sniffed the fudge.") for a subsequent

explicit recognition test. At test, participants made "Old/New" recognition judgments,

followed by "Remember/Know" decisions to items judged "old." Finally, participants

made temporal order decisions (List 1 vs. List 2). Study ERPs, sorted by recognition

decision, showed a widespread Dm effect among young participants for Remember

decisions only but for both Remember and Know decisions in the older participants.

Friedman and Trott proposed that encoding of items by older participants was less

contextually rich, even for old items judged "Remember," than encoding of "Remember"

items by younger participants. Alternatively (or in conjunction with this shallower

encoding proposal), Friedman and Trott suggested that older participants might have

applied a more liberal criterion to the Remember/Know judgments than did the younger

participants. Moreover, they pointed out that, unexpectedly, there was no correspondence

between Dm effects associated with Remember/Know judgments and source list

judgments. They noted that Remember judgments could be assigned if any aspect of the

encoding session was retrieved, regardless of whether the list from which the item was

taken was retrieved. Thus, Remember responses may or may not have been accompanied

by correct source list judgments.









Therefore, although Friedman and Trott used a nominally associative encoding

task in which participants were given two unrelated words within a sentence, neither the

test (identification of each word as old or new), nor the instructions ("memorize the

nouns for a subsequent memory test) explicitly demanded that the words be encoded

together. In fact, at test, rearranged old items required "Old/Old" responses. Thus,

encoding the items as a pair could make it more difficult to respond "old" to the second

word if it were repaired. Therefore, Friedman and Trott showed a widespread Dm effect

that varied by the response type (Remember/Know) and interacted with age. The study

does not, however, clarify the nature of the encoding at study that produced the Dm

although their suggestions that lack of differences between Remember and Know

responses for older participants reflected shallower encoding, or less elaborated traces, is

attractive.

A recent study, however, suggests that ERP effects produced by a levels-of-

processing (LOP) manipulation differ in onset, duration, and topography from ERP Dm

effects. Schott and colleagues (2002) found an LOP effect (deep > shallow) at fronto-

central regions that began at about 600 ms after stimulus onset and lasted until the end of

the recording interval. This contrasted with widespread Dm effects from 600 800 ms

that were associated with only the shallow encoding condition and a right frontal Dm

from 900 1200 ms associated with both study conditions. Schott and colleagues argued

that the LOP effects might represent differences in retrieval from semantic memory

required by the two tasks whereas the Dm effects might represent the establishment of an

episodic memory trace. They disagreed with Van Petten and Senkfor's (1996) conclusion

that Dm effects for meaningful words, but not for meaningless, novel visual patterns,









suggests that the Dm reflects retrieval from semantic memory and point out that Van

Petten and Senkfor's "Dm" effects might have consisted of both differences in study

processing and in establishment of the memory trace. Schott and colleagues argued

further that the early and late Dm effects (which were modulated by LOP) likely reflect

different processes. For example, the early, widespread Dm (which was present only for

words studied in the shallow condition) might have been associated with the formation of

memory traces containing distinctive orthographic/phonological information. On the

other hand, the late, right prefrontal Dm, which occurred with both study conditions,

might signify the establishment of a memory trace with semantic-associative information.

They argue that occurrence of this Dm in the shallow study condition might simply

reflect the activation of semantic-associative information during the shallow study task

and note that a similar Dm was found during a rote rehearsal task by Fernandez and

colleagues (1998).

Finally, Fernandez and colleagues (1998) interpreted the existing subsequent-

memory-effect literature as consisting of two effects: a centroparietal effect associated

with rote encoding strategies, regardless of distinctiveness, and a frontal effect that is

associated with elaborative encoding (Fabiani, Karis & Donchin, 1990; Karis, Fabiani &

Donchin, 1984; Weyerts et al., 1997). In their own study, they examined the differences

in encoding ERPs associated with item distinctiveness, associative elaboration, or other

"direct" encoding processes. They presented 40, 15-item, word lists, consisting of high

and very low frequency words. Each list was followed by a brief distraction period and a

free recall task. One half of the lists were blocked by word frequency and one half of the

lists contained both high and very low frequency words. Fernandez and colleagues









assumed that associative (inter-item) encoding would facilitate recall of the high- versus

low-frequency words in the blocked lists, and yield intermediate (relative to low-

frequency/mixed and high frequency/blocked) recall rates in the mixed lists. Moreover,

they assumed, consistent with Karis, Fabiani & Donchin (1984), that the amplitude of the

N400 and P300 (LPC) would be associated with distinctiveness and thus should be

associated with low-frequency, more than high-frequency, words. Any subsequent

memory effects unrelated to distinctiveness detection should be dissociable in

topography, amplitude, and/or time course from the enhanced N400/LPC. Moreover, a

subsequent memory effect that was greater for high- than for low-frequency words, and

enhanced further in the blocked condition, would be likely to correspond to associative

processing. If the subsequent memory effect did not interact with word frequency and

presentation (blocked/mixed), then it would be likely to be related to nonassociative

encoding processes.

Subsequent memory effects were dissociable into separable components. One

effect arose at centroparietal and frontopolar sites at about 200 ms for high-frequency

words and at about 350 ms for low-frequency words. It shifted to a single frontopolar

maximum at about 900 ms that differed both in topography and time course from the

distinctiveness effects associated with word frequency. A second subsequent memory

effect, located at a right frontopolar site at between 900 and1300 ms, occurred for high-,

but not low-, frequency words. Fernandez and colleagues concluded that they had

identified subsequent memory effects that were associated neither with distinctiveness

nor with associative processing. Although the second effect was located at right

frontobasal electrodes, as predicted, it was associated only with successfully recalled









high-frequency words. Further inspection showed that only unsuccessfully recalled high-

frequency words failed to elicit any effect at this site; all other types (high-frequency,

successfully recalled and low-frequency, successfully and unsuccessfully recalled)

showed greater amplitude at the frontobasal sites. However, there was no interaction with

presentation type.

ERP summary

A modest number of studies have examined the ERP correlates of encoding. A

few have compared study phase data across encoding manipulations (e.g., levels of

processing). Numerous methodological difficulties arise with such comparisons (e.g.,

equating memorial success across tasks) but it appears that "deep" encoding (relative to

shallow encoding) yields a long-lasting, centroparietal positivity that onsets about 600 ms

after stimulus presentation. The observed ERP differences might reflect retrieval from

semantic memory required in "deep" encoding, but such conclusions are mostly

speculative. A second group of ERP encoding studies have compared the ERP correlates

of successfully recognized (or recalled) items with old items that are unsuccessfully

recognized (or recalled). These subsequent memory effects (or differences associated

with memory "Dm") have been produced using both recognition and recall tests across

a wide variety of experimental conditions. Although some commentators have found it

difficult to generalize from the disparate subsequent memory effect findings, it appears

that hypotheses regarding the nature of the processes underlying the later subsequent

memory effects may be tested. For example, Schott and colleagues have speculated that

the late, right frontal Dm observed in their study, as well as in others' studies, may

signify the formation of memory traces containing semantic associative information.









Nevertheless, the essence of episodic memory is that it includes both the item and

the item's context. Although some ERP studies have purported to examine memory for

"context" or "source" memory, these studies have largely only had participants associate

a single perceptual attribute (e.g., voice, temporal order, spatial location). Moreover, the

nature of such experiments is to repeat a non-meaningful attribute across items, rather

than having participants encounter each item in a unique context, which is likely to

support retrieval differently than when the context (or perceptual attribute) does not

possess unique characteristics.

A few published event-related potential (ERP) studies (Duarte et al., 2004;

Friedman & Trott, 2000; Guo et al., 2005; Kounios et al., 2001; Mangels et al., 2001;

Smith, 1993; Schott et al., 2002; Weyerts et al., 1997; Yovel & Paller, 2004) have

examined context-event or associative encoding. Some of those studies have used the

"Remember-Know" paradigm to distinguish subsequent memory with context

recognition from subsequent memory without context recognition. The findings across

these studies are inconsistent. For example, Smith (1993) found that the subsequent

memory effects (Dm) were similar in timing and topography, regardless of whether they

were associated with "R" or "K" responses. On the other hand, Friedman and Trott,

(2000) found significant Dm effects for subsequent "R," but not "K," responses in young

participants. However, older participants showed Dm effects to both "R" & "K"

responses. Unlike Smith, however, Friedman and Trott found that the Dm effect was

lateralized (L > R), in the young participants (although not in the older participants).

In contrast to the findings of both Smith (1993) and Friedman and Trott (2000),

Duarte and her colleagues (2004) found transient, left frontal Dm effects for items later









classified as "K" and sustained, bilateral (with right > left) Dm effects for items later

classified as "R." Similarly, Mangels, Picton & Craik (2001) found left-lateralized,

fronto-temporal subsequent memory effects at N340 for both R and K responses, which

didn't differ from one another. The differences across these studies are difficult to

reconcile.

Of the remaining studies, two (Guo et al., 2005; Yovel & Paller, 2004) involved

the encoding of novel faces with associated information (names, occupations). Although

Dm effects were observed for encoding of face-name and face-occupation associations,

these effects were neither lateralized, nor transient. Rather, they were long lasting and

topographically central or centro-posterior.

Functional Imaging and Associative Encoding

A large number of functional MRI and PET studies have examined neural activity

at encoding (see Cabeza & Nyberg, 2000; Fletcher & Henson, 2001; Fletcher, Frith, &

Rugg, 1997; Mayes & Montaldi, 1999; Nyberg, 2002; Schacter & Wagner, 1999 for

reviews). The development of event-related fMRI enabled investigators to sort these

encoding records by subsequent memory performance. Since then, a large number of

studies (Baker, Sanders, Maccotta, & Buckner, 2001; Brewer, Zhao, Desmond, Glover, &

Gabrieli, 1998; Buckner, Wheeler, & Sheridan, 2001; Casasanto et al., 2002; Clark &

Wagner, 2003; Daselaar, Veltman, Rombouts, Raaijmakers, & Jonker, 2003; Davachi,

Maril, & Wagner, 2001; Davachi, Mitchell & Wagner, 2003; Davachi & Wagner, 2002;

Henson, Rugg, Shallice, Josephs, & Dolan, 1999; Jackson & Schacter, 2003; Kensinger,

Clarke, & Corkin, 2003; Kirchoff, Wagner, Maril, & Stern, 2000; Otten, Henson, &

Rugg, 2001; Otten & Rugg, 2001; Ranganath et al., 2004; Reber et al., 2002; Rypma &

D'Esposito, 2003; Sommer, Rose, Weiller & Bichel, 2005; Sperling et al., 2003; Stark &









Okado, 2003; Strange, Otten, Josephs, Rugg, & Dolan, 2002; Wagner et al., 1998) have

identified subsequent memory effects in prefrontal cortex (PFC) and the medial temporal

lobe (MTL) complex. A subset of these studies has examined the encoding that underlies

source memory, memory for context, recollective memory or associative encoding and

they reveal activations in MTL (Davachi et al., 2003; Davachi & Wagner, 2002; Sommer

et al., 2005), PFC (Cansino, Maquet, Dolan, & Rugg, 2002; Henson et al., 1999) or both

(Brewer et al., 1998; Kensinger et al., 2003) that are linked to subsequent successful

associative, source or contextual memory performance.

The authors of these papers have proposed different processing roles for these

regions (and subregions within them) that contribute separately to the formation of

contextually bound, episodic memories. For example, Davachi and colleagues (Davachi,

2003) have isolated subsequent memory effects in hippocampus, and perirhinal, and

parahippocampal (PHC) cortices. Importantly, the activated regions were dissociated by

task (item recognition and source memory). Greater activation of hippocampus and left

PHC at study was associated with accurate item recognition accompanied by correct

source memory than with accurate item memory alone, but not with successful versus

unsuccessful item recognition. Greater perirhinal cortex activation, on the other hand,

was associated with correct recognition (item alone and item + source) than for missed

items, but not with accurate source memory versus memory for item without source.

Finally, Davachi and colleagues found that two regions of anterior left inferior prefrontal

cortex were activated more during encoding of items for which source was subsequently

correctly identified than for encoding of items that were subsequently recognized without

recollection of source. Similarly, Davachi and colleagues (2002) found subsequent









memory effects in bilateral hippocampus for items encoded in a relational encoding task,

but not during rote rehearsal.

In another study of the MTL and relational processing, Bar and Aminoff (2003)

used fMRI to explore the recognition of strongly contextually identified objects (e.g.,

hardhat) with the recognition of items that have only weak contextual associations (e.g.,

fly). They found that portions of the parahippocampal cortex (parahippocampal place

area or "PPA") and retrosplenial cortex, areas previously identified in spatial processing

and episodic encoding were activated more by recognition of strongly context-bound

objects than objects that have only weak contextual associations. Moreover, they

distinguished between anterior and posterior portions of the parahippocampal cortex that

were associated more with non-spatial context recognition and with place-specific

context recognition, respectively. They concluded that this PHC/RSC network might play

a role in the formation of episodic memories by inputting to the hippocampus familiar

associations established through experience (e.g., "which objects belong in a kitchen").

They speculated that this information is subsequently used by the hippocampus to

represent specific instances (e.g., "which objects belong in my kitchen") of this

knowledge (citing Buckner, 2000). Taken together, these studies provide substantial

evidence that MTL structures play an important role in the relational processing of verbal

and visual pictorial stimuli for later subsequent retrieval of those relations. Bar and

Aminoff have proposed that well-established general associative knowledge might be

represented in a PHC/RSC network that is subsequently input to the hippocampus for

participation in episodic encoding processes. The studies by Davachi and colleagues

suggest that encoding of certain contextually related-information (i.e., processes engaged









during a verbal stimulus' original presentation) relies on different neural substrates (e.g.,

PHC/RSC) than the encoding of other relational information (e.g., semantic relations

among verbal stimuli (hippocampus). These studies do little, however, to clarify whether

the associative processes engaged by these different neural systems are mutually

exclusive, the same or partially overlapping.

Henson and his colleagues (1999) and Brewer and his colleagues (1998) used

Tulving's Remember/Know procedure to assess the phenomenological state associated

with retrieval of old information. By sorting hemodynamic records at encoding that were

associated with subsequent Remember or Know responses, the investigators attempted to

measure indirectly the neural correlates of encoding associated with recollective or non-

recollective states of recall (Henson et al., 1999). Although they found prefrontal

subsequent memory effects associated with associative encoding, use of the

Remember/Know technique may have confounded the subsequent memory effects

associated with Remember and Know responses with the strength of item memory

(Cansino et al., 2002). For example, in the study by Henson and colleagues, the

procedure may have produced fewer Know than Remember hits and greater Know than

Remember false alarms. If this is the case, Know responses may have represented

guesses more than veridical memory responses.

In an effort to measure the phenomenological state at retrieval more directly,

Rugg and his colleagues (Cansino et al., 2002) used a paradigm similar to that employed

in ERP and fMRI studies of source memory. Cansino et al. had participants make

animateness judgments to visually presented colored images. Each image was presented

randomly in one of the four quadrants delineated on the computer screen. Following









encoding of the objects, a recognition phase was presented and participants pressed a

button to indicate New or, if Old, a button corresponding to the location where the image

had been presented. Cansino and colleagues found subsequent memory effects associated

with associative encoding in right lateral occipital and left prefrontal cortex, among other

areas. Consistent with earlier findings of Rugg and colleagues (Otten et al., 2001; Otten

& Rugg, 2001) they argued that subsequent memory effects represent a subset of the

neural activation required for encoding in any given task. They also claimed that the

subsequent memory effects reflect the relatively greater semantic and perceptual

processing received by certain items. Cansino and colleagues speculated on the

relationship between perceptual and semantic processing contributions to the subsequent

memory effects. They suggested that the perceptual and semantic processing may have

contributed independently to the subsequent memory or effects or, alternatively, greater

perceptual processing may have been mediated by the occipital cortex and fed into the

prefrontal cortex, allowing for more elaborated and, thus, better remembered, memory

traces (Cansino et al., 2002). Interestingly, Cansino and colleagues failed to obtain

subsequent memory effects in MTL, consistent with previous null findings by Rugg and

colleagues (Otten et al., 2001; but see Otten & Rugg, 2001). They speculated that both

Remember and Know responses may have reflected relatively high levels of hippocampal

encoding related activity, or that the null finding simply reflected a lack of statistical

power sufficient to detect such activity.

Finally, Kensinger and colleagues (2003) also measured indirectly participants'

recollective state associated with memory for visually encoded words that were given

semantic judgments ("abstract" or "concrete"). In an accompanying behavioral study,









participants made semantic judgments to visually presented words while performing

either an easy or a difficult auditory discrimination task. Subsequently, participants were

given a memory test and were required to make Remember or Know responses to words

judged "old." Kensinger and colleagues found a significant effect of distraction task

(Easy vs. Hard) as well as an interaction between distraction task and memory strength

(Remember vs. Know). They concluded that the task manipulation altered the type of

memory trace formed and used the distraction task, (followed by a yes-no recognition

task, also performed under distraction), as the independent variable in the imaging

experiment. Kensinger and colleagues found subsequent memory effects in bilateral PFC

and left MTL. However, in left PFC, these effects were for items encoded only under

easy distraction, whereas right PFC subsequent memory effects were obtained for

encoding under both easy and difficult distraction conditions. Likewise, PHC activation

predicted subsequent memory performance under both distraction conditions whereas left

anterior hippocampal activation predicted subsequent retrieval only for items encoded

under easy distraction. The investigators concluded that the formation of detailed,

contextually rich memory traces depends on activation of the left PFC and left anterior

hippocampus. They also concluded that the formation of contextually rich, detailed traces

depends on the activation of a subset of the neural processes activated by successful

encoding generally.

Habib and colleagues (2003) recently reevaluated the Hemispheric Encoding and

Retrieval Asymmetry (HERA) model proposed by Tulving and colleagues (Nyberg,

Cabeza, & Tulving, 1996; Tulving, Kapur, Craik, Moscovitch, & Houle, 1994). They

concluded that existing PET and fMRI data still support the conclusion that the left PFC









shows greater activation in encoding tasks (relative to retrieval tasks) than the right PFC.

Conversely, the right PFC shows greater activation during retrieval tasks (relative to

encoding tasks) than the left PFC. They asserted that such a process-specific

lateralization could co-exist with the material-specific (e.g., verbal vs. non-verbal

materials) lateralization, observed by a number of investigators. Habib and colleagues

reiterated the notion that the preferential left PFC activation during episodic encoding is

likely to be associated with semantic processing of incoming and on-line information.

Recent work involving transcranial magnetic stimulation (rTMS) supports HERA

showing disruptive effects to encoding by application of magnetic pulse trains to left PFC

and degradation of retrieval by application of magnetic pulses to right PFC (Rossi et al.,

2001).

Thus, there is conflicting evidence as to whether the right prefrontal cortex is

involved in associative encoding. There are few fMRI studies of long-term memory

studies that show right PFC subsequent memory effects, and the HERA model accords

the left PFC a predominant role in LTM encoding. In contrast, a few ERP studies

(Kounios et al., 2001; Schott et al., 2002; Weyerts et al., 1997) have identified subsequent

memory effects at right prefrontal electrodes for associative encoding. Furthermore, the

right PFC finding in these studies is supported by fMRI findings of Prabhakaran and his

colleagues (2000) and Johnson and her colleagues (Mitchell et al., 2000a) in working

memory studies of object and location binding. The role of the processes underlying these

effects is still controversial.

The Present Investigation

One challenge, therefore, is to (1) examine the creation of memorable episodes by

identifying those cognitive and neural processes that link events and their contexts; and









(2) determine whether or not those processes are consistent with the current theoretical

accounts of the relationship between working memory and episodic memory. The

experiments presented in this dissertation are an attempt to tackle a piece of that

challenge. The first purpose of this dissertation is to test whether associating concrete,

highly imageable items and complex contextual scenes into events (e.g., elephant at an

intersection) in working memory is a key component of successful long-term episodic

memory performance. The second purpose of this dissertation is to test whether

successful long-term memory "binding" produces a temporally and topographically

unique electrocortical "signature" during working memory that distinguishes it from

unsuccessfully bound items and contexts as well as from unbound, but remembered,

items and contexts.














CHAPTER 2
EXPERIMENT ONE

As described above, there is substantial evidence that the neural underpinnings of

the relational binding (both episodic and associative) which subserve long-term memory

can be observed by two basic strategies: a comparison of encoding tasks that do, and do

not, require some sort of relational or contextual encoding between elements; and a post-

hoc sorting of encoding events that do, and do not, result in subsequent memory for

contextual versus item information. But there is little consensus about the conditions that

produce such a binding "fingerprint." Moreover, the details (timing and topography) of

the fingerprint are even less obvious, in large part due to the inconsistency with which

subsequent memory effects are detected.

One likely source of this inconsistency is the wide variety of encoding tasks, on

the one hand, and ways of assessing memory, on the other, that have been used by

different researchers. In some cases, for example, the relational task is qualitatively

different from the nonrelational task on dimensions other than relational processing as

such. More challenging memory tasks (e.g., recall) and test responses that reflect more

elaborative memories (e.g., "Remember" vs. "Know" responses) more often produce

subsequent memory effects than simple yes no recognition, but at the same time, are

themselves complex enough to introduce constructive and inferential processes at

retrieval that may interact with any encoding processes being studied.

In each of the present experiments, a simple yes-no recognition task was used to

minimize the role of retrieval factors in any subsequent-memory effects. As importantly,









a task was adopted in which the encoding processes were as similar as possible, while

some groups attempted to integrate the two stimuli, and others did not.

The materials. Concrete, imageable words, as well as pictures, have been shown

to elicit Dms, whereas abstract words and symbols are less likely to produce subsequent

memory effects. The requirements of the stimulus materials were three-fold. First, they

had to consist of familiar, integrable, item and spatial location pairs. Second, they had to

produce sufficient levels of recognition failure during a paired recognition test. Third, in

contrast to the materials of Craik (1989), each item (and pair) had to be unique, to avoid

potential cross-pair interactions during either encoding or retrieval that could complicate

the subsequent-memory analysis. As discussed below, the stimulus materials selected

fulfill each of the foregoing criteria.

The task. An experimental task in which participants were required to associate

(bind) unique spatial locations and objects was contrasted with a condition in which

participants would be required to process the same materials in the same way without

integrating the two. Following Bower (1970), a task was designed to promote integrative

encoding in a paired recognition task.

Expected Results.

Following Bower (1970), we expected that the overall pattern of results for paired

recognition performance between the two groups (Bound and Separate) would reflect

better recognition of previously presented pairs by the group that formed integrated

images (Bound) than by the group that maintained separate images (Separate). Overall,

we expected the between-group manipulation to produce similar ERPs at encoding due

to the identity of the stimulus materials and the similarity of the experimental conditions

between the two groups. We anticipated that distinctive between-group ERPs would most









likely be reflected to the second word at which point the integrative processing in the

Bound, but not the Separate, condition could begin. Alternatively, however, task-related

ERP contrasts could be associated with attentional or other "set" differences. These

differences could show up as broader differences in the ERP patterns encompassing the

first word, and even the pre-stimulus interval. However, we predicted that, in addition to

differences in the second word interval, the task manipulation would be most likely to

reflect differences late in the first word interval, as participants prepared for the second

word. In any event, we expected ERP task differences to be minimized by our decision to

manipulate the task as a between-subjects factor, thus producing significantly more

variability for it than for the within-subjects factor (memory performance).

While we expected the overall pattern of results between the two groups to be

similar, primarily yielding differences following the second word where integrative

processing would be reflected in Bound but not Separate ERPs we anticipated that the

subsequent memory effect differences (Dms) would reflect the relative role of the first

and second words in associative encoding.

In this regard, our method provides a unique advantage over previous paradigms

that have used a limited number of associative possibilities (male/female voice; limited

spatial locations, etc.). Thus, the "binding" or relational processing could only take place

upon presentation of the second word. Consequently, we expected a unique signature in

the Dm to the second word for cases in which paired recognition failure was a result of

unsuccessful binding of the object and location. Given that several of the few

experiments in which Dms were produced for associative or elaborative processing

yielded frontal Dm effects (Duarte et al., 2004; Fabiani et al., 1990; Fernandez et al.,









1998; Karis et al., 1984; Kounios et al., 2001; Mangels et al., 2001; Schott et al., 2002;

Weyerts et al., 1997) we anticipated that associative subsequent memory effects would be

produced to the second, but not the first, word, at frontal electrode sites, with right frontal

locations possibly showing greater effects than left. Moreover, we expected these

differences to begin later in the interval (-1000 ms after word 2 onset) and be sustained

throughout the interval.

We also predicted that a set of frontal transient Dms, similar to those observed by

Mangels and colleagues (2001) and Duarte and colleagues (2004), beginning as early as

400 ms following word onset, might be produced to both the first and second words.

Based on previous findings, these earlier Dms would likely be either left-lateralized or

bilateral.

Method

Participants

Seventy-four undergraduates (47 females) at the University of Florida participated

in this experiment. Additionally, 37 undergraduates participated as pilot participants

during development of the tasks and materials. Twenty-two of these pilot participants

were used to ensure that Bound and Separate encoding produced different levels of

subsequent memory performance and the stimulus delivery and data recording program

was operating as anticipated. Fifteen additional pilot participants were used to test the

effectiveness of an alternative instruction directing participants to respond"old" only if

they were sure that they had seen the pair as presented before. Participants were

randomly assigned to the two experimental groups. Participants received credit toward an

introductory psychology course requirement. Of the 74 participants who began the

experiment, three left without completing the test phase so neither behavioral nor EEG









data were useable for those participants. 11 other participants had too few (< 10)

incorrect responses to the memory test portion to permit construction of reliable (based

on visual inspection) waveforms. Finally, as discussed in the Results section, of the

remaining 60 participants, various technical and signal-to-noise problems prevented

analysis of another 21 EEG datasets.

Materials and Apparatus

Stimulus display and response recording

The entire experiment was administered in a small, dimly lit room (approximately

5.5' x 6.5') on a personal computer using a conventional CRT monitor with a screen size

of approximately 13" measured diagonally. Participants were seated about 24" from the

monitor. A program written in the Delphi programming language (Borland Software

Corp.) controlled stimulus presentation and recording of behavioral responses.

Participants viewed stimulus items in the middle of screen and responded to stimulus

events by using a standard two-button mouse. During the recognition phase, participants

made affirmative recognition responses by pressing the left mouse button and negative

responses using the right mouse button.

EEG recording

Electroencephalographic activity (EEG) was recorded using a standard elastic cap

(Electro-Cap International, Inc.) with 13 embedded tin electrodes placed in standard 10-

20 system (Jasper, 1958) locations (Fpl, Fp2, F3, F4, FT7, FT8, Cz, TP7, TP8, P3, P4,

01, 02). The cap was linked to a set ofbioamplifiers (SA Instrumentation Co.). Data

were filtered (high pass 0.01 Hz; low pass 50 Hz), amplified 50K, digitally converted

using National Instruments analog to digital converter and stored for subsequent off-line

analysis. In addition to the scalp-recorded EEG, horizontal electro-oculogram (hEOG)









was recorded with a pair of tin electrodes placed on the outside canthus of each eye. A

second pair of tin electrodes placed above and below the left eye recorded vertical EOG

(vEOG). The gain for both EOG channels was 20K. A third pair of tin electrodes was

placed on the skin above the mastoid bone behind each ear. During recording, Cz was

used as a common reference for all other scalp and mastoid sites. During subsequent data

analysis, the EEG was rereferenced to the average of the left and right mastoid sites. The

sampling rate throughout the experiment was 100 Hz.

Stimulus materials

The stimulus materials consisted of 360 words drawn from various sources

(Battig & Montague, 1969, Rubin & Friendly, 1986) and experimenter-generated items.

These words were evenly divided between location and object (people, animals,

inanimate objects) words. We reduced original lists of locations (n = 283) and objects (n

= 656) by eliminating rare (e.g., boomslang, oceanographer) or difficult to image (e.g.,

albatross, charlatan) items as well as obvious synonyms (e.g., physician, doctor; ocean,

sea) or category-exemplars (sheep lamb; spider tarantula). The resulting lists were

submitted to the MRC Linguistic database

(http://www.psy.uwa.edu.au/MRCDataBase/uwamrc.htm) to obtain normative data on

written word frequency, imageability, meaningfulness, and concreteness. Outliers ( 2

s.d.'s from jp) were excluded. The final list of 160 pairs consisted of various objects,

including people/occupations (n = 41), animals (n = 47), tools (n = 14), vehicles (n = 14),

toys (n = 10), weapons (n = 13), musical instruments (n = 8) and furniture (n = 13). These

people, animals and inanimate objects were paired randomly with locations and were

manually examined to eliminate pairings with obvious pre-experimental associations









(bartender bar; clown circus). Once was the list of 160 pairs was generated, one-half

of the pairs were re-sorted to produce a test list consisting of 50% intact and 50%

rearranged pairs. In the test list, the intact and rearranged pairs consisted of

approximately the same number of object types described above.

Design

The design for the study phase was a single factor (Encoding Task: Bound versus

Separate encoding of the words in a pair) between-subjects design. During the test phase,

all participants were given the paired recognition test.

Procedure

After giving informed consent to the procedure, participants were fitted with the

electro-cap and other electrodes. Generally, impedances, measured against Cz, were kept

under 8K Ohms. Once participants were prepared for EEG recording, the experimenter

gave an overview of the experimental procedure (i.e., "You'll be viewing words

presented on the screen and generating mental images of each of the words. You'll rate

the ease with which you generated the image. Following the image generation task, you'll

be given a memory test for the words."). Following this instruction, participants read, on

the screen, a more detailed set of instructions regarding the study phase.

In brief, all participants were instructed that they would view pairs of words, each

consisting of, first, an object (person, animal or object) and, second, a location.

Participants were instructed to generate and maintain a "rich, vivid" visual image of the

word's representation upon its presentation. Participants were instructed to rate,

following the location word, the ease with which they generated the imagess. For this

purpose, participants were shown, on the screen, four clickable radio buttons captioned

with a rating scale (Really easy, Somewhat easy, Somewhat difficult, Really difficult).









Participants were instructed to make this judgment relatively automatically, giving their

"first impression." Instructions between the two (Bound/Separate) groups differed only

regarding the generation of the imagess. Participants assigned to the Bound group were

instructed to generate a mental image in which the first word (object/person/animal) and

the second word (location) were integrated into a single image or scene. They were

instructed to make this "scene" as visually rich and vivid as they could. Participants

assigned to the Separate group were instructed to maintain the image of the object and the

location separately. Specifically, it was suggested that participants "place the image of

the [first word] on the far left side of your "imaginary visual field and the image of the

[second word] on the far right side of the imaginary visual field." Virtually all

participants expressed comprehension of this instruction. The experimenter eliminated

any confusion with further explanation.

As displayed in figure 2-1, the study phase, and each trial, commenced with a

fixation cross, displayed for 300 ms, followed by a 700 ms post-fixation interval during

which the screen was blank. Following the post-fixation interval, participants viewed

words, presented singly in 28-point Arial font, each displayed for 500 ms with a 2500 ms

interstimulus interval (ISI). EEG recording began 100 ms following offset of the fixation

cross, and hence 600 ms prior to onset of the first word of each pair, and continued

through 2600 ms after onset of the second word. Following the second word ISI, the

ratings buttons were displayed until the participant selected one. The intertrial interval

(ITI) between this mouse press and initiation of the next trial was fixed at 1000 ms.

Following the presentation of each forty consecutive trials, the program paused for a









participant-paced rest. Most participants, however, continued the experiment without a

significant rest period. At the end of 160 trials, the program stopped.


O L
S700 ms | S 2500 ms 2500 ms // ITI
300 ms
500 ms 500 ms //1000 ms

Figure 2-1. Schematic representation of a single trial during the study phase in
Experiment 1. is fixation cross, "0" and "L" are presentation of object and
location words, respectively. "//" is the self-paced, response interval during
which participants indicated the ease of image generation. "ITI" indicates
intertrial interval. Light colored line is EEG recording interval.



The experimenter then engaged the participant briefly in unrelated conversation to

prevent overt rehearsal of the last few presented items and to give the participant a brief

break (- 5 minutes) from the task. Thus, the mean latency from a pair's appearance in the

study phase to its appearance in the test phase was approximately 40 minutes.

The timing and appearance of stimulus items in the test phase was the same as in

the study; viz, a fixation cross, an object word, and then a location word were presented.

However, the interstimulus interval between object and location words was reduced to

1500 ms and only a single break (rather than the three in study phase) was provided.

Participants in both conditions (Bound/Separate) were instructed to indicate, using the

left mouse button for affirmative responses and the right mouse button for negative

responses, whether the OBJECT-LOCATION pair had been shown earlier (yes left

mouse button) or whether the pair consisted of an object and location that had been

paired earlier with other items (no right mouse button). Participants were instructed to

make these responses as quickly as possible due to the measurement of response times.

Following the old-new mouse presses, the monitor displayed a three-choice alternative









("The second word presented," "Another word not presented," "No other word") to

which participants were instructed to respond. The participants were told to respond

according to their reaction to the first (object) word of the trial. That is, if presentation of

the first word immediately elicited a word, participants were instructed to click on one of

the first two choices (depending on the second word that was presented). On the other

hand, if the presentation of the first word failed to elicit another word, participants were

instructed to select "No other word."

Results

Behavioral Data

As expected, participants in the Bound condition were better at discriminating

intact pairs from rearranged pairs (hits: M = 59.1, SE = 1.66; false alarms: M = 9.0, SE =

1.33) than participants in the Separate condition (hits : = 46.5, SE = 1.87; false alarms: M

= 26.7, SE = 2.54) during the test phase, t(37) = 11.50, p < .001. Performance differences

between the two groups were not attributable to speed-accuracy tradeoff; the groups did

not differ in their correct response times to intact pairs (Bound: M = 1341 ms, SE = 69;

Separate: M = 1337 ms, SE = 64, p >.10). These findings, coupled with participants'

post-hoc comments to the experimenter, suggest that they were, at least overall,

generating and maintaining integrated or separate images in the two conditions as

instructed.

EEG Data Preprocessing

Prior to averaging, the raw EEG data were inspected manually for the presence of

blinks and other artifacts on a trial-by-trial basis. In the next phase, EEG for each trial

was digitally low-pass filtered at 30 Hz for smoothing, and the mean amplitude set to

zero for that trial to correct for baseline shifts. During this phase, trials marked as









containing artifacts were subjected to a componential analysis and reconstruction process

to attempt to remove blink and other artifacts from the waveforms, using the Independent

Component Analysis (ICA) procedures and routines from the EEGLAB toolbox

(Delorme & Makeig, 2004), and a locally written Matlab script. Typically, one or two

components were clearly identifiable with blinks/artifacts, and successfully removed. A

maximum of four components (out of 16, limited by the number of recording sites) were

allowed to be removed before rejecting the trial as unusable.

ERP waveforms

ERPs elicited by word pairs in the study phase were, as noted, computed on the

basis of participants' responses on the subsequent paired recognition test. Data from the

study phase were sorted according to the following test phase responses: "old" responses

to intact pairs were classified as IC (Intact Correct); "new" responses to intact pairs were

classified as IE (Intact Error); "old" responses to rearranged pairs were classified as RE

(Rearranged Error); and "new" responses to rearranged pairs were classified as RC

(Rearranged Correct)). As noted above, a large number of participants' data were

excluded from analysis to be presented here. Thirty-nine participants (19 Bound/20

Separate) provided data for the analysis described below. Each participant's averaged

data were then averaged with other participants' averaged data to calculate grand

averaged data for each class. Baseline adjustment (setting the mean amplitude during a

prestimulus interval for each condition equal to zero) was not performed, since it was

possible that important ERP differences between various classes, including between pairs

that were subsequently recognized and those that were subsequently forgotten, might be

reflected in the prestimulus interval.









ERPs to the Bound condition from the 13 scalp electrode sites are presented in

Figure 2-2 below. The black line indicates ERPs for subsequently recognized intact pairs

(IC), and the grey line indicates ERPs for intact pairs erroneously identified as re-paired

(IE). Over the course of the 6200 ms interval, early event-related potentials (N100, P200,

N400) to each word are clearly visible across most channels, followed by a broader later

positivity around 600 ms, and a slow wave whose direction and magnitude differed

widely across channel, and sometimes continues through the end of the epoch for each

word.

Differences as a function of subsequent memory performance can be seen later in

the interval during the slow wave epoch, which appear greatest bilaterally at the frontal

electrodes. For example, at the frontopolar (Fp) electrode sites, a sizeable difference

between IC and IE traces is noted beginning at about 1600 ms from the beginning of the

interval (1000 ms post-first word onset). Interestingly, in this case, correctly recognized

pairs show a greater negativity during the interval than do subsequently forgotten pairs.

The difference lasts until about 4000 ms when a significant reversal is noted, with IE

becoming more negative than IC. A small difference between IC and IE is also visible

during the earliest part of the interval (prestimulus through word one presentation),

particularly at the frontal electrodes.

The waveforms to the Separate condition (Figure 2-3) show marked contrasts to

those from the Bound condition. There is little visible difference between the IC and IE

waveforms. The large, slow wave differences between IC and IE that are present

beginning in the 1000 ms range in the Bound condition are absent in the Separate

condition. As with the Bound condition, however, activity at the frontopolar sites is









distinguished, for both IC and IE responses, from the activity at all other locations by a

positive, slow change beginning about 1600 ms from the beginning of the interval. Other

locations are characterized either by a negative change during the interval, or by no

change. Additionally, the Separate waveforms are distinguished from the Bound

waveforms, especially at frontopolar sites, by the presence of two distinct positive peaks

following the presentation of each word. The first of these peaks would appear to be a

P200 to the onset of the words. The second positive peak is close to 200 ms after the

offset of the word (after 500 ms) and may well be an offset response to the offset of the

stimulus (see, e.g., Janata, 2001).

Finally, a comparison between the IC responses to the Bound and Separate

conditions is presented in Figure 2-4. Waveform differences between the IC responses

that are evoked by the different task demands are apparent, if small. The second half of

the last interval (-5500 ms) appears to show differences between the two correct

responses in the right hemisphere. In addition, FT7 appears to reflect a difference

between the IC responses that mimics, temporally, the differences observed at the

frontopolar sites between IC and IE responses in the Bound condition. In addition, early,

prestimulus differences between Bound and Separate IC responses are similar to those

accompanying IC-IE responses in the Bound condition.

Statistical analysis of waveforms

Visual inspection of the grand averaged waveforms led us to identify windows of

interest for subsequent statistical analysis. Subsequent memory effects were quantified by

measuring mean amplitudes during each of ten successive latency intervals relative to

onset of each word ([wl] -600 0, 0 300, 300 600, 600 1200, 1200 2600; [w2] -400
















F3




SFT7

=, II .a,


TP7


' 1 1 1 I I I I" w '' I


1 a


u YV 0' 1 0 0 r 2 0 0 1 0 0 0 2




-1-
0 1000 2000 0 1000 2000


Time since onset
of first word (ms)


Fp2




' F4





FT8









TP8









\ 02

k Imu i,,, AM^M^


Time since onset
of second word (ms)


Figure 2-2. ERPs to two words during study phase later shown as intact pairs during test
in Experiment 1, Bound Encoding group. Bars indicate onset and offset times
of the words. Black waveform is for pairs later correctly recognized as intact;
grey waveform is for pairs later misrecognized as repaired.















F3




~ivim


TP7





P3


P4


02


-1 I--


0 1000 2000
Time since onset
of first word (ms)


0 1000 2000
Time since onset
of second word (ms)


Figure 2-3. ERPs to two words during study phase later shown as intact pairs during test
in Experiment 1, Separate Encoding group. Bars indicate onset and offset
times of the words. Black waveform is for pairs later correctly recognized as
intact; grey waveform is for pairs later misrecognized as repaired.


Fp2



F4





FT8



























TP7
TP8



i P3

....Ik i .. .,


0-1
0 1000 2000 0 1000 2000


Time since onset
of first word (ms)


Time since onset
of second word (ms)


Figure 2-4. ERPs to two words during study phase later shown and correctly recognized
as intact pairs during test in Experiment 1. Bars indicate onset and offset times
of the words. Black waveform is for the Bound Encoding group; Grey
waveform is for the Separate Encoding group.


Fpl



F3F





/ I I


Fp2



F4




FT8
fay/l, ,--...--,









- 0, 0 300, 300 600, 600 1200, 1200 2600 ms). Initial analyses were conducted by

running, for each condition (Bound and Separate), an analysis of variance (ANOVA) on

the mean interval amplitudes of ten "windows" that comprised the total 6200 ms trial

interval to test whether they differed across the selected factors. In addition to the

subsequent memory factor, two regional EEG factors were created from the 12 lateralized

sites, excluding Cz. The ANOVAs thus utilized a 3-factor (Performance: Hit/Miss;

Hemisphere: Left/Right; Anterior-Posterior ("AntPos"): 6 levels of electrode site), 2 x 2 x

6, within-subj ects design. In addition, a third set of ANOVAs was run to compare the

mean amplitudes produced by correct responses to intact pairs between the Bound and

Separate conditions in each window. Thus, this ANOVA was a 3 factor (Task:

Bound/Separate; Hemisphere: Left/Right; Anterior-Posterior: 6 levels of electrode site), 2

x 2 x 6, mixed design. In all analyses, the Greenhouse-Geisser correction (E) was applied

for violations of the assumptions of sphericity for comparisons involving two or more

degrees of freedom.

Bound condition

As presented in Table 2-1, in the Bound condition, there were two intervals that yielded

significant interactions involving subsequent memory. In the long interval (1200 2600

ms) between presentation of the first and second word there was a significant interaction

between Performance and the AntPos factor, F(5, 90) = 3.68, p = 0.013, a = .696,

reflecting the larger subsequent memory effects across the frontal electrodes than at the

more posterior sites. A second ANOVA which analyzed only the differences between the

right and left Fp electrodes (excluding the other electrode sites) revealed no differences

between the hemispheres (p > .10). Differences similar to those found to the first word

were observed, in the form of a significant Perfx AntPos interaction, F(5,90) = 5.671, p =









0.005, F = .452, in the comparable interval (1200 2600 ms) to the second word. This

interaction reflects, again, the larger Dm in the frontal sites. However, unlike in the Dm

to the first word, the amplitudes of IC items are more positive than of IE items. Finally,

there was a main effect of Performance

Table 2-1. Time Intervals in Experiment 1 during which Amplitude Differences were
Significant
Factor
Interval Perf Perf x AntPos Perf x Hem
[w]-600 0 S
0 -300
300 600 H
600- 1200
1200 2600 B
[w2]-400 0 S
0 -300
300 600 B
600- 1200
1200 2600 B


Note. "B" = significant Dm effects in Bound task, "S" significant Dm effects in Separate
task, "H" = significant differences in Bound Separate correct recognition (Hit)
comparison. For all comparisons, a = .05.


in one of the early intervals (300 600 ms) to the second word in which the IE items were

more positive than the IC items. Thus, quantification of the mean amplitudes through the

various time windows that make up a single trial revealed subsequent memory effects

that were larger toward the frontal part of the scalp than toward more posterior regions.

Moreover, these effects appeared at approximately the same latency following the onset

of each word. Finally, a more generalized Dm was observed early after the onset of the

second word.









Separate condition

The Separate condition yielded significant Dm effects at two intervals, neither of

which overlapped with the effects observed in the Bound condition (Table 2-1). A

significant Perfx Hemisphere interaction was observed in the prestimulus interval (0 -

600 ms), F(1, 1) = 6.162, p = 0.023, F = 1.0, reflecting the greater right-sided Dm in the

prestimulus interval. A second Dm effect was observed in the -400 0 ms interval, just

prior to onset of the second word. This effect did not interact with either of the other

factors, F(1, 19) = 4.862, p = 0.040, s = 1.0.

Bound vs. separate conditions

The final analysis run in the first experiment was to compare the IC responses

produced in the Bound and Separate conditions. A significant interaction between the

Task factor and AntPos occurred in the first word interval at 300 600 ms [F(5,90) =

4.332, p = 0.021]. This interval captures the temporal window during which the N400,

and P300 components typically are observed. There is little consistency reflected in this

particular interaction; Differences are noted between frontal and posterior sites and the

effects of task are opposite to one another between the two.

Discussion

Following Bower (1970), participants showed impaired recognition performance

after encoding items in the Separate, compared to the Bound, task instructions. This

shows that even with the co-presentation of a given pair of words, and their high

imageability, participants in the Separate task were, to a great degree, capable of keeping

the two items separate and distinct, as instructed. Moreover, when queried, all

participants in the Bound condition reported being able to "bind" the object and location

into a single image. Likewise, all participants in the Separate condition reported being









able to generate and maintain object and location images separately. Occasional

participants in the Separate condition reported that "on a couple of trials" they "couldn't

help putting (binding) the images together." These reports were sporadic and no

participants reported this "problem" to have occurred on more than 2 or 3 of the 160

study pairs. Thus, the instructions to generate and integrate and object and location in the

Bound Condition (and generate and maintain unique images in the Separate condition)

can be assumed to have acted as intended.

As expected, we obtained reliable subsequent associative memory effects using a

task that places demands on associative encoding. Participants were instructed, in both

the Bound and Separate conditions, to generate mental images that were as clear and

visually rich as possible. We anticipated that the instructions in both conditions would

promote extensive cognitive effort by participants that would yield observable differences

between later forgotten and later remembered pairs. We hypothesized that successful pair

recognition would depend on (a) adequate processing of each element (object and

location) of the pair, and (b) the creation and maintenance of a link between the two.

Thus, the strongest prediction that we made was that the Bound condition would reveal

transient frontal Dm effects that would appear in response to each word, as well as a

later-appearing Dm in response to the integrative demands of the task. On the other hand,

we predicted that the Dm effects for the Separate condition would follow from the lack of

integrative instructions in the task. We expected any subsequent memory effects to reflect

the establishment of strong memory traces for the objects and locations individually and

there should be no late, frontal, second-word-only effects associated in the Bound

condition with integrative activity.









The Bound condition ERPs revealed a striking pair of subsequent memory effects

that occurred with the same latency following onset of the first word and the second word

(1200 2600 ms) and the same largely symmetric frontal topography. Notably, however,

the relative polarity of the difference was the opposite between the first and second word

late Dm effects. That is, the Dm to the first word was negative (Bound < Separate) and

the Dm to the second word was positive (Bound > Separate). Although the negative Dm

has been identified in only one other study (Dm for name recognition Guo, 2005), we

can speculate, in the present case, why it may have occurred. Our particular paradigm has

some similarities to a CNV-producing S1-S2 (Go No-Go) paradigm. Therefore, as noted,

we expected to find a "slow wave" component, analogous but not identical to the CNV,

between the first and second word. We assumed that this slow wave might show

significant amplitude shift. In the current study, we noted that this slow wave activity

appears over several electrode sites. We pointed out, however, that the shift was in a

positive direction only at the Fp sites. If the positive change at the Fp sites reflects a

preparatory, maintenance-like state (similar to the E-wave of the CNV), then the greater

negativity of the hits (relative to the misses) at the Fp sites might reflect ongoing

processing of the first word and/or preparation for the second word that underlies

successful binding. Conversely, the greater positivity of the hits in the late interval

following the second word is more typical of a Dm effect and, perhaps more expected, in

light of the fact that the final word of the pair has been presented and the participant need

anticipate no further events in the trial. Thus, the later (1200 2600 ms) second-word

Dm could reflect both the processing of the second (location) word and the establishment

of a successfully integrated trace.









We also noted the presence of a third, widespread Dm in the Bound condition,

occurring just after the presentation of the second word (at 300 ms) but not after the first

word, which suggests that the cognitive processing associated with the late (1200 ms)

second-word Dm may not have reflected identical processing to that during the earlier

(1200 ms), first-word Dm. This difference is reflected in a greater positivity for

subsequently missed pairs than for subsequently recognized pairs. It could be that the

early, second word Dm reflects the initiation of cognitive mechanisms to respond to the

integrative task demands, or the assessment of the presented location as appropriate for

integration. Although it is not entirely clear, it appears that the differences at this interval

are from a reduction in the N400, an enhancement of the P300, or some combination of

the two in the later-missed items.

Unexpectedly, however, the Separate condition also revealed a temporally similar

(-400 ms) widespread second-word Dm effect. The timing is especially surprising. Two

possibilities seem plausible. One possibility is that, despite instructions to keep the

images of location and object separate, participants were, in fact, integrating the two.

Thus, the Dm at the end of the first word interval could reflect preparatory activity for

binding the just-presented object word to the to-be-presented location word. This

conclusion, however, seems unwarranted for at least two reasons. First, participants in the

Separate condition reported that they were successful in generating and maintaining

separate images for the object-location pair. The significantly poorer recognition rates for

the Separate condition further supports their contention. Second, there is no ERP

evidence, in the form of a Dm effect, that participants are doing anything during the

second word that distinguishes remembered from forgotten pairs. An alternative, more









plausible, explanation is that the late, first-word Dm in the Separate condition represents

further processing of the object word or a preparatory attentional shift or disengagement

from the first word in anticipation of the second word. This attentional shift effect would

be expected to be present in the Separate condition, if an adequate trace was established

to the first word, but not in the Bound condition where the object and location are

required to be integrated. In any event, the pattern of Dm effects shows differences in

timing and topography between the two experimental conditions, reflecting the likely

engagement of a different set of neural and cognitive processes that yield success or

failure in each condition.

Finally, the lack of differences between the IC responses in the Bound and

Separate conditions, except along a brief, early 300 ms interval to the first word, is

somewhat unexpected. Our strongest prediction was that the integrative activity, present

in the Bound but not the Separate task, would have discernable effects on the scalp

related ERPs. Given the temporal and topographical differences between the Dms in the

Bound and Separate conditions, we expected that the correctly recognized, intact pairs

would likewise show differences between the Bound and Separate conditions. The lack of

differences between the groups may be attributable togreatervariability between the

groups than the within-group variability in the Dm comparisons.

Nevertheless, the pattern of subsequent memory differences associated with

paired recognition of objects and locations provides important evidence that areas of the

prefrontal cortex have an important role in establishing the relationship between the

items. This electrophysiological response occurs following both the first and second

words and is consistent with a variety of accounts (e.g., Craik, 1989; Hunt & Einstein,






51


1981) of memory encoding suggesting that relational processing has a separate cognitive

basis from item processing. In the second experiment, we used the same materials and

instructions but gave participants an item recognition test to examine the event-related

potentials associated with subsequent memory success.

We anticipated that, if the pattern ofERP Dms we identified in the first

experiment were associated with relational encoding, a different pattern of Dm effects

would be present for the item recognition test.














CHAPTER 3
EXPERIMENT TWO

The first experiment demonstrated some physiological evidence of the cognitive

underpinnings of episodic encoding. However, by its design, the experiment left unclear

whether the subsequent memory effects that were identified were those capable of

supporting single item recognition, associative recognition alone, or both item and

associative recognition. The second experiment attempted to isolate the processes that

support subsequent item recognition and contrast these from processes that support

associative recognition. We used the same materials and task at study as in Experiment 1.

However, the subsequent recognition test was for individual words presented as objects

or as locations during the study phase. Memory for objects and locations were tested

separately to enable the creation of ERP records that could be conditionalized on either

object recognition or location recognition.

Expected Results

We anticipated that the Bound versus Separate manipulation would have little

impact on overall performance on the memory test. That is, item recognition would be

relatively unaffected by whether participants attempted to integrate objects and locations

at study. Given that participants in both conditions utilized similar semantic encoding

strategies and were both instructed to make their visual images "as rich and vivid as

possible," we presumed that there would be no difference in item recognition levels.

We expected that the ERPs in the Bound condition, conditionalized on subsequent

item recognition, would be associated with the processing of individual items, not with









the integration of objects and locations. Consequently, we anticipated little difference

between the Bound and Separate Dm effects. Moreover, our design, which tested object

and location recognition separately, enabled us to isolate ERP correlates of subsequent

performance associated with each item of the studied pair. Thus, we anticipated, for

example, that an object recognition test would yield subsequent memory effects

associated with the prior presentation of objects, but not locations. If, as we posited, the

Dms in Experiment 1 were associated with relational encoding, the Dms isolated in the

current experiment should differ from those in Experiment 1 by timing, topography, or

both.

Given the inconsistent findings as to topography of Dm effects in item

recognition, we make no specific predictions about the likely topography of Dm effects in

the current experiment. However, the findings from Experiment 1 and the existing Dm

literature provide some guidance as to the anticipated timing of Dm effects. Given that

Dm effects putatively associated with relational encoding occurred in the long (1200 -

2600 ms post word I/post word 2) intervals in Experiment 1 we anticipated that item-

specific Dm would be associated with earlier intervals and show greater transience.

As in Experiment 1, we expected the difference between the task-specific ERPs

(Bound correct responses vs. Separate correct responses) to be minimized by the

increased variability associated with the between-subjects nature of the comparison. Any

differences should be associated with inter-task "set" differences. Thus, early,

prestimulus differences could be reflected in the task comparisons.









Method

Participants

Fifty-one undergraduates (32 females) at the University of Florida participated in

this experiment. Additionally, 7 undergraduates participated as pilot participants.

Participants were randomly assigned to the two experimental groups. Participants

received credit toward an introductory psychology course requirement or a nominal

payment. Of the 51 participants who began the experiment, one failed to return for the

test phase so neither behavioral nor EEG data were available for that participant. Two

other participants had too few (< 10) incorrect responses to the memory test portion to

permit construction of reliable waveforms. Finally, of the remaining 48 participants,

various technical and signal-to-noise problems prevented analysis of another 13 EEG

datasets.

Materials and Apparatus

Stimulus display and response recording

The study phase portion of the experiment was conducted under the same

conditions and in the same location as the first experiment. The recognition phase (during

which EEG was not recorded), which was held about 24 hours after completion of the

study phase, was held in a brightly lit room not used for EEG recording. The recognition

phase of Experiment Two was delayed after pilot testing indicated that there would be too

few misses for item recognition to obtain interpretable ERPs in that condition. The same

computer program used to display the material in Experiment 1 was used to display

material in Experiment 2.

EEG recording

EEG recording was accomplished using the same parameters as in Experiment 1.









Stimulus materials

The study phase list from the first experiment was used to create a pair of study

phase lists for the current experiment. The 160 object-location pairs were divided into 2

80-pair lists, each list serving, in one case, as study phase items and test phase foils, and,

in the other case, as test phase foils and study phase items, respectively. All of the object-

location pairs were separated to create two pairs of test phase lists (Objects/Locations)

with each participant being shown an object list and its corresponding location list.

Design

The design for the study phase was a single factor (Bound, Separate) between-

participants design. During the test phase, all participants were given item recognition

tasks in which items were presented at the same rate as during Experiment 1. However,

participants were required to respond "Old" or "New" (with same mouse press

arrangement in Experiment 1) to each item, rather than following each pair. In addition,

confidence ratings were obtained following each "Old/New" response. At the completion

of the first 160 object or location recognition test, participants took the remaining (object

or location) recognition test. Upon concluding the recognition test, participants completed

a 32-item questionnaire (VVIQ-R; McKelvie, 2001) on the vividness of their visual

imagery experiences.

Procedure

The study phase procedure was as described for Experiment 1 except that the

participants were shown only one half of the object-location pairs. The remaining 80

pairs served as foils in the item recognition tests. Participants were reminded that they

would return to the lab approximately 24 hours after completing the study phase to take a

memory test and complete a questionnaire on mental imagery.









As in Experiment 1, the study phase, and each trial, commenced with a fixation

cross, displayed for 300 ms, followed by a 700 ms post-fixation interval during which the

screen was blank. Following the post-fixation interval, participants viewed words,

presented singly in 28-point Arial font, each displayed for 500 ms with a 2500 ms

interstimulus interval (ISI). Following the second word ISI, the ratings buttons were

displayed until the participant selected one. At the end of 80 trials, the program stopped.

The experimenter then disconnected the participant from the EEG equipment,

confirmed the following day's appointment, and dismissed the participant.

On the following day, the timing and appearance of stimulus items in the test

phase was similar their presentation in the study phase; viz, a fixation cross, and an

object, or a location, word were presented. However, the interstimulus interval between

object and location words was reduced to 1500 ms and participants were shown either a

list or 160 object words followed by a list of 160 location words or vice versa. List order,

task and stimulus set was counterbalanced between subjects. Participants in both

conditions (Bound/Separate) were instructed to indicate, using the left mouse button for

affirmative responses and the right mouse button for negative responses, whether the

word had been shown earlier (yes left mouse button) or whether the word consisted of

an object or location that had not been shown earlier (no right mouse button).

Participants were instructed to make these responses as quickly as possible due to the

measurement of response times. Following the old-new mouse presses, the monitor

displayed a three-choice alternative confidence rating ("Very confident," "Somewhat

confident," "Just guessing") to which participants were instructed to respond. Participants










were instructed to indicate the confidence with which they made their previous "old-new"

responses.

Results

Behavioral Data

As displayed in Figure 1, in contrast to Experiment 1, in which large effects of the

Task manipulation were observed, whether participants processed pairs under Bound

(hits: M = 57.3, SE = 3.21; false alarms: M = 23.7, SE = 4.25) or Separate (hits: M =

55.9, SE = 2.74; false alarms: M = 21.3, SE = 3.45) imagery instructions had no impact

on the probability of subsequent recognition of either objects or locations. On the other

hand there was a large test effect. That is, collapsed across task, location words (hits: M =

51.7, SE = 3.26; false alarms: M =25.8, SE = 3.78) were less well remembered than

object words (hits: M = 61.5, SE = 2.69; false alarms: M = 19.2, SE = 3.93).




Item Recognition

60
6 Separate
50- H Bound
40
E 30
S- 20
10

LOCATIONS OBJECTS
Item Type

Figure 3-1. Behavioral performance in Experiment 2 (hits false alarms) compared
between the two encoding groups (Bound vs. Separate) and test type
(Location vs. Object).









Furthermore, there was no impact on overall item recognition of whether

participants were first given the object word or location word test. Likewise, as expected,

there was no difference in recognition performance for the two stimulus sets.

EEG Data

ERP waveforms

ERPs elicited by word pairs in the study phase were, as noted, computed on the

basis of participants' responses on the subsequent object recognition test. Data from the

study phase were sorted as "hits or misses." As noted above, a large number of

participants' data were excluded from analysis. Thirty-five participants (18 Bound/17

Separate) provided data for the analyses described below. Each participant's averaged

data were then averaged with other participants' averaged data to calculate grand

averaged data for each class.

ERPs to the Bound condition from the 13 scalp electrode sites are presented in

Figure 3-2 below. As in the first experiment, over the course of the 6200 ms interval,

discernible evoked responses to the onset of the first and second words (N100, P200,

N400) are apparent at most sites across both conditions. Importantly, there are strong

similarities between the waveforms generated by the participants in the Bound condition

in Experiment 1 and those in the Bound condition in Experiment 2. These similarities are

most apparent in the positive slow wave from 1200 ms to 3000 ms in the frontopolar

sites, as well as a corresponding negative slow wave during the same interval at the Cz

electrode. Thus, and to maintain consistent analysis across the two experiments, the same

time windows were used for analysis in the second experiment as in the first experiment.

In the right hemisphere, the waveforms for hits and misses are nearly

indistinguishable. Small differences, with remembered items being more positive than









forgotten items, appear in the prestimulus interval at FT7, as well as immediately

preceding the onset of the second word at frontal sites (Fpl, Fp2, F3). Other differences

appear in the region of the N400 to the second word at FT7. One difference that is similar

to the Dm effects observed in Experiment 1 is found at Fp2 prior to the onset of the

second word. In this case, forgotten items are more positive than remembered items.

There are also late differences at F3 and FT7 with subsequently recognized items being

more positive than forgotten items.

ERPs to the Separate condition from the 13 scalp electrode sites are presented in

Figure 3-3 below. The waveforms again show marked deformations at standard

component latencies (N100, P200, N400). As in the Bound condition, there is a notable

positive-going slow wave between the first and second words at the frontal polar

locations. At other locations, this time frame is either characterized by negative going

activity or by little change in the overall polarity of the waveform. Unlike in the Bound

condition, there is little difference in the waveforms between those to items later

recognized and those subsequently missed, although some separation between hits and

misses is noted beginning about 900 ms after the onset of the first word at Fpl.

The correct responses to old items in the Bound and Separate conditions are

compared in Figure 3-4. Although collected from two different groups of participants

under two different task instructions, the waveforms track each other closely, especially

at posterior electrode sites. There appear to be differences, however, between the

Separate and Bound groups, and these differences seem to be larger in the left

hemisphere than in the right hemisphere and more pronounced at the anterior, than at the

posterior, electrode sites.









Statistical analysis of waveforms

As in the first experiment, mean differences in EEG amplitudes during the study

phase were conducted by running, for each condition (Bound and Separate), an analysis

of variance (ANOVA) on the ten "windows" identified in Experiment 1 that comprised

the total 6200 ms trial interval. The ANOVAs tested the same 3 factors (Performance:

Hit/Miss; Hemisphere: Left/Right; Anterior-Posterior ("AntPos"): as in Experiment 1. In

addition, a third set of ANOVAs was run to compare the mean amplitudes produced by

correct responses to old items between the Bound and Separate conditions in each

window. Thus, this ANOVA was a 3 factor (Condition: Bound/Separate; Hemisphere:

Left/Right; Anterior-Posterior: 6 levels of electrode site), 2 x 2 x 6, mixed design. The

Greenhouse-Geisser correction (E) was applied for violations of the assumptions of

sphericity for comparisons involving two or more degrees of freedom.

Bound condition

Although there were no significant main effects of Performance or interactions

between Performance and either AntPos or Hemisphere across any of the intervals,

marginal effects were observed in the interval immediately preceding the onset of the

second word (-400 0 ms) (Perfx Hemisphere: F(1,17) = 4.349, p =. 052, E = 1.00 and

early in the second word interval (0 300 ms), Perf: F(1,17) = 4.326, = .053, E = 1.00.

Neither the Perfx. AntPos, nor Perf x. AntPos, x Hemisphere interactions were

significant (all p's > .05).

Separate condition

The Separate condition showed no effects related to task performance (Perf, Perf

x Hemisphere, Perfx AntPos, Perfx Hemisphere x AntPos: all p's > .10).





















FT7
/^..(^ U i


Fp2




F4





FT8

'A' IN il II-1,--- 6


TP7


P4




02


I i-1 I :

-1 -- o


0 1000 2000
Time since onset
of first word (ms)


0 1000 2000
Time since onset
of second word (ms)


Figure 3-2. ERPs to two words during study phase during test in Experiment 2, Bound
Encoding group. Bars indicate onset and offset times of the words. Black
waveform is for first words (items: people, animals, objects) later correctly
recognized as studied; grey waveform is for words later missed.









S Fp1
LA jAi,. nj^ .I O^


Fp2




F4


I I PM,' II-





TP7
TP8




P3 P4




01 02







-1 0 1000 2000 0 1000 2000
0 1000 2000 0 1000 2000


Time since onset
of first word (ms)


Time since onset
of second word (ms)


Figure 3-3. ERPs to two words during study phase during test in Experiment 2, Separate
Encoding group. Bars indicate onset and offset times of the words. Black
waveform is for first words (actors and objects) later correctly recognized as
studied; grey waveform is for words later missed.







63



pF1 Fp2




F3 F4




FT7
FT8




Cz



TP7





P3
P4




01 02







S-1
0 1000 2000 0 1000 2000
Time since onset Time since onset
of first word (ms) of second word (ms)


Figure 3-4. ERPs to two words during study phase later shown and correctly recognized
as intact pairs during test in Experiment 2. Bars indicate onset and offset times
of the words. Black waveform is for the Bound Encoding group; Grey
waveform is for the Separate Encoding group









Bound vs. separate analysis

The third analysis consisted of comparing the mean differences between study

pairs that yielded subsequent correct recognition of objects, people or animals, at test, in

each of the two conditions. Differences were identified near the onset of the first word (0

- 300 ms) (Task x Hemisphere: (F(1,33) = 7.164, p = .011, Task x Hemisphere x AntPos



Table 3-1.Time Intervals in Experiment 2 during which Amplitude Differences were
Significant
Factor
Interval Perf Perf x AntPos Perfx Hem Perf x Ant Pos x Hem
[wl] -600 0
0 300 H H
300 600
600- 1200
1200 -2600
[w2] -400 0 B
0-300 B H
300 600
600- 1200
1200 -2600


Note. "B" = marginally significant Dm effects in Bound task, "H" = significant

differences in Bound Separate correct recognition (Hit) comparison. There were no

significant Dm effects in Separate task. Bold: .06 > p > .05. For all comparisons, a = .05.



F(2.271, 74.935) = 3.216, p = .040) and near the onset of the second word (0 -300)

(Task x Hemisphere x AntPos: F(2.655, 87.615) = 3.043, p = .039).









Discussion

Behavioral measures of performance in the second experiment revealed that

single item recognition, whether for the first or second word of a pair, is unaffected by

whether those items are the subject of relational processing. The trace that is generated in

the encoding phase by either relational or single-item processing is sufficient to produce

comparable levels of single item recognition. Levels of item recognition varied, however,

by the nature of the target. Location words were less well recognized than object words.

Location words, however, were always presented following object words in the study

phase so it cannot be determined whether the decrement in location word recognition is

attributable solely to the type of stimulus, or whether order effects also contributed to

their poorer recognition performance. It might be argued, however, that if order effects

were responsible, in part, for the decrement in location recognition, it would suffer less in

the Bound condition than in the Separate condition, by virtue of the order being less

salient to the encoding. However, the Perfx Item interaction was non-significant,

suggesting that its recognition decrement was due primarily to the difficulty in encoding

the locations.

The pattern of subsequent memory effects was different from, and less extensive

than, that found in Experiment 1. Although none of the comparisons reached significance

in the Bound condition, the marginally significant comparisons (Table 3-1) are discussed

below. The separation of hits and misses in the end of the first word interval (reflected in

a marginally significant Task x Hemisphere interaction (p = .053)), is characterized by

left, but not right-sided amplitudes for the "miss" responses being larger than "hit"

responses. In fact, visual inspection of the waveforms suggests that the differences are

driven by a deflection of the "miss" responses. Whether this characterization is accurate









is difficult to determine but it suggests that the separation between hits and misses in the

Bound condition is associated with some processing, or failure to process, the first word

late in the interval. This interpretation is further supported by the lack of differences in

the correct recognition responses to old words presented in the Bound and Separate

conditions at those corresponding intervals.

Although there were suggestions in the waveforms of the Separate condition of

subsequent memory differences, especially over the left hemisphere frontal electrodes at

about 1200 ms and again at about 2000 ms at central locations, none of the hit-miss

comparisons at any of the intervals reached significance. Although we predicted that the

Separate (as well as the Bound) condition would yield subsequent memory effects for

item recognition, at least to the first word, there is a possible explanation as to why no

effects were observed. First, Dm effects have been shown to be extraordinarily sensitive

to task demands. Thus, Paller et al. have shown that, under certain conditions, cued recall

produces large Dm effects while item recognition does not. Similarly, recognition

responses classified as "Remembered" (according to Tulving's scheme) are more likely

to produce Dm effects than "Know" responses. Thus, correct recognition responses to

previously viewed items in the current experiment are likely to have included some

proportion of guesses, or at the very least, trials on which the relational encoding failed

(and thus would have been "Misses" in the first experiment). Analysis of the confidence

ratings that participants gave during the recognition test and sorting of the study phase

ERPs into more confident versus less confident responses is more likely to yield Dm

effects.









The nature of the item Dm effects observed in the Bound, but not the Separate,

condition is open for speculation. Visual inspection of the waveforms suggests that the

Bound and Separate hits closely resemble the misses in the Separate condition in the

interval (-400 0 ms) during which subsequent memory differences were observed. As

noted, the misses in the Bound condition show a significant positive deflection. It could

be that some aspect of relational processing has a detrimental effect on item recognition.

For example, it could be that, in the Bound, but not the Separate, condition, participants

shifted their attention from the first word in preparation for the presentation and

integration of the second word. If an incomplete trace of the first (object) word was

established at the time of the shift, then the miss trials might be associated with effects

not observed in either the hit trials in the Bound and Separate conditions or the miss trials

in the Separate condition.

The subsequent memory differences observed in the next window (0 300 ms),

again for the Bound but not the Separate condition, may reflect the operation of similar

cognitive processes that support (or impair) item recognition but have no impact on

paired recognition. Thus, for example, a premature shift of attention to the second

(location) word, in the Bound but not the Separate condition, could be reflected in the

impaired recognition of the object but not the pair.

While the nature of the Dm effects observed for the Bound, but not the Separate,

condition can be speculated at, there were clear task related differences in the second

experiment, the nature of which seem to be more apparent. Correctly recognized first

words were associated with differences in ERPs for the Bound and the Separate

conditions at two similar intervals over the course of the trial. That is, at the onset of each









word (600 ms and 3600 ms), the differences between amplitude means for the Task x

AntPos x Hem conditions were significant. These differences are likely to reflect strategy

differences between the two tasks, given that an object to be integrated with a location

may be processed differently than one that will face no such demands. Likewise, upon

presentation of a location, there are demand differences for how that location will be

processed in the Bound and Separate conditions.

The second experiment revealed a unique pattern of subsequent memory effects

associated with item recognition, differing from those identified in Experiment 1. In

contrast to the Dms that accompany paired recognition, item recognition Dms were

restricted to the first word. This makes perfect sense, since the Dms were conditionalized

on recognition of the first word. Somewhat unexpectedly, however, the subsequent

memory effects were present only in the Bound task, and the relative similarity of the

waveforms between the Bound and Separate conditions suggests that those differences

may reflect error-related processing in the Bound case that may have been related to the

integrative task.














CHAPTER 4
GENERAL DISCUSSION

The cognitive and neural processes that underlie successful episodic memory

encoding include the creation of a memory trace that encompasses both an item or event

and its spatiotemporal context. Little is known, however, about the way in which an item

and its context are linked at the time of their presentation. By carefully manipulating the

encoding task and memory test type, and sorting, post-hoc, encoding trials by subsequent

memory performance the two ERP experiments reported here represent a novel approach

to examining the cognitive and neural correlates of episodic memory encoding. Using

this approach, we identified what is, to our knowledge, a unique set of ERP subsequent

memory effects. Most notably, these include a frontopolar, positive-going, slow-wave

potential late after the presentation of the first word of a pair that is more negative for

pairs later successfully recognized, following imagistic processing of concrete nouns in

an integrative encoding task (Bound condition, Experiment 1). This effect makes clear

that relational processing begins even prior to the onset of the second item (here, the

spatiotemporal "context") in a pair, and suggests that prefrontal areas play an important

role in this processing.

Distinctive Aspects of the Present Approach

Many ERP studies of episodic encoding compare the neural activity and

behavioral performance associated with one type of task or process with that in a second

task or process. Some other ERP studies sort, on a post-hoc basis, encoding trials by

subsequent memory performance to compare the neural responses during trials associated









with later successful memory performance with those associated with later unsuccessful

memory performance. We have utilized both elements while maintaining tight control

over the stimulus materials and tasks.

By analyzing encoding ERPs according to subsequent memory performance, we

have avoided encoding manipulations such as levels of processing as a proxy for memory

performance. Such manipulations putatively generate better or worse memory

performance but invariably include errors in the deep condition trials and correct

responses in the shallow condition trials. Moreover, by not using such a manipulation, we

were able to manipulate, systematically, an encoding strategy that addresses directly the

question in which we were interested are there discernable neural and cognitive

processes associated with binding objects and locations in episodic memory? Thus,

cognitive and neural processes associated with item-context binding that lead to

successful episodic memory were isolated in a pair of carefully controlled experiments.

In addition to using a unique paradigm, our experiments carefully controlled both the

stimulus materials and task parameters to make comparisons between conditions and

experiments valid.

So, for example, while one group of participants in the first experiment generated

and maintained isolated images of the items and locations, another group generated and

maintained integrated images of the same item-location pairs following identical

presentation parameters. Moreover, the two groups were tested with identical stimuli,

using the same test methods and instructions. Likewise, the second experiment used the

same stimulus materials, method and instruction as in the first experiment. The only

difference between the two experiments was in the test phase. Moreover, we used unique









item-context pairs throughout, thereby avoiding the stimulus repetition effects that make

working memory and source memory paradigms difficult to implement in the study of

long-term memory encoding. Although unique face-name (Guo et al., 2005) or face-

occupation (Yovel & Paller, 2004) pairs have been used to study relational encoding,

given the controversy surrounding the special cognitive and neural mechanisms of face

encoding and recognition, our paradigm is more generalizable than face encoding studies.

Moreover, in contrast to the remaining associative encoding studies, our experiments

specify the nature of the relational encoding to be performed. Thus, the nature of any

processes associated with one task, but not the other, can be described more precisely.

Finally, our experiments uniquely yielded the ability to contrast ERPs to the first

and second stimulus item in a relational encoding paradigm. This feature enabled us to

pose an as-yet unasked question: Are there cognitive and neural processes engaged by the

presentation of the first item of a pair that are to be relationally encoded that are

preparatory to the presentation of the second item? This question, which seems

fundamental to notions of relational encoding, has not been addressed in any ERP study

of which we are aware. In sum, no other subsequent memory ERP study has provided the

degree of control, or the possibilities for isolating the constituent elements of relational

encoding as the current pair of experiments.

Memory-Related ERPs and Integrative Episodic Encoding

The pair of experiments yielded clear evidence of task and test-dependent ERP

effects that were associated with subsequent memory performance. As predicted, these

effects differed in timing and topography that depended on both the encoding instructions

and the retrieval demands imposed by the type of memory test given. Consistent with our

predictions, when memory was queried by paired recognition, the subsequent memory









effects showed significantly different patterns between the two tasks. As expected in the

integrative encoding condition, these effects arose at frontal electrode sites; importantly,

there were no hemispheric differences. Moreover, the Dm effects occurred in response to

each word, suggesting that some kind of item-related processing contributes to relational

encoding success. In addition, an unexpected, widespread effect was observed early in

the second word interval. This effect may have signaled the allocation of cognitive

resources in preparation for subsequent integrative activity. Alternatively, it may be a

carry-over of the preparatory Dm observed late in the first-word interval (see Figure 2-1).

In contrast to the pattern of activity observed in connection with the integrative

instructions, Dm effects associated with the Separate instructions were restricted to (a) a

prestimulus hemispheric difference, and (b) a transient, widespread effect immediately

before the onset of the second word. The prestimulus Dm, which has not been reported

before, may be an important indicator of attentional or other cognitive "set" differences

that contributed to paired recognition success. Likewise, the widespread, transient effect

just prior to onset of the second word may reflect the allocation of necessary attentional

resources that separates later remembered from later-forgotten pairs. Although it is

tempting to interpret the long-duration differences to each word in the Bound condition to

item processing, their absence in the Separate condition suggests that those effects were

not merely indicators of item-only processing. Rather, they likely reflect some degree of

processing of the item as, to the first word, a to-be-integrated stimulus feature. The

comparable second word Dm may be associated with the integration of the location with

the object. Thus, contrary to our strongest predictions about the differences between the

two conditions, successful encoding in the Bound task was not simply successful









encoding in the Separate task with an integrative component added on. Inclusion of the

integrative component changed the entire pattern of neurocognitive activity associated

with subsequent correct paired recognition.

Although memory was tested by paired recognition in Experiment 1, the patterns

of results could have been due to the contributions of relational processing, item-only

processing, or both. Thus, we conducted a second experiment using an item recognition

test to discriminate between relational encoding processes that support pair recognition

and item encoding processes that support simple recognition. We predicted that there

would be little difference between the processes underlying item recognition whether or

not a relational encoding strategy was used. Thus, we expected the patterns of subsequent

memory differences between the Bound and the Separate conditions to be very similar

when memory was tested by item recognition. As in the first experiment, there was no

overlap between the Dm effects associated with the Bound encoding and Separate

encoding strategies. In fact, there were no significant subsequent memory effects at all in

the Separate condition, and only two intervals showed marginal Dm effects in the Bound

condition. This finding is consistent with findings that recollection and recall tasks are

more likely to produce Dm effects than item recognition and it suggests that the Bound

Dm effects represent item-specific encoding processes, rather than relational encoding

effects. The timing of the marginal Dm effects in the Bound condition is also consistent

with our predictions. Encoding trials were classified on subsequent recognition of the

item (person, animal, object) word, which was always presented as the first word of the

pair during each study trial. Thus, Dm effects would be expected in response to the first,

rather than the second, word interval. The marginally significant effects at the









presentation of the second word may reflect spillover from the sustained processing of

the first word at the end of the interval.

Overall, then, the study produced, in each condition and experiment, a pattern of

ERP differences that were, largely, consistent with our predictions.

Comparisons to Previous Findings

While there is a scarcity of findings regarding the ERP correlates of item-context

encoding, Kounios and colleagues (2001) found that fusion association, in which two

concepts are fused together to form a qualitatively distinguishable third concept (e.g.,

computer + virus = computer virus), has distinct neural correlates from juxtapositional

association, in which two concepts are associated by contiguity. Fusion association was

distinguishable by activity in right prefrontal cortex following the second word and

waveform differences between quickly and slowly retrieved word pair orders at test at

bilateral frontotemporal sites from 200 ms to 3000 ms after the onset of the second word.

While theoretical and methodological differences between Kounios and

colleagues' work and the present study make direct comparisons difficult, it is worth

noting that Kounios claimed that the difference in topography, timing and polarity

between juxtapositional and fusion effects supported the idea that the two different

cognitive processes are engaged by the different tasks. Likewise, in the current study,

timing and topography differences between the effects found for encoding in the Bound

condition and those found for Separate encoding, as classified by paired recognition

performance support the idea that the two types of encoding recruit different cognitive

processes. This claim is further buttressed by the finding that these effects differ from the

Bound and Separate encoding effects that underlies item memory.









Of the ERP findings in the study, perhaps none is more striking than the pair of

Dm effects that occur in similar, long (1400 ms) intervals following the presentation of

the first and second words in the Bound, but not Separate, encoding task when trials are

classified by performance on the paired recognition test. These effects, which have a

frontal topography, differ from other observed frontal effects in associative memory

encoding in two ways.

First, the effects in the interval following the first word ride on a positive-going

slow wave beginning around 1400 ms after word onset. While we could not identify any

studies other than Kounios and colleagues' (2001) that use a sequential S1-S2 word

presentation paradigm in recognition memory, the positive-going nature of the slow

wave, in contrast to the reversal (negative-going slow wave) at more posterior electrode

sites is consistent with sustained positivity at frontopolar sites in other studies (Duarte et

al., 2004; Mangels et al., 2001). As noted above, if our task is analogous to an S1-S2 task

that typically elicits a negative-going slow wave, it is not surprising, perhaps, that the Fp

sites yield positive-going slow change that persists until the onset of the second word.

While the first word interval is followed by a frontal, positive-going slow wave, the

second word is followed by a widespread negativity (with a notable exception at FT8).

The Dm in this interval consists of the more typically observed pattern; subsequently

recognized pairs are of greater positivity than subsequently missed items.

Second, the Dm for first words at frontopolar sites is of negative polarity

(subsequent misses > subsequent hits). We have been able to identify only one other

study (Guo et al., 2005) in which, at frontal sites, the amplitude of subsequently

unrecognized items was more positive than that of subsequently recognized items.









Although it is unclear to what the negative Dm in Guo and colleagues' study can be

attributed, it, too, was embedded in a sustained positive-going frontal wave (albeit only

through the end of the Is trial interval). Guo and colleagues' study involved participants

intentionally encoding concurrently presented visually presented faces and auditorily

presented names. The fact that face recognition Dms were significant in the later part of

the interval and the name recognition Dms were significant only in the early interval

suggests that the name and face were processed sequentially. Thus, it is possible that the

negative Dm effects Guo and colleagues observed for name recognition reflect

completion of the name processing and preparation for face name binding or maintenance

of the name during face processing. This explanation, of course, is speculative and

warrants further investigation.

The pattern of the Dms to the first and second word in the long interval is

intriguing. Duarte and colleagues (2004), who found distinct subsequent memory effects

for pictures subsequently classified as "remembered" or subsequently classified as

"known" versus those that were missed, concluded that the sustained bilateral frontal

activity associated with "remember" responses were attributable to "more extensive

processing" than those later classified as "known." It could be that, in the Bound

condition, participants were mentally "manipulating" or refining their images of the first

word object in preparation for the required upcoming integration. No participants

reported to us, however, any deliberate strategy in response to the first word. A better

understanding of this first word, as well as its second word parallel, effect will be

important in using ERPs to elucidate relational encoding.









The second pair of intriguing Dms are those that occurred, in the Bound condition

conditionalized on paired recognition performance, to the second word; one a

widespread, early (300 ms post-word 2 onset) effect, the other a late (1500 ms post-word

2 onset), frontal effect.

In the only experiment that we have identified that sequentially presented

successive words for associative processing, and then measured ERPs to the second

word, Kounios and colleagues (2001), found that participants who successfully fused

word pairs into a unitary concept (e.g., computer + virus = "computer virus") showed

ERP differences according to whether they later quickly or slowly identified the order in

which pairs were earlier presented. These differences persisted over the three-second

interval following presentation of the second word. Interestingly, the initial differences

(200-800 ms) were marked by activation in right prefrontal cortex. From 800 2100 ms

following the second word, however, activation shifted to a region in left medial superior

frontal cortex. In many respects, differences in experimental protocols between our

experiments and Kounios and colleagues' make comparisons between the two difficult.

However, the fact that Kounios found a subsequent memory effects that persisted

throughout a three second post word interval and our results point to a pair of Dm

effects that lasted nearly two seconds, is striking.

Likewise, in addition to an earlier set of subsequent memory effects, Mangels,

Picton and Craik (2001) found sustained prefrontal positive and sustained posterior

negative subsequent memory effects beginning at about 1000 ms after the onset of the

word. They speculated that these effects reflected the interaction of a fronto-posterior

network where the posterior portion of the network was responsible for sustained object









representation and the frontal part of the network, particularly at the Fp electrodes,

playing a role in the elaborative processes that facilitate subsequent recollection and

recognition. It is important to note that, similar to our studies, the Fp electrodes recorded

a positive-going wave from about 1000 ms to the end of Mangel and colleagues' interval

(2000 ms). Likewise, we found a positive slow wave at Fp sites from about 1000 ms

following the first word until the first 200 ms following presentation of the second word.

An important difference between Mangels' and our findings was that the slow wave in

Mangels study was positive-going across most frontal electrodes (Fp, AF, F). The slow

wave only became negative-going at posterior sites. In our experiments, the slow wave

was positive-going only at Fp sites, and negative-going at other electrode locations.

Another important difference, however, was in the polarity of the difference wave

between Mangels' findings (positive at frontal sites, negative at posterior locations), and

ours (negative at frontopolar sites).

Limitations and Future Directions

While the results from the experiments presented are unique and contribute to our

understanding of the neurocognitive basis of relational encoding and long-term memory

performance, there are aspects of the experimental design that limit the conclusions that

can be drawn from them. First, although the design provides a unique amount of control

over stimulus and task factors that could otherwise confound the results, the static item-

location design fails to capture either the temporal or the dynamic aspects of episodic

memory. As Craik (1989) notes, episodes, as described by Tulving (1984), consist of a

series of events, which in turn consist of item/context pairs. Thus, by limiting the

"episodes" here to single pairs, we have excluded participants' experience of ongoing

events and the cognition that accompanies it. An initial foray into the dynamic aspect of









episodic encoding might include having participants generate dynamic images (i.e.,

visualize an ELEPHANT falling off of a CLIFF). Likewise, encoding activity for the

temporal order of location-item pairs could be tested.

Second, by their design, the experiments allowed for the possibility of some

overlap between the correct and incorrect response classes in that some trials in the

correct response category (Experiment 1: Intact Correct; Experiment 2: Old Old) may

have been the result of low confidence guesses. Analysis of the confidence ratings in the

test phase of Experiment 2 and resorting encoding trials into High and Medium

Confidence correct responses and Low Confidence (Guessing) correct and incorrect trials

would point to the degree of overlap and the contribution of guessing in the correct

responses ERPs. No such confidence ratings were collected in the first experiment so

defining the contribution of guessing trials to the correct responses would be more

difficult.

Third, what role the ease with which pairs were capable of being imaged might

have played is unclear. Although data regarding the ease of imagery were collected, these

data have not been analyzed. It could be that, if these data were sorted into Easy and

Difficult, they would correspond highly to correct and incorrect subsequent memory

performance, suggesting a prominent role for the ease of imagery in encoding related

memory effects. On the other hand, it might be that the greater cognitive effort expended

in generating and maintaining difficult images would yield better memory performance.

Fourth, although low-density localization techniques (e.g., LORETA: Pascual-

Marqui, Michel & Lehmann, 1994) are available, the use of a low-density (16 electrode)

array made it difficult to attempt more serious source localization analysis. Nevertheless,









our use of traditional interval analyses and ANOVAs that included hemisphere and

anterior-posterior groups as factors enabled us to generally identify regional activity. This

regional activity was in accord with at least some previous findings where the source of

neural activity has been identified using EEG localization (e.g., Kounios et al., 2001) and

fMRI (e.g., Prabhakaran et al., 2000) techniques.

Fifth, keeping the encoding instructions and test type as between-subjects factors,

one of the strengths of the design, also weakens the cross-task comparisons. The benefits

of implementing the instruction and test type manipulation between subjects are clear.

Participants are less likely to employ relational encoding strategies, even unintentionally,

if they haven't engaged in them in a preceding study block. Likewise, if the test type

were implemented as a within-subjects manipulation, participants would be likely to have

received the benefit of item-retrieval in the paired recognition task (if it followed item

recognition). Thus, the use of a between-subjects design for these factors largely keeps

strategies and memory processes discrete from one another. However, the manipulations

introduce a greater degree of variability than would be produced if they were manipulated

within-subjects and, thus, tend to weaken the statistical comparison. It is possible that

encoding comparisons would have produced more extensive differences than were

observed. Moreover, manipulating test type as a within-subjects factor (if a way of

keeping it from being confounded with retrieval practice could be ascertained) would

facilitate the direct comparisons of the encoding processes that support the two memory

retrieval types.

Finally, these data could be subjected to a coherence analysis, a technique that is

being used increasingly in the study of memory encoding (e.g., Summerfield & Mangels,









2005; Weiss, Muller & Rappelsberger, 2000; Weiss & Rappelsberger, 2000). The

essence of this type of analysis is to identify regions in which EEG bands from separate

brain regions experience phase synchronization of neural oscillations over discrete

temporal windows. This phase locking is a considered a candidate code by which

information is shared between spatially distinct brain areas (Summerfield & Mangels,

2005). While coherence analysis has been used to examine feature binding (Summerfield

& Mangels, 2005) and perception (Engel, Fries & Singer, 2001), it seems to be a

potentially important tool for investigating mechanisms that underlie the relational

encoding that constitute item-context binding in episodic memory.
















APPENDIX A
EXPERIMENT 1, STUDY PHASE WORD PAIRS

2.1 A WHEELBARROW/RINK / PIG/WATERFALL /
FIREFLY/SIDEWALK / 2.24 A 5.45 A
1.2 A COCKROACH/CARNIVAL RAKE/FARM /
SURFER/STUDIO / / 7.46 0
3.3 0 1.25 0 TAXI/ZOO /
STEREO/MAILBOX / KING/DUMP / 1.47 0
2.4 0 1.26 0 BANDIT/CASINO /
BUTTERFLY/BAKERY / DRUMMER/JUNGLE / 2.48 0
2.5 I 3.27 0 GIRAFFE/CABINET /
POSSUM/LOFT / LAMP/TREE / 1.49 I
2.6 I 7.28 0 MECHANIC/THICKET /
FLY/GLACIER / TRACTOR/DARKROOM / 2.50 0
2.7 0 2.29 A ANT/CATHEDRAL /
TIGER/BARN / HIPPOPOTAMUS/GALLER 3.51 0
5.8 A Y / COUCH/HILL /
PAINTBRUSH/RANCH / 7.30 A 5.52 0
2.9 0 HELICOPTER/PHARMACY MOWER/CORNER /
GORILLA/HOUSE / / 4.53 I
1.10 A 4.31 A KNIFE/PYRAMID /
SOLDIER/PARTY / ROCKET/CRATER / 2.54 A
4.11 A 2.32 I BIRD/MARKET /
BAYONET/WAREHOUSE / HORSE/BOX / 6.55 0
1.12 A 3.33 0 VIOLIN/WEDDING /
SKATER/STADIUM / TOILET/STEPS / 2.56 0
3.13 I 2.34 A OSTRICH/AMPHITHEATR
DRESSER/MUSEUM / MOUSE/KENNEL / E /
1.14 I 1.35 A 2.57 0
SINGER/ROOF / DETECTIVE/CAMP / BUFFALO/RESORT /
3.15 0 8.36 A 4.58 0
STOOL/RIDGE / DOLLHOUSE/DRIVE-IN BOMB/MOON /
6.16 0 / 2.59 0
TAMBOURINE/CLIFF / 7.37 0 DUCK/HIGHWAY /
2.17 I SUBMARINE/AIRPORT / 1.60 A
FROG/BILLBOARD / 1.38 I BARTENDER/VALLEY /
1.18 0 HIKER/BANK / 1.61 I
LUMBERJACK/AVALANCH 7.39 0 DANCER/CAFE /
E / AIRPLANE/REEF / 2.62 I
3.19 0 2.40 I COW/MORGUE /
CRIB/OFFICE / OWL/RAVINE / 2.63 A
2.20 0 8.41 0 SEAL/BATHROOM /
GRASSHOPPER/PENTHOU BUBBLES/DISCO / 4.64 0
SE / 1.42 0 ROPE/CONVENTION /
1.21 I DOORMAN/PLANTATION 4.65 0
FIREMAN/BAR / / CANNON/CABIN /
2.22 0 5.43 I 7.66 I
TURTLE/FIREPLACE / PENCIL/MANSION / SKATES/TORNADO /
5.23 I 2.44 A 2.67 0











OX/CHAPEL /
1.68 0
SWIMMER/HAYLOFT /
1.69 A
SLAVE/DESERT /
8.70 0
ROBOT/RIVER /
1.71 A
EXPLORER/TOWER /
5.72 A
SHOVEL/DOORWAY /
8.73 0
GLOVE/EARTHQUAKE /
1.74 0
BAKER/DAM /
2.75 0
CHIMPANZEE/ICEBERG
/
2.76 A
HEDGEHOG/THUNDERSTO
RM /
7.77 A
TANK/CEMETERY /
5.78 0
DRILL/ISLAND /
7.79 0
SCOOTER/LABORATORY
/
4.80 A
BOOK/GROCERY /
8.81 0
HORSESHOE/CLOSET /
2.82 A
OCTOPUS/PARK /
4.83 A
GUN/FENCE /
2.84 A
MANATEE/NURSERY /
1.85 A
CLOWN/KITCHEN /
2.86 0
LIZARD/MEADOW /
1.87 A
MAGICIAN/TEPEE /
1.88 0
MAID/MOUNTAIN /
1.89 A
NUN/CORRAL /
2.90 0
WOLF/BOULDER /
7.91 A
STREETCAR/SUNSET /
7.92 I
BALLOON/MOSQUE /
5.93 0
TOOLBOX/COTTAGE /
2.94 A


TOUCAN/DRIVEWAY /
1.95 I
SAILOR/ATTIC /
1.96 I
GYMNAST/PATH /
1.97 0
DENTIST/TOMB /
2.98 A
LEOPARD/SKYSCRAPER
/
2.99 A
BEAR/BEDROOM /
1.100 A
CARPENTER/DUSK /
5.101 0
PLANE/CELLAR /
4.102 A
ARROW/PLAYHOUSE /
1.103 I
INMATE/CHIMNEY /
1.104 0
MILKMAN/COFFIN /
2.105 A
PORCUPINE/OUTHOUSE
/
1.106 0
PROSTITUTE/MALL /
4.107 A
FORK/TUNNEL /
7.108 A
BICYCLE/BALCONY /
2.109 0
LADYBUG/FORT /
8.110 A
CRAYONS/SEWER /
2.111 A
GOAT/BASEMENT /
1.112 A
ASTRONAUT/WELL /
3.113 A
STOVE/DECK /
7.114 A
TRUCK/QUARRY /
1.115 A
GIRL/VOLCANO /
2.116 A
PANDA/LAUNDRY /
1.117 0
BRIDE/SHOWER /
7.118 0
RICKSHAW/UNIVERSITY
/
3.119 A
VASE/ESCALATOR /
6.120 0
FLUTE/FOREST /
8.121 0


DOLL/DORMITORY /
1.122 0
BOY/SWAMP /
1.123 0
SKIER/APARTMENT /
5.124 A
LADDER/SNOWSTORM /
2.125 A
ANTELOPE/PLAYGROUND
/
2.126 0
CHICKEN/CANYON /
3.127 0
CHAIR/CITY /
1.128 A
CONDUCTOR/CLOUD /
2.129 0
CRAB/ALLEY /
2.130 A
RACCOON/DAYBREAK /
5.131 0
VISE/HAILSTORM /
4.132 0
PILLOW/PRAIRIE /
2.133 0
SQUIRREL/IGLOO /
8.134 A
PUPPET/TENT /
5.135 0
WRENCH/PORCH /
8.136 I
BOOMERANG/CREVICE /
7.137 0
SURFBOARD/WINDOW /
8.138 A
SOFTBALL/RAMP /
1.139 A
JUGGLER/HOTEL /
3.140 0
RADIO/ELEVATOR /
2.141 A
FERRET/LIBRARY /
6.142 0
CLARINET/GYMNASIUM
/
2.143 A
SPIDER/CAGE /
3.144 0
BED/FLOOD /
1.145 A
JUDGE/HOSPITAL /
1.146 0
GROOM/JAIL /
1.147 0
BRICKLAYER/VILLAGE
/
2.148 A











LION/HUT /
6.149 A
GUITAR/CREEK /
4.150 0
SPEAR/FOG /
5.151 0
SCREWS/CIRCUS /
2.152 A


WHALE/SLAUGHTERHOUS
E /
6.153 0
HARP/GARAGE /
6.154 A
ACCORDION/GEYSER /
5.155 0
SANDPAPER/OCEAN /
6.156 A


CYMBALS/LAKE /
3.157 A
PICTURE/SCHOOL /
4.158 0
SWORD/CAVE /
1.159 0
COWBOY/HARBOR /
2.160 0
RAT/BEACH /




















EXPERIMENT 1,




2.1 A 0
FIREFLY/SIDEWALK /
1.2 A 0
SURFER/STUDIO /
3.3 0 0
STEREO/MAILBOX /
2.4 0 0
BUTTERFLY/BAKERY /
2.5 I 0
OPOSSUM/LOFT /
2.6 I 0
FLY/GLACIER /
2.7 0 0
TIGER/BARN /
5.8 A 0
PAINTBRUSH/RANCH /
2.9 0 0
GORILLA/HOUSE /
1.10 A 0
SOLDIER/PARTY /
4.11 A 0
BAYONET/WAREHOUSE /
1.12 A 0
SKATER/STADIUM /
3.13 I 0
DRESSER/MUSEUM /
1.14 I 0
SINGER/ROOF /
3.15 0 0
STOOL/RIDGE /
6.16 0 0
TAMBOURINE/CLIFF /
2.17 I 0
FROG/BILLBOARD /
1.18 0 0
LUMBERJACK/AVALANCH
E /
3.19 0 0
CRIB/OFFICE /
2.20 0 0
GRASSHOPPER/PENTHOU
SE /
1.21 I 0
FIREMAN/BAR /
2.22 0 0


APPENDIX B
PAIRED RECOGNITION




TURTLE/FIREPLACE /
5.23 I 0
WHEELBARROW/RINK /
2.24 A 0
COCKROACH/CARNIVAL
/
1.25 0 0
KING/DUMP /
1.26 0 0
DRUMMER/JUNGLE /
3.27 0 0
LAMP/TREE /
7.28 0 0
TRACTOR/DARKROOM /
2.29 A 0
HIPPOPOTAMUS/GALLER
Y /
7.30 A 0
HELICOPTER/PHARMACY
/
4.31 A 0
ROCKET/CRATER /
2.32 I 0
HORSE/BOX /
3.33 0 0
TOILET/STEPS /
2.34 A 0
MOUSE/KENNEL /
1.35 A 0
DETECTIVE/CAMP /
8.36 A 0
DOLLHOUSE/DRIVE-IN
/
7.37 0 0
SUBMARINE/AIRPORT /
1.38 I 0
HIKER/BANK /
7.39 0 0
AIRPLANE/REEF /
2.40 I 0
OWL/RAVINE /
8.41 0 0
BUBBLES/DISCO /
1.42 0 0


TEST WORD PAIRS




DOORMAN/PLANTATION
/
5.43 I 0
PENCIL/MANSION /
2.44 A 0
PIG/WATERFALL /
5.45 A 0
RAKE/FARM /
7.46 0 0
TAXI/ZOO /
1.47 0 0
BANDIT/CASINO /
2.48 0 0
GIRAFFE/CABINET /
1.49 I
MECHANIC/THICKET /
2.50 0 0
ANT/CATHEDRAL /
3.51 0 0
COUCH/HILL /
5.52 0 0
MOWER/CORNER /
4.53 I 0
KNIFE/PYRAMID /
2.54 A 0
BIRD/MARKET /
6.55 0 0
VIOLIN/WEDDING /
2.56 0 0
OSTRICH/AMPHITHEATR
E /
2.57 0 0
BUFFALO/RESORT /
4.58 0 0
BOMB/MOON /
2.59 0 0
DUCK/HIGHWAY /
1.60 A 0
BARTENDER/VALLEY /
1.61 I 0
DANCER/CAFE /
2.62 I 0
COW/MORGUE /
2.63 A 0
SEAL/BATHROOM /











4.64 0 0
ROPE/CONVENTION /
4.65 0 0
CANNON/CABIN /
7.66 I 0
SKATES/TORNADO /
2.67 0 0
OX/CHAPEL /
1.68 0 0
SWIMMER/HAYLOFT /
1.69 A 0
SLAVE/DESERT /
8.70 0 0
ROBOT/RIVER /
1.71 A 0
EXPLORER/TOWER /
5.72 A 0
SHOVEL/DOORWAY /
8.73 0 0
GLOVE/EARTHQUAKE /
1.74 0 0
BAKER/DAM /
2.75 0 0
CHIMPANZEE/ICEBERG
/
2.76 A 0
HEDGEHOG/THUNDERSTO
RM /
7.77 A 0
TANK/CEMETERY /
5.78 0 0
DRILL/ISLAND /
7.79 0 0
SCOOTER/LABORATORY
/
4.80 A 0
BOOK/GROCERY /
8.81 A N
HORSESHOE/LAKE /
2.82 I N
OCTOPUS/ATTIC /
4.83 A N
GUN/SCHOOL /
2.84 A N
MANATEE/SNOWSTORM /
1.85 0 N
CLOWN/COFFIN /
2.86 0 N
LIZARD/MOUNTAIN /
1.87 A N
MAGICIAN/SKYSCRAPER
/
1.88 A N
MAID/SEWER /
1.89 A N
NUN/HOTEL /
2.90 0 N


WOLF/DORMITORY /
7.91 0ON
STREETCAR/ELEVATOR
/
7.92 O N
BALLOON/IGLOO /
1.93 O N
LADYBUG/SUNSET /
5.94 A N
TOOLBOX/HOSPITAL /
2.95 O N
TOUCAN/CITY /
7.96 A N
RICKSHAW/ESCALATOR
/
1.97 O N
SAILOR/CELLAR /
1.98 A N
GYMNAST/VOLCANO /
1.99 A N
DENTIST/PLAYGROUND
/
2.100 0 N
LEOPARD/FLOOD /
2.101 A N
BEAR/OUTHOUSE /
1.102 0 N
CARPENTER/FORT /
5.103 A N
PLANE/CREEK /
4.104 I N
ARROW/CREVICE /
1.105 0 N
INMATE/BOULDER /
1.106 0 N
MILKMAN/APARTMENT /
5.107 0 N
BED/FOG /
2.108 0 N
PORCUPINE/CIRCUS /
1.109 0 N
PROSTITUTE/GARAGE /
4.110 A N
FORK/TENT /
7.111 A N
BICYCLE/LAUNDRY /
8.112 A N
CRAYONS/BASEMENT /
2.113 0 N
GOAT/SWAMP /
3.114 A N
STOVE/TEPEE /
2.115 0 N
CHICKEN/HAILSTORM /
7.116 0 N
TRUCK/ALLEY /
1.117 0 N


GIRL/BEACH /
2.118 0 N
PANDA/VILLAGE /
1.119 A N
BRIDE/RAMP /
3.120 0 N
VASE/MALL /
6.121 A N
FLUTE/KITCHEN /
8.122 0 N
DOLL/JAIL /
1.123 A N
BOY/LIBRARY /
1.124 0 N
SKIER/PORCH /
5.125 0 N
LADDER/OCEAN /
2.126 A N
ANTELOPE/NURSERY /
1.127 0 N
CONDUCTOR/CAVE /
2.128 0 N
CRAB/FOREST /
2.129 A N
RACCOON/DRIVEWAY /
5.130 A N
VISE/DUSK /
4.131 A N
PILLOW/QUARRY /
3.132 A N
CHAIR/FENCE /
8.133 A N
PUPPET/SLAUGHTERHOU
SE /
5.134 A N
WRENCH/TUNNEL /
8.135 I N
BOOMERANG/MOSQUE /
7.136 0 N
SURFBOARD/MEADOW /
8.137 0 N
SOFTBALL/WINDOW /
1.138 0 N
JUGGLER/CLOSET /
3.139 A N
RADIO/CAGE /
2.140 A N
FERRET/CORRAL /
6.141 0 N
CLARINET/CANYON /
2.142 A N
SPIDER/BALCONY /
3.143 A N
ASTRONAUT/PLAYHOUSE
/
1.144 0 N
JUDGE/COTTAGE /











1.145 A N
GROOM/CLOUD /
1.146 A N
BRICKLAYER/PARK /
2.147 A N
LION/GEYSER /
6.148 0 N
GUITAR/HARBOR /
2.149 0 N
SQUIRREL/PRAIRIE /
4.150 I N
SPEAR/CHIMNEY /


5.151 A N
SCREWS/BEDROOM /
2.152 0 N
WHALE/SHOWER /
6.153 0 N
HARP/UNIVERSITY /
6.154 A N
ACCORDION/HUT /
5.155 A N
SANDPAPER/DAYBREAK


CYMBALS/GYMNASIUM /
3.157 A N
PICTURE/WELL /
4.158 0 N
SWORD/TOMB /
1.159 I N
COWBOY/PATH /
2.160 A N
RAT/DECK /


6.156 0 N
















APPENDIX C
EXPERIMENT 2, STUDY PHASE WORD PAIRS

2.1 A WHEELBARROW/RINK / PIG/WATERFALL /
FIREFLY/SIDEWALK / 2.24 A 5.45 A
1.2 A COCKROACH/CARNIVAL RAKE/FARM /
SURFER/STUDIO / / 7.46 0
3.3 0 1.25 0 TAXI/ZOO /
STEREO/MAILBOX / KING/DUMP / 1.47 0
2.4 0 1.26 0 BANDIT/CASINO /
BUTTERFLY/BAKERY / DRUMMER/JUNGLE / 2.48 0
2.5 I 3.27 0 GIRAFFE/CABINET /
POSSUM/LOFT / LAMP/TREE / 1.49 I
2.6 I 7.28 0 MECHANIC/THICKET /
FLY/GLACIER / TRACTOR/DARKROOM / 2.50 0
2.7 0 2.29 A ANT/CATHEDRAL /
TIGER/BARN / HIPPOPOTAMUS/GALLER 3.51 0
5.8 A Y / COUCH/HILL /
PAINTBRUSH/RANCH / 7.30 A 5.52 0
2.9 0 HELICOPTER/PHARMACY MOWER/CORNER /
GORILLA/HOUSE / / 4.53 I
1.10 A 4.31 A KNIFE/PYRAMID /
SOLDIER/PARTY / ROCKET/CRATER / 2.54 A
4.11 A 2.32 I BIRD/MARKET /
BAYONET/WAREHOUSE / HORSE/BOX / 6.55 0
1.12 A 3.33 0 VIOLIN/WEDDING /
SKATER/STADIUM / TOILET/STEPS / 2.56 0
3.13 I 2.34 A OSTRICH/AMPHITHEATR
DRESSER/MUSEUM / MOUSE/KENNEL / E /
1.14 I 1.35 A 2.57 0
SINGER/ROOF / DETECTIVE/CAMP / BUFFALO/RESORT /
3.15 0 8.36 A 4.58 0
STOOL/RIDGE / DOLLHOUSE/DRIVE-IN BOMB/MOON /
6.16 0 / 2.59 0
TAMBOURINE/CLIFF / 7.37 0 DUCK/HIGHWAY /
2.17 I SUBMARINE/AIRPORT / 1.60 A
FROG/BILLBOARD / 1.38 I BARTENDER/VALLEY /
1.18 0 HIKER/BANK / 1.61 I
LUMBERJACK/AVALANCH 7.39 0 DANCER/CAFE /
E / AIRPLANE/REEF / 2.62 I
3.19 0 2.40 I COW/MORGUE /
CRIB/OFFICE / OWL/RAVINE / 2.63 A
2.20 0 8.41 0 SEAL/BATHROOM /
GRASSHOPPER/PENTHOU BUBBLES/DISCO / 4.64 0
SE / 1.42 0 ROPE/CONVENTION /
1.21 I DOORMAN/PLANTATION 4.65 0
FIREMAN/BAR / / CANNON/CABIN /
2.22 0 5.43 I 7.66 I
TURTLE/FIREPLACE / PENCIL/MANSION / SKATES/TORNADO /
5.23 I 2.44 A 1.68 0











SWIMMER/HAYLOFT /
1.69 A
SLAVE/DESERT /
8.70 0
ROBOT/RIVER /
1.71 A
EXPLORER/TOWER /
5.72 A
SHOVEL/DOORWAY /
8.73 0


GLOVE/EARTHQUAKE /
1.74 0
BAKER/DAM /
7.77 A
TANK/CEMETERY /
5.78 0
DRILL/ISLAND /
8.81 0
HORSESHOE/CLOSET /
1.85 A


CLOWN/KITCHEN /
1.87 A
MAGICIAN/TEPEE /
6.120 0
FLUTE/FOREST /
6.142 0
CLARINET/GYMNASIUM
/



















APPENDIX D
EXPERIMENT 2, OBJECT WORD TEST ITEMS


2.1 A 0
FIREFLY /
1.2 A 0
SURFER /
3.3 0 0
STEREO /
2.4 0 0
BUTTERFLY /
2.5 I 0
POSSUM /
2.6 I 0
FLY /
2.7 0 0
TIGER /
5.8 A 0
PAINTBRUSH /
2.9 0 0
GORILLA /
1.10 A 0
SOLDIER /
4.11 A 0
BAYONET /
1.12 A 0
SKATER /
3.13 I 0
DRESSER /
1.14 I 0
SINGER /
3.15 0 0
STOOL /
6.16 0 0
TAMBOURINE /
2.17 I 0
FROG /
1.18 0 0
LUMBERJACK /
3.19 0 0
CRIB /
2.20 0 0
GRASSHOPPER /
1.21 I 0
FIREMAN /
2.22 0 0
TURTLE /
5.23 I 0
WHEELBARROW /
2.24 A 0


COCKROACH /
1.25 0 0
KING /
1.26 0 0
DRUMMER /
3.27 0 0
LAMP /
7.28 0 0
TRACTOR /
2.29 A 0
HIPPOPOTAMUS /
7.30 A 0
HELICOPTER /
4.31 A 0
ROCKET /
2.32 I 0
HORSE /
3.33 0 0
TOILET /
2.34 A 0
MOUSE /
1.35 A 0
DETECTIVE /
8.36 A 0
DOLLHOUSE /
7.37 0 0
SUBMARINE /
1.38 I 0
HIKER /
7.39 0 0
AIRPLANE /
2.40 I 0
OWL /
8.41 0 0
BUBBLES /
1.42 0 0
DOORMAN /
5.43 I 0
PENCIL /
2.44 A 0
PIG /
5.45 A 0
RAKE /
7.46 0 0
TAXI /
1.47 0 0
BANDIT /


2.48 0 0
GIRAFFE /
1.49 I 0
MECHANIC /
2.50 0 0
ANT /
3.51 0 0
COUCH /
5.52 0 0
MOWER /
4.53 I 0
KNIFE /
2.54 A 0
BIRD /
6.55 0 0
VIOLIN /
2.56 0 0
OSTRICH /
2.57 0 0
BUFFALO /
4.58 0 0
BOMB /
2.59 0 0
DUCK /
1.60 A 0
BARTENDER
1.61 I 0
DANCER /
2.62 I 0
COW /
2.63 A 0
SEAL /
4.64 0 0
ROPE /
4.65 0 0
CANNON /
7.66 I 0
SKATES /
1.67 0
SWIMMER /
1.68 A 0
SLAVE /
8.69 0 0
ROBOT /
1.70 A 0
EXPLORER /
5.71 A 0












SHOVEL /
8.72 0 0
GLOVE /
1.73 0 0
BAKER /
7.74 A 0
TANK /
5.75 0 0
DRILL /
8.76 0 0
HORSESHOE /
1.77 A 0
CLOWN /
1.78 A 0
MAGICIAN /
6.79 O0
FLUTE /
6.80 0 0
CLARINET /
2.810 N
OX /
2.82 0 N
CHIMPANZEE
2.83 A N
HEDGEHOG /
7.84 0 N
SCOOTER /
4.85 A N
BOOK /
2.86 A N
OCTOPUS /
4.87 A N
GUN /
2.88 A N
MANATEE /
2.89 0 N
LIZARD /
1.90 0 N
MAID /
1.91 A N
NUN /
2.92 0 N
WOLF /
7.93 A N
STREETCAR /
7.94 I N
BALLOON /
5.95 0 N
TOOLBOX /
2.96 A N
TOUCAN /
1.97 I N
SAILOR /
1.98 I N
GYMNAST /
1.99 0 N
DENTIST /


2.100 A N
LEOPARD /
2.101 A N
BEAR /
1.102 A N
CARPENTER /
5.103 0 N
PLANE /
4.104 A N
ARROW /
1.105 I N
INMATE /
1.106 0 N
MILKMAN /
2.107 A N
PORCUPINE /
1.108 0 N
PROSTITUTE /
4.109 A N
FORK /
7.110 A N
BICYCLE /
2.111 0 N
LADYBUG /
8.112 A N
CRAYONS /
2.113 A N
GOAT /
1.114 A N
ASTRONAUT /
3.115 A N
STOVE /
7.116 A N
TRUCK /
1.117 A N
GIRL /
2.118 A N
PANDA /
1.119 0 N
BRIDE /
7.120 0 N
RICKSHAW /
3.121 A N
VASE /
8.122 0 N
DOLL /
1.123 0 N
BOY /
1.124 0 N
SKIER /
5.125 A N
LADDER /
2.126 A N
ANTELOPE /
2.127 0 N
CHICKEN /
3.128 0 N


CHAIR /
1.129 A N
CONDUCTOR /
2.130 0 N
CRAB /
2.131 A N
RACCOON /
5.132 0 N
VISE /
4.133 0 N
PILLOW /
2.134 0 N
SQUIRREL /
8.135 A N
PUPPET /
5.136 0 N
WRENCH /
8.137 I N
BOOMERANG /
7.138 0 N
SURFBOARD /
8.139 A N
SOFTBALL /
1.140 A N
JUGGLER /
3.141 0 N
RADIO /
2.142 A N
FERRET /
2.143 A N
SPIDER /
3.144 0 N
BED /
1.145 A N
JUDGE /
1.146 0 N
GROOM /
1.147 0 N
BRICKLAYER
2.148 A N
LION /
6.149 A N
GUITAR /
4.150 0 N
SPEAR /
5.151 0 N
SCREWS /
2.152 A N
WHALE /
6.153 0 N
HARP /
6.154 A N
ACCORDION /
5.155 0 N
SANDPAPER /
6.156 A N
CYMBALS /




Full Text

PAGE 1

EPISODIC MEMORY, INTEGRATIVE PROCESSING, AND MEMORYCONTINGENT BRAIN ACTIVITY DURING ENCODING By BRIAN G. HOWLAND A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2005

PAGE 2

Copyright 2005 by Brian G. Howland

PAGE 3

iii ACKNOWLEDGMENTS Although many have shaped the work presented here, in the interest of space, I mention but a few. Foremost, I thank Prof. Ira Fischler whose scientific curiosity, creativity and patience enabled me to come full circle from ERPs to fMRI and back again. I am indebted to Prof. Fischler for his many years of inspiration and tireless effort on my behalf, as well as on behalf of, literally, the thousands of University of Florida students whose lives he has enriched with his spirit. The other members of my committee, Prof. Keith Berg, Dr. Bill Perlstein, especially Dr. Lise Abrams, have provided valuable feedback and criticism. I give many thanks to Dr. Abrams for her encouragement when the work was in danger of stalling. I was also inspired by my office mate, Jesse Itzkowitz, and especially by the words of, and the example set by, Dr. Michael Membrino. I also thank the undergraduate research assistants, whose enthusiasm for the project was certainly displayed in their hard work. Among them, but without ignoring the others not mentioned, I thank especially S. Jones, B. Lawson, K. Tobago, J. Lapnawan, A. Persons, B. Yocum, A. Schweit, and A. Mejia. Finally, no acknowledgment of any undertaking this size would be complete without thanking family and friends. I thank my parents, Lois and Paul Howland, for being supportive of the middle-aged intellectual wanderings of their son; my father-in-law, Dr. Alan Sheppard for his scientific curiosity; and, most especially, for their great perseverance, patience, love and support, my family: Dena, Caley, Jonathan, and Julia. A special note of thanks to Dena, who selflessly put my interests ahead of her own at a most critical juncture. IÂ’ll always be grateful.

PAGE 4

iv TABLE OF CONTENTS page ACKNOWLEDGMENTS..................................................................................................iii LIST OF TABLES.............................................................................................................vi LIST OF FIGURES...........................................................................................................vii ABSTRACT.....................................................................................................................vi ii 1 INTRODUCTION.......................................................................................................1 Events, Context, and Episodic Memory.......................................................................1 Creating Episodic Memories: Theories and Data........................................................4 Working Memory Studies of Object-Location Binding.......................................5 ERPs and Episodic Encoding...............................................................................8 The Dm/subsequent memory effect...............................................................9 Dm and associative encoding......................................................................10 Dm and elaborative encoding......................................................................12 ERP summary..............................................................................................18 Functional Imaging and Associative Encoding..................................................20 The Present Investigation...........................................................................................26 2 EXPERIMENT ONE.................................................................................................28 Expected Results........................................................................................................29 Method.......................................................................................................................31 Participants..........................................................................................................31 Materials and Apparatus.....................................................................................32 Stimulus display and response recording....................................................32 EEG recording.............................................................................................32 Stimulus materials.......................................................................................33 Design.................................................................................................................34 Procedure............................................................................................................34 Results........................................................................................................................ 37 Behavioral Data..................................................................................................37 EEG Data Preprocessing.....................................................................................37 ERP waveforms...........................................................................................38 Statistical analysis of waveforms................................................................40 Bound condition..........................................................................................44 Separate condition.......................................................................................46

PAGE 5

v Bound vs. separate conditions.....................................................................46 Discussion..................................................................................................................46 3 EXPERIMENT TWO................................................................................................52 Expected Results........................................................................................................52 Method.......................................................................................................................54 Participants..........................................................................................................54 Materials and Apparatus.....................................................................................54 Stimulus display and response recording....................................................54 EEG recording.............................................................................................54 Stimulus materials.......................................................................................55 Design..........................................................................................................55 Procedure.....................................................................................................55 Results........................................................................................................................ 57 Behavioral Data..................................................................................................57 EEG Data............................................................................................................58 ERP waveforms...........................................................................................58 Statistical analysis of waveforms................................................................60 Bound condition..........................................................................................60 Separate condition.......................................................................................60 Bound vs. separate analysis.........................................................................64 Discussion..................................................................................................................65 4 GENERAL DISCUSSION........................................................................................69 Distinctive Aspects of the Present Approach.............................................................69 Memory-Related ERPs and Integrative Episodic Encoding......................................71 Comparisons to Previous Findings............................................................................74 Limitations and Future Directions.............................................................................78 APPENDIX A EXPERIMENT 1, STUDY PHASE WORD PAIRS.................................................82 B EXPERIMENT 1, PAIRED RECOGNITION TEST WORD PAIRS......................85 C EXPERIMENT 2, STUDY PHASE WORD PAIRS.................................................88 D EXPERIMENT 2, OBJECT WORD TEST ITEMS..................................................90 E EXPERIMENT 2, LOCATION WORD TEST ITEMS............................................93 LIST OF REFERENCES..................................................................................................96 BIOGRAPHICAL SKETCH..........................................................................................104

PAGE 6

vi LIST OF TABLES Table page 2-1. Time I ntervals in E x periment 1 during which Amplitude Differences were Significant................................................................................................................45 3-1.Time I ntervals in E x periment 2 during which Amplitude Differences were Significant................................................................................................................64

PAGE 7

vii LIST OF FIGURES Figure page 2-1. Schematic representation of a sin g le trial during the study phase in Experiment 1............................................................................................................36 2-2. ERPs to two words during study phase later shown as intact pairs during test in Experiment 1, Bound Encoding group..........................................................41 2-3. ERPs to two words during study phase later shown as intact pairs during test in Experiment 1, Separate Encoding group......................................................42 2-4. ERPs to two words during study phase later shown and correctly recognized as intact pairs during test in Experiment 1, Bound vs Separate............43 3-1. Behavioral p e rforma n ce in E x perime n t 2 ........... ....................................................... 57 3-2. ERPs to two words during study phase during test in E x periment 2, B ound Encoding group........................................................................................................61 3-3. ERPs to two words during study phase during test in E x periment 2, Separate Encoding group.........................................................................................62 3-4. ERPs to two words during study phase later shown and correctly rec o g ni z ed as intact pa i r s during test in E x perime n t 2, B ound vs. Sepa r a te .......... 63

PAGE 8

viii Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy EPISODIC MEMORY, INTEGRATIVE PROCESSING AND MEMORYCONTINGENT BRAIN ACTIVITY DURING ENCODING By Brian G. Howland August 2005 Chair: Ira Fischler, Ph.D. Major Department: Psychology A fundamental element of encoding an experience is establishing a link between an object or event and its spatiotemporal context. Current theories posit important roles for the prefrontal cortex and medial temporal lobe complex in successful episodic “binding.” We conducted two experiments to isolate the timing and scalp topography of event-context encoding effects using event-related brain potentials (ERPs). Participants were shown sequential (3 sec. apart) word pairs (e.g., ELEPHANT . BATHROOM) while their electroencephalograms (EEG) were recorded. Some participants were instructed to generate a single, integrated mental image while other participants generated a pair of separate images. Their ability to recognize intact pairs was then tested. As expected, recognition was better for pairs studied under Bound than under Separate instructions. ERP encoding differences between later recognized pairs and later forgotten pairs ( D ifferences associated with subsequent m emory performance or “ Dm” ), especially at frontal sites, were found for the Bound, but not the Separate, condition. These slow-

PAGE 9

ix wave differences were seen late following both words; however, the differences were of opposite polarities and contrasting morphology. An early, first-word difference between the waveforms for intact pairs that were subsequently recognized in the Bound versus the Separate conditions suggested different preparatory sets in the two tasks. In the second experiment, participants were given the same imagery tasks but tested subsequently for item, rather than pair, recognition. Unlike the first experiment, participants showed no difference in recognition performance by image generation task. As in the first experiment, there were ERP differences for correctly recognized vs. unrecognized items in the Bound condition, but these item-specific DmÂ’s were earlier and of a different topographic distribution than the DmÂ’s for pair recognition. No Dm effects were noted for the Separate condition. The contrasting ERPs between the Separate and Bound conditions, and the contrasting DmÂ’s for ERPs conditionalized on item versus pair recognition, suggest that relational processing contributing to successful object-location memory requires effortful processing, and is associated with frontal or prefrontal regions of the cortex.

PAGE 10

1 CHAPTER 1 INTRODUCTION This dissertation presents a pair of experiments that explore the cognitive and neural bases of episodic memory encoding. In particular, the studies examine the creation of a mental link between events and their spatiotemporal contexts by recording event-related brain potentials (ERPs) and conditionalizing those electrophysiological measures on subsequent memory performance in different tasks and under different instructions. Events, Context, and Episodic Memory As Tulving (1984) has noted, the basic unit of an individual’s perception of time is an “event;” that is, some occurrence at a given place at a given time. An ongoing series of events make up an “episode.” Episodic memory enables humans to “time travel;” that is, to place ourselves in the recent or distant past, or even, the imagined future (Tulving, 1985). Without episodic memory, one lives in a constant, immediate present, like the well-known amnesic musician, Clive Wearing. Successful episodic memory performance, therefore, requires that the episode’s context be linked with the focal event itself upon its initial occurrence. It is this linkage of spatiotemporal context and focal event that enables us to separate personally experienced, geographically distinct, but close in time, events from one another (“First I was in the kitchen, then I went into the dining room”). Moreover, we use episodic memory to distinguish identical or similar events by the order of their temporal occurrence (“I saw a dog run across the street yesterday. I saw the same dog run across the street this morning”).

PAGE 11

2 As important as this spatiotemporal linking of event and context is for memory, it apparently is not an obligatory or automatic one; indeed, one of the classic “sins of memory” (Schacter, 2001) is to remember an event but forget the context, or remember it falsely in the wrong context. A wide variety of experimental protocols have shown that both healthy participants (Chalfonte & Johnson, 1996; Henkel, Johnson, & DeLeonardis, 1998; Mitchell, Johnson, Raye, Mather, & D'Esposito, 2000; Mitchell, Johnson, Raye, & D'Esposito, 2000) as well as neurologically impaired patients (Turriziani, Fadda, Caltagirone, & Carlesimo, 2004) and can retain relatively good levels of item recognition yet show substantial decrements in the ability to identify either the spatiotemporal context in which a recognized item was presented or to recognize (an) additional feature(s) with which the to-be remembered item was to be associated. Thus, the processes that support item recognition or recall appear to be distinguishable from those that support contextual memory. A widely held analogy has been drawn between the ability to remember the contextual features of an event, and the subjective sense of remembering that has been termed “recollection.” Indeed, since the mid-1980’s, the qualitative distinction between “remembering” and “knowing” (Tulving, 1985) (or “recollection” versus “familiarity” for others) has been a central topic in the study of episodic memory (e.g., Yonelinas, 2002). The precise natures of the processes that support recollection are still largely undefined. It is clear that sensory information must be transformed into internal representations. However, to permit successful subsequent recollection, the elements of sensory experience, together with any relevant internally generated cognition and emotional

PAGE 12

3 states, must be combined in such a way that the experience is capable of later being reinstated (Paller & Wagner, 2002). Understanding the cognitive and neural processes that support successful episodic memory performance involves analyzing events both at encoding and at time of retrieval. While there has been a significant amount of research examining the retrieval-associated processes in episodic memory, far fewer studies have pursued the encoding processes that underlie episodic memory formation. For example, the degree to which, and conditions under which, attention plays a role in the encoding of context is unknown. Given the continuous stream of information one encounters, it seems likely that some degree of automaticity is required for everyday episodic memory to function efficiently. However, while Hasher and Zacks (1979) suggested that fundamental information such as time, spatial location and frequency of occurrence may be encoded relatively automatically, Craik (1989) suggested that, in some cases, attention might play an important role in the integration of an event with its context. Furthermore, the content and the context may interact to make the context more memorable (e.g., an elephant on the sidewalk is more memorable than a jogger on the sidewalk but a jogger in the zoo may be as memorable as an elephant in the zoo). Finally, it is unclear whether locations and their associated objects are bound together (and subsequently retrieved) in a single representation in holistic fashion, or if a link or pointer is formed that connects independently created and maintained episodic representations. The experiments presented in this dissertation are an attempt to identify some of the neural and cognitive encoding processes that support successful item + context

PAGE 13

4 retrieval and distinguish them from the encoding processes that support successful itemonly retrieval. Creating Episodic Memories: Theories and Data Long-term memory research has identified a variety of encoding factors (e.g., organization among items in a list; “depth” or degree of elaborative item processing, item frequency or familiarity) that are associated with successful long-term memory. The processes by which items or events and their contexts are bound, however, have been little explored. While traditional principles of associative learning (e.g., intraand extraitem organization) may apply, it is possible (or even likely) that other discoverable, Gestalt-like principles may be at work (Craik, 1989; Kounios, Smith, Yang, Bachman, & D'Esposito, 2001). In a series of unpublished studies, Craik (1989) explored the effects of stimulus integrability and attention on the degree to which item and context recall were independent. Overall, he found that context recall declined more rapidly initially than item recognition as attention was diverted during study, but, as context recall approached chance levels, item recognition then dropped quite rapidly. Moreover, for items and contexts that were thought to be more “integrable,” memory performance for items and contexts were more closely bound to one another. An important, yet unanswered question is what factors might affect integration of item and context. Craik suggested that the emotional content of the item-context could affect the ease of integration. Nevertheless, there has been little work, to date, on the cognitive and neural processes that successfully link events to their contexts in long-term memory. Three areas of research, reviewed below, may provide some guidance. First, a few working memory studies (Chalfonte & Johnson, 1996; Luck & Vogel, 1997; Mitchell et

PAGE 14

5 al., 2000a; Mitchell et al., 2000b; Prabhakaran, Narayanan, Zhao, & Gabrieli, 2000; cf., Bor, Duncan, Wiseman, & Owen, 2003) have examined the binding of an object and other feature information (e.g., spatial location, color) in working memory. If the processes engaged during working memory binding are utilized in long-term memory encoding, these studies are important in revealing the basis of episodic encoding. Second, use of physiological measures of brain activity such as electroencephalographic (EEG) recording to identify event-related brain potentials (ERPs) have revealed cognitive and neural processes that differentiate successful and unsuccessful long-term memory encoding. Third, event-related functional magnetic resonance imaging (ER-fMRI) studies have begun to reveal subcortical regions and areas in medial temporal (MTL) and prefrontal (PFC) cortex that distinguish successful and unsuccessful long-term memory encoding. Each of these areas of research will be discussed below. Working Memory Studies of Object-Location Binding In a number of studies, participants were required to briefly maintain two or more stimulus features or dimensions either separately, or in an integrated representation. These studies have shown that the ability to remember an object-location association is distinguishable from the ability to remember objects or features separately. Thus, for example, the deficits that older adults show in source memory cannot be attributed merely to the inability to remember a greater number of objects or features (Chalfonte & Johnson, 1996; Mitchell et al., 2000a; Mitchell et al., 2000b). The processes that underlie the short-term maintenance and manipulation of objects in working memory have been characterized as “reflective processes” (Johnson, 1992), but it is unclear whether these processes play a role in successful long-term memory for object and context binding. It is unclear, also, whether working memory and long-term memory encoding share a set of

PAGE 15

6 common cognitive processes. If so, these working memory studies may reveal some of the principles at work in successful episodic binding. Recently, there has been some convergence on this issue (Baddeley, 2000; Fletcher & Henson, 2001; Johnson, 1992; Wagner, 1999). Some investigators have claimed that working memory and long-term memory encoding share a common set of processes (Wagner, 1999) while others (Baddeley, 2000) have proposed common structural components between working memory and long-term memory. According to one view (Baddeley, 2000), binding of information from verbal and non-verbal slave systems takes place in an “episodic buffer” that stores that information in a multimodal code or representation. This bound information can then be passed back and forth between working memory and long-term “episodic” memory. According to this view, working memory and long-term episodic memory share an interlocking component (the episodic buffer) and a set of common processes (binding and maintenance in the buffer). An alternative view is one that emphasizes the commonality of the cognitive processes that underlie working memory and long-term memory performance (e.g., Fletcher & Henson, 2001; Johnson, 1992; Wagner, 1999) In its most developed form, this approach is process-specific (versus task-specific) and its goal is to identify and define the processes that underlie a variety of mnemonic and other cognitive phenomena. The most developed example of this type of model is Johnson’s (1992; Johnson & Hirst, 1993) multiple-entry, modular memory (MEM) framework. It presupposes that a common set of cognitive subprocesses act on a variety of cognitive tasks. Thus, according to the model, the subprocesses used in working memory binding operate in successful long-term episodic encoding as well. The MEM model includes high-level

PAGE 16

7 subprocesses such as initiating plans; discovering relations among stimuli; rehearsing and retrieving and lower-level subprocesses such as noting relations; shifting attention; refreshing currently active representations and reactivating stored representations. Johnson and colleaguesÂ’ work has examined the nature of the object-location binding deficits observed in older (relative to younger) adults (Chalfonte & Johnson, 1996; Mitchell et al., 2000a; Mitchell et al., 2000b). In various studies (Mitchell et al., 2000a; Mitchell et al., 2000b; Ranganath, Johnson, & D'Esposito, 2003; Raye, Johnson, Mitchell, Reeder, & Greene, 2002), she and her colleagues have suggested that deficits in feature binding may be attributable to difficulties in reactivating stored representations and in failing to refresh currently active representations. Although Johnson and colleaguesÂ’ studies provide evidence of a set of cognitive subprocesses that are involved in binding object and location features in working memory, their findings have not been extended to long-term memory encoding. In a recent study using event-related functional imaging, or fMRI (see following), however, Ranganath and colleagues (2003) have compared the areas of neural activation associated with working memory and successful long-term memory encoding. They found that separate face-encoding tasks, with identical stimuli but different encoding loads and retention intervals, activated similar areas of prefrontal cortex. While this finding suggests that the same cognitive (or at least neural) processes underlie certain working memory tasks and long-term memory encoding, methodological and experimental constraints make such conclusions tentative at best. Thus, a few studies (Chalfonte & Johnson, 1996; Luck & Vogel, 1997; Mitchell et al., 2000a; Mitchell et al., 2000b; Prabhakaran et al., 2000) have considered the issue of

PAGE 17

8 how working memory binds separate stimulus dimensions or features into an integrated whole. To date, however, there has been, to my knowledge, no direct measurement of long-term memory performance following the systematic manipulation of stimulus features, dimensions or combinations of stimuli to promote or inhibit working memory binding.A relatively direct way to study the neural and cognitive bases of long-term memory encoding is to record some physiological index of cognitive activity during an encoding event and then sort those records by subsequent memory performance during a later memory test. Such procedures have yielded reliable differences between laterremembered and later-forgotten items using both EEG and fMRI measures. Functional MRI studies suggest that these subsequent memory effects are associated with heightened medial temporal lobe (MTL) and prefrontal cortex (PFC) activation. However, the precise role of these structures in long-term memory encoding is still unclear. Findings from EEG studies, while isolating subsequent memory differences from about 400 ms onward following stimulus presentation, have varied substantially from one another both in the locus and timing of subsequent memory effects. As a result, EEG subsequentmemory studies have done little to identify the important processes that link event and context in memory encoding. ERPs and Episodic Encoding The use of stimulus-locked, event-related brain potentials has been a valuable tool in identifying a variety of neurocognitive memory processes over the past two decades. For example, a robust finding is that correctly recognized old items show a greater positivity than correctly identified new items. These “Old-New” effects have been shown with a large variety of stimulus materials in many test formats. A less robust, but wellreplicated, finding is that, under certain circumstances, studied stimulus materials that

PAGE 18

9 will be subsequently recognized as “Old” show a greater positivity than items that will later be classified erroneously as “New.” These subsequent memory effects (or “Dm” D ifferences associated with subsequent m emory performance) have been used to examine encoding in long-term memory. The Dm/subsequent memory effect Most ERP studies of encoding have used the same basic paradigm. In this paradigm, electrical activity is recorded while subjects are presented with stimuli that are subsequently tested under either explicit (e.g., recognition, cued recall) or implicit (e.g., stem-completion) conditions. These records are then classified according to subsequent memory (recalled/recognized vs. unrecalled/unrecognized) performance. Quite a number of investigators (Besson & Kutas, 1993; Duarte, Ranganath, Winward, Hayward, & Knight, 2004; Fabiani & Donchin, 1995; Fabiani, Karis, & Donchin, 1986; Fernandez et al., 1999; Fernandez et al., 1998; Friedman, 1990; Friedman, Ritter, & Snodgrass, 1996; Friedman & Trott, 2000; Gonsalves & Paller, 2000; Guo, Voss, & Paller, 2005; Guo, Zhu, Ding, Fan & Paller, 2004; Karis, Fabiani, & Donchin, 1984; Lian, Goldstein, Donchin, & He, 2002; Mangels, Picton & Craik, 2001; Mecklinger & Muller, 1996; Munte, Heinze, Scholz, & Kunkel, 1988; Neville, Kutas, Chesney, & Schmidt, 1986; Sanquist, Rohrbaugh, Syndulko & Lindsley, 1980; Schott, Richardson-Klavehn, Heinze, & Duzel, 2002; Smith, 1993; Summerfield & Mangels, 2005; Van Petten & Senkfor, 1996; Weyerts, Tendolkar, Smid, & Heinze, 1997; Yovel & Paller, 2004) have shown differential scalp recorded electrical activity at encoding between subsequently remembered and unremembered stimuli (“difference associated with memory” (Dm) or “subsequent memory effects”). These differences usually, but not always, consist of a greater positivity for remembered items than for unremembered items, although the

PAGE 19

10 timing and topography of these effects vary depending upon the precise experimental conditions. Some published studies report the effects as containing a frontal maximum. Others show robust midline effects; and yet other studies have a posterior maximum (Johnson, 1995; Rugg & Allan, 2000). Some portion of this variance may be due to the difference in stimulus materials across studies (Johnson, 1995). Dm and associative encoding Only a handful of published studies (Guo et al., 2005; Fernandez et al., 1998; Kounios et al., 2001; Weyerts et al., 1997; Yovel & Paller, 2004) have sorted encoding ERPs for associative information by memorial success. Kounios and colleagues (2001) isolated electrical activity associated with faster (better) subsequent memory for associated words than for more slowly recognized associated words. They explored whether two proposed processes of cognitive association, juxtaposition and fusion, have different neural bases. They presented word pairs that either could be fused to create a novel concept (e.g., computer-virus ) or could not easily be combined into a single unique concept (e.g., salt-pepper ). Using a dense electrode array, they measured electrocortical activity as participants decided whether or not fusion was possible. Subsequently, pairs were re-presented, one half in the same order (e.g., salt-pepper), one half reordered (e.g., virus-computer). Participants judged whether the pairs were as presented previously or reordered. Faster word pair order judgments were deemed to represent better memory (and hence better encoding). A median split of the word pair order judgment RTs showed that fusible pairs that were judged fusible more quickly at study were also responded to more quickly at test. Conversely, non-fusible (juxtaposed) pairs to which participants responded more quickly at study were responded to more slowly at test.

PAGE 20

11 As with the behavioral data, ERP data were classified by response speed at test. Both juxtaposed and fusion pairs showed an effect at study of the speed of the responses at test. This “subsequent memory effect,“ however, is different from the standard subsequent memory effects discussed above. First, it is important to emphasize that the ERP memory differences are between (ostensibly) better and more poorly remembered word pair orders, not between remembered and unremembered items. Second, although the fused pairs showed a greater positivity for faster than for slower judgments, the juxtaposition pairs showed the opposite pattern, with slower word order judgments being associated with a greater positivity than faster judgments. These retrieval speed effects persisted throughout the recording epoch. Kounios and colleagues interpreted the ERP retrieval differences and the subsequent localization of those differences to the right prefrontal cortex as indexing processes associated with an attempt to fuse the words of the pair. Such processes might include maintenance of the pair in working memory, construction of candidate fusions, and evaluation of these fusions. Implementation of such processes would explain why the ERP effects would be present in the early epoch (200 – 800 ms) of both fusion and juxtaposition pairs, but would persist into the middle epoch (800 – 2100 ms) only for juxtaposition pairs – in which the search for an appropriate fusion might be expected to continue. Weyerts and colleagues (1997) examined the ERP correlates of two semantic encoding tasks. One task required determining whether either word of a pair was associated with a given color; thus the task demanded semantic evaluation of both words, but the associative relationship between the pair was irrelevant (nonassociative task). The second task required participants to judge whether the words of the pair were

PAGE 21

12 semantically related to one another. Again, semantic analysis was required of each word of the pair, but the task further required participants to judge the items’ interrelatedness (associative task). Old and new word pairs were presented in a subsequent incidental recognition memory test. Subsequent memory effects for words encoded in the associative tasks were found at frontal sites, with right frontal effects greater than left frontal effects. No subsequent memory effects were found for the nonassociative encoding task. The authors interpreted the difference in subsequent memory effects between the associative and nonassociative tasks as reflecting the creation of a more elaborated memory trace in the associative task than in the nonassociative task. Dm and elaborative encoding Another group of studies (Duarte et al., 2004; Fernandez et al., 1998; Friedman & Trott, 2000; Guo et al., 2004; Mangels et al., 2001; Schott et al., 2002; Smith, 1993) has observed subsequent memory effects that appear to be associated with elaborative processing (when the task does not explicitly demand that participants process stimuli relationally). For example, Mangels, Picton and Craik (2001) had participants memorize lists of 45 words for subsequent explicit recall and recognition tests. The words were studied under either full attention or divided-attention (not discussed) Participants were given no particular instructions for memorizing the words. At test, participants first were given three minutes to recall as many of the words as possible. Thereafter, they were shown a series of words (50% old) to which they responded, “remember,” “know,” or “new” (following the Remember-Know paradigm of Tulving, 1985). Subsequent memory effects showed both an anterior positive and a posterior negative sustained potential. Mangels and colleagues pointed out that the sustained anterior positivity was consistent with a few earlier findings in which late, sustained anterior subsequent memory effects

PAGE 22

13 were induced when the study task involved biasing participants toward the elaboration of the stimuli. They pointed out that it was unclear which types of elaborative processes were involved in their task. They speculated, however, that the anterior positivity may have consisted of two separate components: (1) an earlier, left-sided positivity representing activation of processes involved in associational or relational processing between the stimulus and information in semantic memory and (2) a later, more rightsided effect representing retrieval of previous list items no longer in current awareness and comparison processes necessary for strategic organization of list items. Despite their speculation, the task used by the investigators did not explicitly manipulate any of these purported processes; so further evidence is necessary to confirm their claims. Mangels and colleagues attributed the sustained posterior negativity to the representation of the concrete object represented by the word or the representation of the word itself. They pointed out, however, that such posterior inferior sustained negativity had been identified in only one other study. They attributed this lack of similar findings in the subsequent memory literature to their choice of using an average, rather than a mastoid or earlobe, reference. They pointed out a similar negativity at the mastoid electrodes (TP9/10) which would have been subtracted out had they been used as a reference. Mangels and colleagues also identified a parietal positivity (P280) and a frontotemporal negativity (N340) that separated old words that were subsequently missed from recognized words (but which did not distinguish between R and K items). They concluded that processing up to about 340 ms consisted of the perceptual analysis and selection of the item as task relevant followed by item-specific semantic processing

PAGE 23

14 (N340). Once this processing is completed, the information is made available to the MTL system for long-term storage (P550). Thereafter, relational and elaborative processing takes place via the sustained interaction between frontal and inferior temporal regions beginning at about 1000 ms. In an aging study (Friedman & Trott, 2000), participants studied sentences containing two unrelated nouns (e.g., “The dragon sniffed the fudge.”) for a subsequent explicit recognition test. At test, participants made “Old/New” recognition judgments, followed by “Remember/Know” decisions to items judged “old.” Finally, participants made temporal order decisions (List 1 vs. List 2). Study ERPs, sorted by recognition decision, showed a widespread Dm effect among young participants for Remember decisions only but for both Remember and Know decisions in the older participants. Friedman and Trott proposed that encoding of items by older participants was less contextually rich, even for old items judged “Remember,” than encoding of “Remember” items by younger participants. Alternatively (or in conjunction with this shallower encoding proposal), Friedman and Trott suggested that older participants might have applied a more liberal criterion to the Remember/Know judgments than did the younger participants. Moreover, they pointed out that, unexpectedly, there was no correspondence between Dm effects associated with Remember/Know judgments and source list judgments. They noted that Remember judgments could be assigned if any aspect of the encoding session was retrieved, regardless of whether the list from which the item was taken was retrieved. Thus, Remember responses may or may not have been accompanied by correct source list judgments.

PAGE 24

15 Therefore, although Friedman and Trott used a nominally associative encoding task in which participants were given two unrelated words within a sentence, neither the test (identification of each word as old or new), nor the instructions (“memorize the nouns for a subsequent memory test) explicitly demanded that the words be encoded together. In fact, at test, rearranged old items required “Old/Old” responses. Thus, encoding the items as a pair could make it more difficult to respond “old” to the second word if it were repaired. Therefore, Friedman and Trott showed a widespread Dm effect that varied by the response type (Remember/Know) and interacted with age. The study does not, however, clarify the nature of the encoding at study that produced the Dm although their suggestions that lack of differences between Remember and Know responses for older participants reflected shallower encoding, or less elaborated traces, is attractive. A recent study, however, suggests that ERP effects produced by a levels-ofprocessing (LOP) manipulation differ in onset, duration, and topography from ERP Dm effects. Schott and colleagues (2002) found an LOP effect (deep > shallow) at frontocentral regions that began at about 600 ms after stimulus onset and lasted until the end of the recording interval. This contrasted with widespread Dm effects from 600 800 ms that were associated with only the shallow encoding condition and a right frontal Dm from 900 – 1200 ms associated with both study conditions. Schott and colleagues argued that the LOP effects might represent differences in retrieval from semantic memory required by the two tasks whereas the Dm effects might represent the establishment of an episodic memory trace. They disagreed with Van Petten and Senkfor’s (1996) conclusion that Dm effects for meaningful words, but not for meaningless, novel visual patterns,

PAGE 25

16 suggests that the Dm reflects retrieval from semantic memory and point out that Van Petten and Senkfor’s “Dm” effects might have consisted of both differences in study processing and in establishment of the memory trace. Schott and colleagues argued further that the early and late Dm effects (which were modulated by LOP) likely reflect different processes. For example, the early, widespread Dm (which was present only for words studied in the shallow condition) might have been associated with the formation of memory traces containing distinctive orthographic/phonological information. On the other hand, the late, right prefrontal Dm, which occurred with both study conditions, might signify the establishment of a memory trace with semantic-associative information. They argue that occurrence of this Dm in the shallow study condition might simply reflect the activation of semantic-associative information during the shallow study task and note that a similar Dm was found during a rote rehearsal task by Fernandez and colleagues (1998). Finally, Fernandez and colleagues (1998) interpreted the existing subsequentmemory-effect literature as consisting of two effects: a centroparietal effect associated with rote encoding strategies, regardless of distinctiveness, and a frontal effect that is associated with elaborative encoding (Fabiani, Karis & Donchin, 1990; Karis, Fabiani & Donchin, 1984; Weyerts et al., 1997). In their own study, they examined the differences in encoding ERPs associated with item distinctiveness, associative elaboration, or other “direct” encoding processes. They presented 40, 15-item, word lists, consisting of high and very low frequency words. Each list was followed by a brief distraction period and a free recall task. One half of the lists were blocked by word frequency and one half of the lists contained both high and very low frequency words. Fernandez and colleagues

PAGE 26

17 assumed that associative (inter-item) encoding would facilitate recall of the highversus low-frequency words in the blocked lists, and yield intermediate (relative to lowfrequency/mixed and high frequency/blocked) recall rates in the mixed lists. Moreover, they assumed, consistent with Karis, Fabiani & Donchin (1984), that the amplitude of the N400 and P300 (LPC) would be associated with distinctiveness and thus should be associated with low-frequency, more than high-frequency, words. Any subsequent memory effects unrelated to distinctiveness detection should be dissociable in topography, amplitude, and/or time course from the enhanced N400/LPC. Moreover, a subsequent memory effect that was greater for highthan for low-frequency words, and enhanced further in the blocked condition, would be likely to correspond to associative processing. If the subsequent memory effect did not interact with word frequency and presentation (blocked/mixed), then it would be likely to be related to nonassociative encoding processes. Subsequent memory effects were dissociable into separable components. One effect arose at centroparietal and frontopolar sites at about 200 ms for high-frequency words and at about 350 ms for low-frequency words. It shifted to a single frontopolar maximum at about 900 ms that differed both in topography and time course from the distinctiveness effects associated with word frequency. A second subsequent memory effect, located at a right frontopolar site at between 900 and1300 ms, occurred for high-, but not low-, frequency words. Fernandez and colleagues concluded that they had identified subsequent memory effects that were associated neither with distinctiveness nor with associative processing. Although the second effect was located at right frontobasal electrodes, as predicted, it was associated only with successfully recalled

PAGE 27

18 high-frequency words. Further inspection showed that only unsuccessfully recalled highfrequency words failed to elicit any effect at this site; all other types (high-frequency, successfully recalled and low-frequency, successfully and unsuccessfully recalled) showed greater amplitude at the frontobasal sites. However, there was no interaction with presentation type. ERP summary A modest number of studies have examined the ERP correlates of encoding. A few have compared study phase data across encoding manipulations (e.g., levels of processing). Numerous methodological difficulties arise with such comparisons (e.g., equating memorial success across tasks) but it appears that “deep” encoding (relative to shallow encoding) yields a long-lasting, centroparietal positivity that onsets about 600 ms after stimulus presentation. The observed ERP differences might reflect retrieval from semantic memory required in “deep” encoding, but such conclusions are mostly speculative. A second group of ERP encoding studies have compared the ERP correlates of successfully recognized (or recalled) items with old items that are unsuccessfully recognized (or recalled). These subsequent memory effects (or differences associated with memory “Dm”) have been produced using both recognition and recall tests across a wide variety of experimental conditions. Although some commentators have found it difficult to generalize from the disparate subsequent memory effect findings, it appears that hypotheses regarding the nature of the processes underlying the later subsequent memory effects may be tested. For example, Schott and colleagues have speculated that the late, right frontal Dm observed in their study, as well as in others’ studies, may signify the formation of memory traces containing semantic associative information.

PAGE 28

19 Nevertheless, the essence of episodic memory is that it includes both the item and the item’s context. Although some ERP studies have purported to examine memory for “context” or “source” memory, these studies have largely only had participants associate a single perceptual attribute (e.g., voice, temporal order, spatial location). Moreover, the nature of such experiments is to repeat a non-meaningful attribute across items, rather than having participants encounter each item in a unique context, which is likely to support retrieval differently than when the context (or perceptual attribute) does not possess unique characteristics. A few published event-related potential (ERP) studies (Duarte et al., 2004; Friedman & Trott, 2000; Guo et al., 2005; Kounios et al., 2001; Mangels et al., 2001; Smith, 1993; Schott et al., 2002; Weyerts et al., 1997; Yovel & Paller, 2004) have examined context-event or associative encoding. Some of those studies have used the “Remember-Know” paradigm to distinguish subsequent memory with context recognition from subsequent memory without context recognition. The findings across these studies are inconsistent. For example, Smith (1993) found that the subsequent memory effects (Dm) were similar in timing and topography, regardless of whether they were associated with “R” or “K” responses. On the other hand, Friedman and Trott, (2000) found significant Dm effects for subsequent “R,” but not “K,” responses in young participants. However, older participants showed Dm effects to both “R” & “K” responses. Unlike Smith, however, Friedman and Trott found that the Dm effect was lateralized (L > R), in the young participants (although not in the older participants). In contrast to the findings of both Smith (1993) and Friedman and Trott (2000), Duarte and her colleagues (2004) found transient, left frontal Dm effects for items later

PAGE 29

20 classified as “K” and sustained, bilateral (with right > left) Dm effects for items later classified as “R.” Similarly, Mangels, Picton & Craik (2001) found left-lateralized, fronto-temporal subsequent memory effects at N340 for both R and K responses, which didn’t differ from one another. The differences across these studies are difficult to reconcile. Of the remaining studies, two (Guo et al., 2005; Yovel & Paller, 2004) involved the encoding of novel faces with associated information (names, occupations). Although Dm effects were observed for encoding of face-name and face-occupation associations, these effects were neither lateralized, nor transient. Rather, they were long lasting and topographically central or centro-posterior. Functional Imaging and Associative Encoding A large number of functional MRI and PET studies have examined neural activity at encoding (see Cabeza & Nyberg, 2000; Fletcher & Henson, 2001; Fletcher, Frith, & Rugg, 1997; Mayes & Montaldi, 1999; Nyberg, 2002; Schacter & Wagner, 1999 for reviews). The development of event-related fMRI enabled investigators to sort these encoding records by subsequent memory performance. Since then, a large number of studies (Baker, Sanders, Maccotta, & Buckner, 2001; Brewer, Zhao, Desmond, Glover, & Gabrieli, 1998; Buckner, Wheeler, & Sheridan, 2001; Casasanto et al., 2002; Clark & Wagner, 2003; Daselaar, Veltman, Rombouts, Raaijmakers, & Jonker, 2003; Davachi, Maril, & Wagner, 2001; Davachi, Mitchell & Wagner, 2003; Davachi & Wagner, 2002; Henson, Rugg, Shallice, Josephs, & Dolan, 1999; Jackson & Schacter, 2003; Kensinger, Clarke, & Corkin, 2003; Kirchoff, Wagner, Maril, & Stern, 2000; Otten, Henson, & Rugg, 2001; Otten & Rugg, 2001; Ranganath et al., 2004; Reber et al., 2002; Rypma & D'Esposito, 2003; Sommer, Rose, Weiller & Bchel, 2005; Sperling et al., 2003; Stark &

PAGE 30

21 Okado, 2003; Strange, Otten, Josephs, Rugg, & Dolan, 2002; Wagner et al., 1998) have identified subsequent memory effects in prefrontal cortex (PFC) and the medial temporal lobe (MTL) complex. A subset of these studies has examined the encoding that underlies source memory, memory for context, recollective memory or associative encoding and they reveal activations in MTL (Davachi et al., 2003; Davachi & Wagner, 2002; Sommer et al., 2005), PFC (Cansino, Maquet, Dolan, & Rugg, 2002; Henson et al., 1999) or both (Brewer et al., 1998; Kensinger et al., 2003) that are linked to subsequent successful associative, source or contextual memory performance. The authors of these papers have proposed different processing roles for these regions (and subregions within them) that contribute separately to the formation of contextually bound, episodic memories. For example, Davachi and colleagues (Davachi, 2003) have isolated subsequent memory effects in hippocampus, and perirhinal, and parahippocampal (PHC) cortices. Importantly, the activated regions were dissociated by task (item recognition and source memory). Greater activation of hippocampus and left PHC at study was associated with accurate item recognition accompanied by correct source memory than with accurate item memory alone, but not with successful versus unsuccessful item recognition. Greater perirhinal cortex activation, on the other hand, was associated with correct recognition (item alone and item + source) than for missed items, but not with accurate source memory versus memory for item without source. Finally, Davachi and colleagues found that two regions of anterior left inferior prefrontal cortex were activated more during encoding of items for which source was subsequently correctly identified than for encoding of items that were subsequently recognized without recollection of source. Similarly, Davachi and colleagues (2002) found subsequent

PAGE 31

22 memory effects in bilateral hippocampus for items encoded in a relational encoding task, but not during rote rehearsal. In another study of the MTL and relational processing, Bar and Aminoff (2003) used fMRI to explore the recognition of strongly contextually identified objects (e.g., hardhat) with the recognition of items that have only weak contextual associations (e.g., fly). They found that portions of the parahippocampal cortex (parahippocampal place area or “PPA”) and retrosplenial cortex, areas previously identified in spatial processing and episodic encoding were activated more by recognition of strongly context-bound objects than objects that have only weak contextual associations. Moreover, they distinguished between anterior and posterior portions of the parahippocampal cortex that were associated more with non-spatial context recognition and with place-specific context recognition, respectively. They concluded that this PHC/RSC network might play a role in the formation of episodic memories by inputting to the hippocampus familiar associations established through experience (e.g., “which objects belong in a kitchen”). They speculated that this information is subsequently used by the hippocampus to represent specific instances (e.g., “which objects belong in my kitchen”) of this knowledge (citing Buckner, 2000). Taken together, these studies provide substantial evidence that MTL structures play an important role in the relational processing of verbal and visual pictorial stimuli for later subsequent retrieval of those relations. Bar and Aminoff have proposed that well-established general associative knowledge might be represented in a PHC/RSC network that is subsequently input to the hippocampus for participation in episodic encoding processes. The studies by Davachi and colleagues suggest that encoding of certain contextually related-information (i.e., processes engaged

PAGE 32

23 during a verbal stimulus’ original presentation) relies on different neural substrates (e.g., PHC/RSC) than the encoding of other relational information (e.g., semantic relations among verbal stimuli (hippocampus). These studies do little, however, to clarify whether the associative processes engaged by these different neural systems are mutually exclusive, the same or partially overlapping. Henson and his colleagues (1999) and Brewer and his colleagues (1998) used Tulving‘s Remember/Know procedure to assess the phenomenological state associated with retrieval of old information. By sorting hemodynamic records at encoding that were associated with subsequent Remember or Know responses, the investigators attempted to measure indirectly the neural correlates of encoding associated with recollective or nonrecollective states of recall (Henson et al., 1999). Although they found prefrontal subsequent memory effects associated with associative encoding, use of the Remember/Know technique may have confounded the subsequent memory effects associated with Remember and Know responses with the strength of item memory (Cansino et al., 2002). For example, in the study by Henson and colleagues, the procedure may have produced fewer Know than Remember hits and greater Know than Remember false alarms. If this is the case, Know responses may have represented guesses more than veridical memory responses. In an effort to measure the phenomenological state at retrieval more directly, Rugg and his colleagues (Cansino et al., 2002) used a paradigm similar to that employed in ERP and fMRI studies of source memory. Cansino et al. had participants make animateness judgments to visually presented colored images. Each image was presented randomly in one of the four quadrants delineated on the computer screen. Following

PAGE 33

24 encoding of the objects, a recognition phase was presented and participants pressed a button to indicate New or, if Old, a button corresponding to the location where the image had been presented. Cansino and colleagues found subsequent memory effects associated with associative encoding in right lateral occipital and left prefrontal cortex, among other areas. Consistent with earlier findings of Rugg and colleagues (Otten et al., 2001; Otten & Rugg, 2001) they argued that subsequent memory effects represent a subset of the neural activation required for encoding in any given task. They also claimed that the subsequent memory effects reflect the relatively greater semantic and perceptual processing received by certain items. Cansino and colleagues speculated on the relationship between perceptual and semantic processing contributions to the subsequent memory effects. They suggested that the perceptual and semantic processing may have contributed independently to the subsequent memory or effects or, alternatively, greater perceptual processing may have been mediated by the occipital cortex and fed into the prefrontal cortex, allowing for more elaborated and, thus, better remembered, memory traces (Cansino et al., 2002). Interestingly, Cansino and colleagues failed to obtain subsequent memory effects in MTL, consistent with previous null findings by Rugg and colleagues (Otten et al., 2001; but see Otten & Rugg, 2001). They speculated that both Remember and Know responses may have reflected relatively high levels of hippocampal encoding related activity, or that the null finding simply reflected a lack of statistical power sufficient to detect such activity. Finally, Kensinger and colleagues (2003) also measured indirectly participants’ recollective state associated with memory for visually encoded words that were given semantic judgments (“abstract” or “concrete”). In an accompanying behavioral study,

PAGE 34

25 participants made semantic judgments to visually presented words while performing either an easy or a difficult auditory discrimination task. Subsequently, participants were given a memory test and were required to make Remember or Know responses to words judged “old.” Kensinger and colleagues found a significant effect of distraction task (Easy vs. Hard) as well as an interaction between distraction task and memory strength (Remember vs. Know). They concluded that the task manipulation altered the type of memory trace formed and used the distraction task, (followed by a yes-no recognition task, also performed under distraction), as the independent variable in the imaging experiment. Kensinger and colleagues found subsequent memory effects in bilateral PFC and left MTL. However, in left PFC, these effects were for items encoded only under easy distraction, whereas right PFC subsequent memory effects were obtained for encoding under both easy and difficult distraction conditions. Likewise, PHC activation predicted subsequent memory performance under both distraction conditions whereas left anterior hippocampal activation predicted subsequent retrieval only for items encoded under easy distraction. The investigators concluded that the formation of detailed, contextually rich memory traces depends on activation of the left PFC and left anterior hippocampus. They also concluded that the formation of contextually rich, detailed traces depends on the activation of a subset of the neural processes activated by successful encoding generally. Habib and colleagues (2003) recently reevaluated the Hemispheric Encoding and Retrieval Asymmetry (HERA) model proposed by Tulving and colleagues (Nyberg, Cabeza, & Tulving, 1996; Tulving, Kapur, Craik, Moscovitch, & Houle, 1994). They concluded that existing PET and fMRI data still support the conclusion that the left PFC

PAGE 35

26 shows greater activation in encoding tasks (relative to retrieval tasks) than the right PFC. Conversely, the right PFC shows greater activation during retrieval tasks (relative to encoding tasks) than the left PFC. They asserted that such a process-specific lateralization could co-exist with the material-specific (e.g., verbal vs. non-verbal materials) lateralization, observed by a number of investigators. Habib and colleagues reiterated the notion that the preferential left PFC activation during episodic encoding is likely to be associated with semantic processing of incoming and on-line information. Recent work involving transcranial magnetic stimulation (rTMS) supports HERA showing disruptive effects to encoding by application of magnetic pulse trains to left PFC and degradation of retrieval by application of magnetic pulses to right PFC (Rossi et al., 2001). Thus, there is conflicting evidence as to whether the right prefrontal cortex is involved in associative encoding. There are few fMRI studies of long-term memory studies that show right PFC subsequent memory effects, and the HERA model accords the left PFC a predominant role in LTM encoding. In contrast, a few ERP studies (Kounios et al., 2001; Schott et al., 2002; Weyerts et al., 1997) have identified subsequent memory effects at right prefrontal electrodes for associative encoding. Furthermore, the right PFC finding in these studies is supported by fMRI findings of Prabhakaran and his colleagues (2000) and Johnson and her colleagues (Mitchell et al., 2000a) in working memory studies of object and location binding. The role of the processes underlying these effects is still controversial. The Present Investigation One challenge, therefore, is to (1) examine the creation of memorable episodes by identifying those cognitive and neural processes that link events and their contexts; and

PAGE 36

27 (2) determine whether or not those processes are consistent with the current theoretical accounts of the relationship between working memory and episodic memory. The experiments presented in this dissertation are an attempt to tackle a piece of that challenge. The first purpose of this dissertation is to test whether associating concrete, highly imageable items and complex contextual scenes into events (e.g., elephant at an intersection) in working memory is a key component of successful long-term episodic memory performance. The second purpose of this dissertation is to test whether successful long-term memory “binding” produces a temporally and topographically unique electrocortical “signature” during working memory that distinguishes it from unsuccessfully bound items and contexts as well as from unbound, but remembered, items and contexts.

PAGE 37

28 CHAPTER 2 EXPERIMENT ONE As described above, there is substantial evidence that the neural underpinnings of the relational binding (both episodic and associative) which subserve long-term memory can be observed by two basic strategies: a comparison of encoding tasks that do, and do not, require some sort of relational or contextual encoding between elements; and a posthoc sorting of encoding events that do, and do not, result in subsequent memory for contextual versus item information. But there is little consensus about the conditions that produce such a binding “fingerprint.” Moreover, the details (timing and topography) of the fingerprint are even less obvious, in large part due to the inconsistency with which subsequent memory effects are detected. One likely source of this inconsistency is the wide variety of encoding tasks, on the one hand, and ways of assessing memory, on the other, that have been used by different researchers. In some cases, for example, the relational task is qualitatively different from the nonrelational task on dimensions other than relational processing as such. More challenging memory tasks (e.g., recall) and test responses that reflect more elaborative memories (e.g., “Remember” vs. “Know” responses) more often produce subsequent memory effects than simple yes no recognition, but at the same time, are themselves complex enough to introduce constructive and inferential processes at retrieval that may interact with any encoding processes being studied. In each of the present experiments, a simple yes-no recognition task was used to minimize the role of retrieval factors in any subsequent-memory effects. As importantly,

PAGE 38

29 a task was adopted in which the encoding processes were as similar as possible, while some groups attempted to integrate the two stimuli, and others did not. The materials. Concrete, imageable words, as well as pictures, have been shown to elicit Dms, whereas abstract words and symbols are less likely to produce subsequent memory effects. The requirements of the stimulus materials were three-fold. First, they had to consist of familiar, integrable, item and spatial location pairs. Second, they had to produce sufficient levels of recognition failure during a paired recognition test. Third, in contrast to the materials of Craik (1989), each item (and pair) had to be unique, to avoid potential cross-pair interactions during either encoding or retrieval that could complicate the subsequent-memory analysis. As discussed below, the stimulus materials selected fulfill each of the foregoing criteria. The task. An experimental task in which participants were required to associate (bind) unique spatial locations and objects was contrasted with a condition in which participants would be required to process the same materials in the same way without integrating the two. Following Bower (1970), a task was designed to promote integrative encoding in a paired recognition task. Expected Results. Following Bower (1970), we expected that the overall pattern of results for paired recognition performance between the two groups (Bound and Separate) would reflect better recognition of previously presented pairs by the group that formed integrated images (Bound) than by the group that maintained separate images (Separate). Overall, we expected the between-group manipulation to produce similar ERPs at encoding – due to the identity of the stimulus materials and the similarity of the experimental conditions between the two groups. We anticipated that distinctive between-group ERPs would most

PAGE 39

30 likely be reflected to the second word – at which point the integrative processing in the Bound, but not the Separate, condition could begin. Alternatively, however, task-related ERP contrasts could be associated with attentional or other “set” differences. These differences could show up as broader differences in the ERP patterns encompassing the first word, and even the pre-stimulus interval. However, we predicted that, in addition to differences in the second word interval, the task manipulation would be most likely to reflect differences late in the first word interval, as participants prepared for the second word. In any event, we expected ERP task differences to be minimized by our decision to manipulate the task as a between-subjects factor, thus producing significantly more variability for it than for the within-subjects factor (memory performance). While we expected the overall pattern of results between the two groups to be similar, primarily yielding differences following the second word – where integrative processing would be reflected in Bound but not Separate ERPs we anticipated that the subsequent memory effect differences (Dms) would reflect the relative role of the first and second words in associative encoding. In this regard, our method provides a unique advantage over previous paradigms that have used a limited number of associative possibilities (male/female voice; limited spatial locations, etc.). Thus, the “binding” or relational processing could only take place upon presentation of the second word. Consequently, we expected a unique signature in the Dm to the second word for cases in which paired recognition failure was a result of unsuccessful binding of the object and location. Given that several of the few experiments in which Dms were produced for associative or elaborative processing yielded frontal Dm effects (Duarte et al., 2004; Fabiani et al., 1990; Fernandez et al.,

PAGE 40

31 1998; Karis et al., 1984; Kounios et al., 2001; Mangels et al., 2001; Schott et al., 2002; Weyerts et al., 1997) we anticipated that associative subsequent memory effects would be produced to the second, but not the first, word, at frontal electrode sites, with right frontal locations possibly showing greater effects than left. Moreover, we expected these differences to begin later in the interval (~1000 ms after word 2 onset) and be sustained throughout the interval. We also predicted that a set of frontal transient Dms, similar to those observed by Mangels and colleagues (2001) and Duarte and colleagues (2004), beginning as early as 400 ms following word onset, might be produced to both the first and second words. Based on previous findings, these earlier Dms would likely be either left-lateralized or bilateral. Method Participants Seventy-four undergraduates (47 females) at the University of Florida participated in this experiment. Additionally, 37 undergraduates participated as pilot participants during development of the tasks and materials. Twenty-two of these pilot participants were used to ensure that Bound and Separate encoding produced different levels of subsequent memory performance and the stimulus delivery and data recording program was operating as anticipated. Fifteen additional pilot participants were used to test the effectiveness of an alternative instruction directing participants to respond”old” only if they were sure that they had seen the pair as presented before. Participants were randomly assigned to the two experimental groups. Participants received credit toward an introductory psychology course requirement. Of the 74 participants who began the experiment, three left without completing the test phase so neither behavioral nor EEG

PAGE 41

32 data were useable for those participants. 11 other participants had too few (< 10) incorrect responses to the memory test portion to permit construction of reliable (based on visual inspection) waveforms. Finally, as discussed in the Results section, of the remaining 60 participants, various technical and signal-to-noise problems prevented analysis of another 21 EEG datasets. Materials and Apparatus Stimulus display and response recording The entire experiment was administered in a small, dimly lit room (approximately 5.5’ x 6.5’) on a personal computer using a conventional CRT monitor with a screen size of approximately 13” measured diagonally. Participants were seated about 24” from the monitor. A program written in the Delphi programming language (Borland Software Corp.) controlled stimulus presentation and recording of behavioral responses. Participants viewed stimulus items in the middle of screen and responded to stimulus events by using a standard two-button mouse. During the recognition phase, participants made affirmative recognition responses by pressing the left mouse button and negative responses using the right mouse button. EEG recording Electroencephalographic activity (EEG) was recorded using a standard elastic cap (Electro-Cap International, Inc.) with 13 embedded tin electrodes placed in standard 1020 system (Jasper, 1958) locations (Fp1, Fp2, F3, F4, FT7, FT8, Cz, TP7, TP8, P3, P4, O1, O2). The cap was linked to a set of bioamplifiers (SA Instrumentation Co.). Data were filtered (high pass 0.01 Hz; low pass 50 Hz), amplified 50K, digitally converted using National Instruments analog to digital converter and stored for subsequent off-line analysis. In addition to the scalp-recorded EEG, horizontal electro-oculogram (hEOG)

PAGE 42

33 was recorded with a pair of tin electrodes placed on the outside canthus of each eye. A second pair of tin electrodes placed above and below the left eye recorded vertical EOG (vEOG). The gain for both EOG channels was 20K. A third pair of tin electrodes was placed on the skin above the mastoid bone behind each ear. During recording, Cz was used as a common reference for all other scalp and mastoid sites. During subsequent data analysis, the EEG was rereferenced to the average of the left and right mastoid sites. The sampling rate throughout the experiment was 100 Hz. Stimulus materials The stimulus materials consisted of 360 words drawn from various sources (Battig & Montague, 1969, Rubin & Friendly, 1986) and experimenter-generated items. These words were evenly divided between location and object (people, animals, inanimate objects) words. We reduced original lists of locations (n = 283) and objects (n = 656) by eliminating rare (e.g., boomslang, oceanographer) or difficult to image (e.g., albatross, charlatan) items as well as obvious synonyms (e.g., physician, doctor; ocean, sea) or category-exemplars (sheep – lamb; spider tarantula). The resulting lists were submitted to the MRC Linguistic database ( http://www.psy.uwa.edu.au/MRCDataBase/uwa_mrc.htm ) to obtain normative data on written word frequency, imageability, meaningfulness, and concreteness. Outliers (+ 2 s.d.’s from ) were excluded. The final list of 160 pairs consisted of various objects, including people/occupations (n = 41), animals (n = 47), tools (n = 14), vehicles (n = 14), toys (n = 10), weapons (n = 13), musical instruments (n = 8) and furniture (n = 13). These people, animals and inanimate objects were paired randomly with locations and were manually examined to eliminate pairings with obvious pre-experimental associations

PAGE 43

34 (bartender – bar; clown – circus). Once was the list of 160 pairs was generated, one-half of the pairs were re-sorted to produce a test list consisting of 50% intact and 50% rearranged pairs. In the test list, the intact and rearranged pairs consisted of approximately the same number of object types described above. Design The design for the study phase was a single factor (Encoding Task: Bound versus Separate encoding of the words in a pair) between-subjects design. During the test phase, all participants were given the paired recognition test. Procedure After giving informed consent to the procedure, participants were fitted with the electro-cap and other electrodes. Generally, impedances, measured against Cz, were kept under 8K Ohms. Once participants were prepared for EEG recording, the experimenter gave an overview of the experimental procedure (i.e., “You’ll be viewing words presented on the screen and generating mental images of each of the words. You’ll rate the ease with which you generated the image. Following the image generation task, you’ll be given a memory test for the words.”). Following this instruction, participants read, on the screen, a more detailed set of instructions regarding the study phase. In brief, all participants were instructed that they would view pairs of words, each consisting of, first, an object (person, animal or object) and, second, a location. Participants were instructed to generate and maintain a “rich, vivid” visual image of the word’s representation upon its presentation. Participants were instructed to rate, following the location word, the ease with which they generated the image(s). For this purpose, participants were shown, on the screen, four clickable radio buttons captioned with a rating scale (Really easy, Somewhat easy, Somewhat difficult, Really difficult).

PAGE 44

35 Participants were instructed to make this judgment relatively automatically, giving their “first impression.” Instructions between the two (Bound/Separate) groups differed only regarding the generation of the image(s ). Participants assigned to the Bound group were instructed to generate a mental image in which the first word (object/person/animal) and the second word (location) were integrated into a single image or scene. They were instructed to make this "scene" as visually rich and vivid as they could. Participants assigned to the Separate group were instructed to maintain the image of the object and the location separately. Specifically, it was suggested that participants “place the image of the [first word] on the far left side of your “imaginary visual field and the image of the [second word] on the far right side of the imaginary visual field.” Virtually all participants expressed comprehension of this instruction. The experimenter eliminated any confusion with further explanation. As displayed in figure 2-1, the study phase, and each trial, commenced with a fixation cross, displayed for 300 ms, followed by a 700 ms post-fixation interval during which the screen was blank. Following the post-fixation interval, participants viewed words, presented singly in 28-point Arial font, each displayed for 500 ms with a 2500 ms interstimulus interval (ISI). EEG recording began 100 ms following offset of the fixation cross, and hence 600 ms prior to onset of the first word of each pair, and continued through 2600 ms after onset of the second word. Following the second word ISI, the ratings buttons were displayed until the participant selected one. The intertrial interval (ITI) between this mouse press and initiation of the next trial was fixed at 1000 ms. Following the presentation of each forty consecutive trials, the program paused for a

PAGE 45

36 participant-paced rest. Most participants, however, continued the experiment without a significant rest period. At the end of 160 trials, the program stopped. Figure 2-1. Schematic representation of a single trial during the study phase in Experiment 1. is fixation cross, “O” and “L” are presentation of object and location words, respectively. “//” is the self-paced, response interval during which participants indicated the ease of image generation. “ITI” indicates intertrial interval. Light colored line is EEG recording interval. The experimenter then engaged the participant briefly in unrelated conversation to prevent overt rehearsal of the last few presented items and to give the participant a brief break (~ 5 minutes) from the task. Thus, the mean latency from a pair’s appearance in the study phase to its appearance in the test phase was approximately 40 minutes. The timing and appearance of stimulus items in the test phase was the same as in the study; viz, a fixation cross, an object word, and then a location word were presented. However, the interstimulus interval between object and location words was reduced to 1500 ms and only a single break (rather than the three in study phase) was provided. Participants in both conditions (Bound/Separate) were instructed to indicate, using the left mouse button for affirmative responses and the right mouse button for negative responses, whether the OBJECT-LOCATION pair had been shown earlier (yes – left mouse button) or whether the pair consisted of an object and location that had been paired earlier with other items (no – right mouse button). Participants were instructed to make these responses as quickly as possible due to the measurement of response times. Following the old-new mouse presses, the monitor displayed a three-choice alternative O *2500 ms 700 ms 300 ms 500 ms 2500 ms L500 ms 1000 ms ITI

PAGE 46

37 (“The second word presented,” “Another word not presented,” “No other word”) to which participants were instructed to respond. The participants were told to respond according to their reaction to the first (object) word of the trial. That is, if presentation of the first word immediately elicited a word, participants were instructed to click on one of the first two choices (depending on the second word that was presented). On the other hand, if the presentation of the first word failed to elicit another word, participants were instructed to select “No other word.” Results Behavioral Data As expected, participants in the Bound condition were better at discriminating intact pairs from rearranged pairs (hits: M = 59.1, SE = 1.66; false alarms: M = 9.0, SE = 1.33) than participants in the Separate condition (hits : = 46.5, SE = 1.87; false alarms: M = 26.7, SE = 2.54) during the test phase, t (37) = 11.50, p < .001. Performance differences between the two groups were not attributable to speed-accuracy tradeoff; the groups did not differ in their correct response times to intact pairs (Bound: M = 1341 ms, SE = 69; Separate: M = 1337 ms, SE = 64, p >.10). These findings, coupled with participants’ post-hoc comments to the experimenter, suggest that they were, at least overall, generating and maintaining integrated or separate images in the two conditions as instructed. EEG Data Preprocessing Prior to averaging, the raw EEG data were inspected manually for the presence of blinks and other artifacts on a trial-by-trial basis. In the next phase, EEG for each trial was digitally low-pass filtered at 30 Hz for smoothing, and the mean amplitude set to zero for that trial to correct for baseline shifts. During this phase, trials marked as

PAGE 47

38 containing artifacts were subjected to a componential analysis and reconstruction process to attempt to remove blink and other artifacts from the waveforms, using the Independent Component Analysis (ICA) procedures and routines from the EEGLAB toolbox ( Delorme & Makeig, 2004 ), and a locally written Matlab script. Typically, one or two components were clearly identifiable with blinks/artifacts, and successfully removed. A maximum of four components (out of 16, limited by the number of recording sites) were allowed to be removed before rejecting the trial as unusable. ERP waveforms ERPs elicited by word pairs in the study phase were, as noted, computed on the basis of participants’ responses on the subsequent paired recognition test. Data from the study phase were sorted according to the following test phase responses: “old” responses to intact pairs were classified as IC (Intact Correct); “new” responses to intact pairs were classified as IE (Intact Error); “old” responses to rearranged pairs were classified as RE (Rearranged Error); and “new” responses to rearranged pairs were classified as RC (Rearranged Correct)). As noted above, a large number of participants’ data were excluded from analysis to be presented here. Thirty-nine participants (19 Bound/20 Separate) provided data for the analysis described below. Each participant’s averaged data were then averaged with other participants’ averaged data to calculate grand averaged data for each class. Baseline adjustment (setting the mean amplitude during a prestimulus interval for each condition equal to zero) was not performed, since it was possible that important ERP differences between various classes, including between pairs that were subsequently recognized and those that were subsequently forgotten, might be reflected in the prestimulus interval.

PAGE 48

39 ERPs to the Bound condition from the 13 scalp electrode sites are presented in Figure 2-2 below. The black line indicates ERPs for subsequently recognized intact pairs (IC), and the grey line indicates ERPs for intact pairs erroneously identified as re-paired (IE). Over the course of the 6200 ms interval, early event-related potentials (N100, P200, N400) to each word are clearly visible across most channels, followed by a broader later positivity around 600 ms, and a slow wave whose direction and magnitude differed widely across channel, and sometimes continues through the end of the epoch for each word. Differences as a function of subsequent memory performance can be seen later in the interval during the slow wave epoch, which appear greatest bilaterally at the frontal electrodes. For example, at the frontopolar (Fp) electrode sites, a sizeable difference between IC and IE traces is noted beginning at about 1600 ms from the beginning of the interval (1000 ms post-first word onset). Interestingly, in this case, correctly recognized pairs show a greater negativity during the interval than do subsequently forgotten pairs. The difference lasts until about 4000 ms when a significant reversal is noted, with IE becoming more negative than IC. A small difference between IC and IE is also visible during the earliest part of the interval (prestimulus through word one presentation), particularly at the frontal electrodes. The waveforms to the Separate condition (Figure 2-3) show marked contrasts to those from the Bound condition. There is little visible difference between the IC and IE waveforms. The large, slow wave differences between IC and IE that are present beginning in the 1000 ms range in the Bound condition are absent in the Separate condition. As with the Bound condition, however, activity at the frontopolar sites is

PAGE 49

40 distinguished, for both IC and IE responses, from the activity at all other locations by a positive, slow change beginning about 1600 ms from the beginning of the interval. Other locations are characterized either by a negative change during the interval, or by no change. Additionally, the Separate waveforms are distinguished from the Bound waveforms, especially at frontopolar sites, by the presence of two distinct positive peaks following the presentation of each word. The first of these peaks would appear to be a P200 to the onset of the words. The second positive peak is close to 200 ms after the offset of the word (after 500 ms) and may well be an offset response to the offset of the stimulus (see, e.g., Janata, 2001). Finally, a comparison between the IC responses to the Bound and Separate conditions is presented in Figure 2-4. Waveform differences between the IC responses that are evoked by the different task demands are apparent, if small. The second half of the last interval (~5500 ms) appears to show differences between the two correct responses in the right hemisphere. In addition, FT7 appears to reflect a difference between the IC responses that mimics, temporally, the differences observed at the frontopolar sites between IC and IE responses in the Bound condition. In addition, early, prestimulus differences between Bound and Separate IC responses are similar to those accompanying IC-IE responses in the Bound condition. Statistical analysis of waveforms Visual inspection of the grand averaged waveforms led us to identify windows of interest for subsequent statistical analysis. Subsequent memory effects were quantified by measuring mean amplitudes during each of ten successive latency intervals relative to onset of each word ([w1] –600 0, 0 300, 300 600, 600 1200, 1200 2600; [w2] -400

PAGE 50

41 Figure 2-2. ERPs to two words during study phase later shown as intact pairs during test in Experiment 1, Bound Encoding group. Bars indicate onset and offset times of the words. Black waveform is for pairs later correctly recognized as intact; grey waveform is for pairs later misrecognized as repaired. FT7 Fp1 Fp2 F3 F4 Cz FT8 TP7 TP8 P3 P4 O2 O1 010002000 -1 1 VTime since onset of first word (ms) 010002000 Time since onset of second word (ms)

PAGE 51

42 Figure 2-3. ERPs to two words during study phase later shown as intact pairs during test in Experiment 1, Separate Encoding group. Bars indicate onset and offset times of the words. Black waveform is for pairs later correctly recognized as intact; grey waveform is for pairs later misrecognized as repaired. FT7 Fp1 Fp2 F3 F4 Cz FT8 TP7 TP8 P3 P4 O2 O1 010002000 -1 1 VTime since onset of first word (ms) 010002000 Time since onset of second word (ms)

PAGE 52

43 Figure 2-4. ERPs to two words during study phase later shown and correctly recognized as intact pairs during test in Experiment 1. Bars indicate onset and offset times of the words. Black waveform is for the Bound Encoding group; Grey waveform is for the Separate Encoding group. FT7 Fp1 Fp2 F3 F4 Cz FT8 TP7 TP8 P3 P4 O2 O1 010002000 -1 1 VTime since onset of first word (ms) 010002000 Time since onset of second word (ms)

PAGE 53

44 0, 0 300, 300 600, 600 1200, 1200 2600 ms). Initial analyses were conducted by running, for each condition (Bound and Separate), an analysis of variance (ANOVA) on the mean interval amplitudes of ten “windows” that comprised the total 6200 ms trial interval to test whether they differed across the selected factors. In addition to the subsequent memory factor, two regional EEG factors were created from the 12 lateralized sites, excluding Cz. The ANOVAs thus utilized a 3-factor (Performance: Hit/Miss; Hemisphere: Left/Right; Anterior-Posterior (“AntPos”): 6 levels of electrode site), 2 x 2 x 6, within-subjects design. In addition, a third set of ANOVAs was run to compare the mean amplitudes produced by correct responses to intact pairs between the Bound and Separate conditions in each window. Thus, this ANOVA was a 3 factor (Task: Bound/Separate; Hemisphere: Left/Right; Anterior-Posterior: 6 levels of electrode site), 2 x 2 x 6, mixed design. In all analyses, the Greenhouse-Geisser correction ( ) was applied for violations of the assumptions of sphericity for comparisons involving two or more degrees of freedom. Bound condition As presented in Table 2-1, in the Bound condition, there were two intervals that yielded significant interactions involving subsequent memory. In the long interval (1200 – 2600 ms) between presentation of the first and second word there was a significant interaction between Performance and the AntPos factor, F (5, 90) = 3.68, p = 0.013, = .696, reflecting the larger subsequent memory effects across the frontal electrodes than at the more posterior sites. A second ANOVA which analyzed only the differences between the right and left Fp electrodes (excluding the other electrode sites) revealed no differences between the hemispheres (p > .10). Differences similar to those found to the first word were observed, in the form of a significant Perf x AntPos interaction, F (5,90) = 5.671, p =

PAGE 54

45 0.005, = .452, in the comparable interval (1200 2600 ms) to the second word. This interaction reflects, again, the larger Dm in the frontal sites. However, unlike in the Dm to the first word, the amplitudes of IC items are more positive than of IE items. Finally, there was a main effect of Performance Table 2-1. Time Intervals in Experiment 1 during which Amplitude Differences were Significant Factor Interval Perf Perf x AntPos Perf x Hem [w1]-600 – 0 S 0 – 300 300 600 H 600 1200 1200 2600 B [w2]-400 – 0 S 0 – 300 300 – 600 B 600 1200 1200 2600 B Note. “B” = significant Dm effects in Bound task, “S” significant Dm effects in Separate task, “H” = significant differences in Bound – Separate correct recognition (Hit) comparison. For all comparisons, = .05. in one of the early intervals (300 600 ms) to the second word in which the IE items were more positive than the IC items. Thus, quantification of the mean amplitudes through the various time windows that make up a single trial revealed subsequent memory effects that were larger toward the frontal part of the scalp than toward more posterior regions. Moreover, these effects appeared at approximately the same latency following the onset of each word. Finally, a more generalized Dm was observed early after the onset of the second word.

PAGE 55

46 Separate condition The Separate condition yielded significant Dm effects at two intervals, neither of which overlapped with the effects observed in the Bound condition (Table 2-1). A significant Perf x Hemisphere interaction was observed in the prestimulus interval (0 600 ms), F (1, 1) = 6.162, p = 0.023, = 1.0, reflecting the greater right-sided Dm in the prestimulus interval. A second Dm effect was observed in the -400 – 0 ms interval, just prior to onset of the second word. This effect did not interact with either of the other factors, F (1, 19) = 4.862, p = 0.040, = 1.0. Bound vs. separate conditions The final analysis run in the first experiment was to compare the IC responses produced in the Bound and Separate conditions. A significant interaction between the Task factor and AntPos occurred in the first word interval at 300 – 600 ms [F (5,90) = 4.332, p = 0.021]. This interval captures the temporal window during which the N400, and P300 components typically are observed. There is little consistency reflected in this particular interaction; Differences are noted between frontal and posterior sites and the effects of task are opposite to one another between the two. Discussion Following Bower (1970), participants showed impaired recognition performance after encoding items in the Separate, compared to the Bound, task instructions. This shows that even with the co-presentation of a given pair of words, and their high imageability, participants in the Separate task were, to a great degree, capable of keeping the two items separate and distinct, as instructed. Moreover, when queried, all participants in the Bound condition reported being able to “bind” the object and location into a single image. Likewise, all participants in the Separate condition reported being

PAGE 56

47 able to generate and maintain object and location images separately. Occasional participants in the Separate condition reported that “on a couple of trials” they “couldn’t help putting (binding) the images together.” These reports were sporadic and no participants reported this “problem” to have occurred on more than 2 or 3 of the 160 study pairs. Thus, the instructions to generate and integrate and object and location in the Bound Condition (and generate and maintain unique images in the Separate condition) can be assumed to have acted as intended. As expected, we obtained reliable subsequent associative memory effects using a task that places demands on associative encoding. Participants were instructed, in both the Bound and Separate conditions, to generate mental images that were as clear and visually rich as possible. We anticipated that the instructions in both conditions would promote extensive cognitive effort by participants that would yield observable differences between later forgotten and later remembered pairs. We hypothesized that successful pair recognition would depend on (a) adequate processing of each element (object and location) of the pair, and (b) the creation and maintenance of a link between the two. Thus, the strongest prediction that we made was that the Bound condition would reveal transient frontal Dm effects that would appear in response to each word, as well as a later-appearing Dm in response to the integrative demands of the task. On the other hand, we predicted that the Dm effects for the Separate condition would follow from the lack of integrative instructions in the task. We expected any subsequent memory effects to reflect the establishment of strong memory traces for the objects and locations individually and there should be no late, frontal, second-word-only effects associated in the Bound condition with integrative activity.

PAGE 57

48 The Bound condition ERPs revealed a striking pair of subsequent memory effects that occurred with the same latency following onset of the first word and the second word (1200 – 2600 ms) and the same largely symmetric frontal topography. Notably, however, the relative polarity of the difference was the opposite between the first and second word late Dm effects. That is, the Dm to the first word was negative (Bound < Separate) and the Dm to the second word was positive (Bound > Separate). Although the negative Dm has been identified in only one other study (Dm for name recognition Guo, 2005), we can speculate, in the present case, why it may have occurred. Our particular paradigm has some similarities to a CNV-producing S1-S2 (Go No-Go) paradigm. Therefore, as noted, we expected to find a “slow wave” component, analogous but not identical to the CNV, between the first and second word. We assumed that this slow wave might show significant amplitude shift. In the current study, we noted that this slow wave activity appears over several electrode sites. We pointed out, however, that the shift was in a positive direction only at the Fp sites. If the positive change at the Fp sites reflects a preparatory, maintenance-like state (similar to the E-wave of the CNV), then the greater negativity of the hits (relative to the misses) at the Fp sites might reflect ongoing processing of the first word and/or preparation for the second word that underlies successful binding. Conversely, the greater positivity of the hits in the late interval following the second word is more typical of a Dm effect and, perhaps more expected, in light of the fact that the final word of the pair has been presented and the participant need anticipate no further events in the trial. Thus, the later (1200 – 2600 ms) second-word Dm could reflect both the processing of the second (location) word and the establishment of a successfully integrated trace.

PAGE 58

49 We also noted the presence of a third, widespread Dm in the Bound condition, occurring just after the presentation of the second word (at 300 ms) but not after the first word, which suggests that the cognitive processing associated with the late (1200 ms) second-word Dm may not have reflected identical processing to that during the earlier (1200 ms), first-word Dm. This difference is reflected in a greater positivity for subsequently missed pairs than for subsequently recognized pairs. It could be that the early, second word Dm reflects the initiation of cognitive mechanisms to respond to the integrative task demands, or the assessment of the presented location as appropriate for integration. Although it is not entirely clear, it appears that the differences at this interval are from a reduction in the N400, an enhancement of the P300, or some combination of the two in the later-missed items. Unexpectedly, however, the Separate condition also revealed a temporally similar (-400 ms) widespread second-word Dm effect. The timing is especially surprising. Two possibilities seem plausible. One possibility is that, despite instructions to keep the images of location and object separate, participants were, in fact, integrating the two. Thus, the Dm at the end of the first word interval could reflect preparatory activity for binding the just-presented object word to the to-be-presented location word. This conclusion, however, seems unwarranted for at least two reasons. First, participants in the Separate condition reported that they were successful in generating and maintaining separate images for the object-location pair. The significantly poorer recognition rates for the Separate condition further supports their contention. Second, there is no ERP evidence, in the form of a Dm effect, that participants are doing anything during the second word that distinguishes remembered from forgotten pairs. An alternative, more

PAGE 59

50 plausible, explanation is that the late, first-word Dm in the Separate condition represents further processing of the object word or a preparatory attentional shift or disengagement from the first word in anticipation of the second word. This attentional shift effect would be expected to be present in the Separate condition, if an adequate trace was established to the first word, but not in the Bound condition where the object and location are required to be integrated. In any event, the pattern of Dm effects shows differences in timing and topography between the two experimental conditions, reflecting the likely engagement of a different set of neural and cognitive processes that yield success or failure in each condition. Finally, the lack of differences between the IC responses in the Bound and Separate conditions, except along a brief, early 300 ms interval to the first word, is somewhat unexpected. Our strongest prediction was that the integrative activity, present in the Bound but not the Separate task, would have discernable effects on the scalp related ERPs. Given the temporal and topographical differences between the Dms in the Bound and Separate conditions, we expected that the correctly recognized, intact pairs would likewise show differences between the Bound and Separate conditions. The lack of differences between the groups may be attributable togreatervariability between the groups than the within-group variability in the Dm comparisons. Nevertheless, the pattern of subsequent memory differences associated with paired recognition of objects and locations provides important evidence that areas of the prefrontal cortex have an important role in establishing the relationship between the items. This electrophysiological response occurs following both the first and second words and is consistent with a variety of accounts (e.g., Craik, 1989; Hunt & Einstein,

PAGE 60

51 1981) of memory encoding suggesting that relational processing has a separate cognitive basis from item processing. In the second experiment, we used the same materials and instructions but gave participants an item recognition test to examine the event-related potentials associated with subsequent memory success. We anticipated that, if the pattern of ERP Dms we identified in the first experiment were associated with relational encoding, a different pattern of Dm effects would be present for the item recognition test.

PAGE 61

52 CHAPTER 3 EXPERIMENT TWO The first experiment demonstrated some physiological evidence of the cognitive underpinnings of episodic encoding. However, by its design, the experiment left unclear whether the subsequent memory effects that were identified were those capable of supporting single item recognition, associative recognition alone, or both item and associative recognition. The second experiment attempted to isolate the processes that support subsequent item recognition and contrast these from processes that support associative recognition. We used the same materials and task at study as in Experiment 1. However, the subsequent recognition test was for individual words presented as objects or as locations during the study phase. Memory for objects and locations were tested separately to enable the creation of ERP records that could be conditionalized on either object recognition or location recognition. Expected Results We anticipated that the Bound versus Separate manipulation would have little impact on overall performance on the memory test. That is, item recognition would be relatively unaffected by whether participants attempted to integrate objects and locations at study. Given that participants in both conditions utilized similar semantic encoding strategies and were both instructed to make their visual images “as rich and vivid as possible,” we presumed that there would be no difference in item recognition levels. We expected that the ERPs in the Bound condition, conditionalized on subsequent item recognition, would be associated with the processing of individual items, not with

PAGE 62

53 the integration of objects and locations. Consequently, we anticipated little difference between the Bound and Separate Dm effects. Moreover, our design, which tested object and location recognition separately, enabled us to isolate ERP correlates of subsequent performance associated with each item of the studied pair. Thus, we anticipated, for example, that an object recognition test would yield subsequent memory effects associated with the prior presentation of objects, but not locations. If, as we posited, the Dms in Experiment 1 were associated with relational encoding, the Dms isolated in the current experiment should differ from those in Experiment 1 by timing, topography, or both. Given the inconsistent findings as to topography of Dm effects in item recognition, we make no specific predictions about the likely topography of Dm effects in the current experiment. However, the findings from Experiment 1 and the existing Dm literature provide some guidance as to the anticipated timing of Dm effects. Given that Dm effects putatively associated with relational encoding occurred in the long (1200 – 2600 ms post word 1/post word 2) intervals in Experiment 1 we anticipated that itemspecific Dm would be associated with earlier intervals and show greater transience. As in Experiment 1, we expected the difference between the task-specific ERPs (Bound correct responses vs. Separate correct responses) to be minimized by the increased variability associated with the between-subjects nature of the comparison. Any differences should be associated with inter-task “set” differences. Thus, early, prestimulus differences could be reflected in the task comparisons.

PAGE 63

54 Method Participants Fifty-one undergraduates (32 females) at the University of Florida participated in this experiment. Additionally, 7 undergraduates participated as pilot participants. Participants were randomly assigned to the two experimental groups. Participants received credit toward an introductory psychology course requirement or a nominal payment. Of the 51 participants who began the experiment, one failed to return for the test phase so neither behavioral nor EEG data were available for that participant. Two other participants had too few (< 10) incorrect responses to the memory test portion to permit construction of reliable waveforms. Finally, of the remaining 48 participants, various technical and signal-to-noise problems prevented analysis of another 13 EEG datasets. Materials and Apparatus Stimulus display and response recording The study phase portion of the experiment was conducted under the same conditions and in the same location as the first experiment. The recognition phase (during which EEG was not recorded), which was held about 24 hours after completion of the study phase, was held in a brightly lit room not used for EEG recording. The recognition phase of Experiment Two was delayed after pilot testing indicated that there would be too few misses for item recognition to obtain interpretable ERPs in that condition. The same computer program used to display the material in Experiment 1 was used to display material in Experiment 2. EEG recording EEG recording was accomplished using the same parameters as in Experiment 1.

PAGE 64

55 Stimulus materials The study phase list from the first experiment was used to create a pair of study phase lists for the current experiment. The 160 object-location pairs were divided into 2 80-pair lists, each list serving, in one case, as study phase items and test phase foils, and, in the other case, as test phase foils and study phase items, respectively. All of the objectlocation pairs were separated to create two pairs of test phase lists (Objects/Locations) with each participant being shown an object list and its corresponding location list. Design The design for the study phase was a single factor (Bound, Separate) betweenparticipants design. During the test phase, all participants were given item recognition tasks in which items were presented at the same rate as during Experiment 1. However, participants were required to respond “Old” or “New” (with same mouse press arrangement in Experiment 1) to each item, rather than following each pair. In addition, confidence ratings were obtained following each “Old/New” response. At the completion of the first 160 object or location recognition test, participants took the remaining (object or location) recognition test. Upon concluding the recognition test, participants completed a 32-item questionnaire (VVIQ-R; McKelvie, 2001) on the vividness of their visual imagery experiences. Procedure The study phase procedure was as described for Experiment 1 except that the participants were shown only one half of the object-location pairs. The remaining 80 pairs served as foils in the item recognition tests. Participants were reminded that they would return to the lab approximately 24 hours after completing the study phase to take a memory test and complete a questionnaire on mental imagery.

PAGE 65

56 As in Experiment 1, the study phase, and each trial, commenced with a fixation cross, displayed for 300 ms, followed by a 700 ms post-fixation interval during which the screen was blank. Following the post-fixation interval, participants viewed words, presented singly in 28-point Arial font, each displayed for 500 ms with a 2500 ms interstimulus interval (ISI). Following the second word ISI, the ratings buttons were displayed until the participant selected one. At the end of 80 trials, the program stopped. The experimenter then disconnected the participant from the EEG equipment, confirmed the following day’s appointment, and dismissed the participant. On the following day, the timing and appearance of stimulus items in the test phase was similar their presentation in the study phase; viz, a fixation cross, and an object, or a location, word were presented. However, the interstimulus interval between object and location words was reduced to 1500 ms and participants were shown either a list or 160 object words followed by a list of 160 location words or vice versa. List order, task and stimulus set was counterbalanced between subjects. Participants in both conditions (Bound/Separate) were instructed to indicate, using the left mouse button for affirmative responses and the right mouse button for negative responses, whether the word had been shown earlier (yes – left mouse button) or whether the word consisted of an object or location that had not been shown earlier (no – right mouse button). Participants were instructed to make these responses as quickly as possible due to the measurement of response times. Following the old-new mouse presses, the monitor displayed a three-choice alternative confidence rating (“Very confident,” “Somewhat confident,” “Just guessing”) to which participants were instructed to respond. Participants

PAGE 66

57 were instructed to indicate the confidence with which they made their previous “old-new” responses. Results Behavioral Data As displayed in Figure 1, in contrast to Experiment 1, in which large effects of the Task manipulation were observed, whether participants processed pairs under Bound (hits: M = 57.3, SE = 3.21; false alarms: M = 23.7, SE = 4.25) or Separate (hits: M = 55.9, SE = 2.74; false alarms: M = 21.3, SE = 3.45) imagery instructions had no impact on the probability of subsequent recognition of either objects or locations. On the other hand there was a large test effect. That is, collapsed across task, location words (hits: M = 51.7, SE = 3.26; false alarms: M =25.8, SE = 3.78) were less well remembered than object words (hits: M = 61.5, SE = 2.69; false alarms: M = 19.2, SE = 3.93). Item Recognition0 10 20 30 40 50 60 LOCATIONSOBJECTS Item TypeHits FAs (80 max) Separate Bound Figure 3-1. Behavioral performance in Experiment 2 (hits – false alarms) compared between the two encoding groups (Bound vs. Separate) and test type (Location vs. Object).

PAGE 67

58 Furthermore, there was no impact on overall item recognition of whether participants were first given the object word or location word test. Likewise, as expected, there was no difference in recognition performance for the two stimulus sets. EEG Data ERP waveforms ERPs elicited by word pairs in the study phase were, as noted, computed on the basis of participants’ responses on the subsequent object recognition test. Data from the study phase were sorted as “hits or misses.” As noted above, a large number of participants’ data were excluded from analysis. Thirty-five participants (18 Bound/17 Separate) provided data for the analyses described below. Each participant’s averaged data were then averaged with other participants’ averaged data to calculate grand averaged data for each class. ERPs to the Bound condition from the 13 scalp electrode sites are presented in Figure 3-2 below. As in the first experiment, over the course of the 6200 ms interval, discernible evoked responses to the onset of the first and second words (N100, P200, N400) are apparent at most sites across both conditions. Importantly, there are strong similarities between the waveforms generated by the participants in the Bound condition in Experiment 1 and those in the Bound condition in Experiment 2. These similarities are most apparent in the positive slow wave from 1200 ms to 3000 ms in the frontopolar sites, as well as a corresponding negative slow wave during the same interval at the Cz electrode. Thus, and to maintain consistent analysis across the two experiments, the same time windows were used for analysis in the second experiment as in the first experiment. In the right hemisphere, the waveforms for hits and misses are nearly indistinguishable. Small differences, with remembered items being more positive than

PAGE 68

59 forgotten items, appear in the prestimulus interval at FT7, as well as immediately preceding the onset of the second word at frontal sites (Fp1, Fp2, F3). Other differences appear in the region of the N400 to the second word at FT7. One difference that is similar to the Dm effects observed in Experiment 1 is found at Fp2 prior to the onset of the second word. In this case, forgotten items are more positive than remembered items. There are also late differences at F3 and FT7 with subsequently recognized items being more positive than forgotten items. ERPs to the Separate condition from the 13 scalp electrode sites are presented in Figure 3-3 below. The waveforms again show marked deformations at standard component latencies (N100, P200, N400). As in the Bound condition, there is a notable positive-going slow wave between the first and second words at the frontal polar locations. At other locations, this time frame is either characterized by negative going activity or by little change in the overall polarity of the waveform. Unlike in the Bound condition, there is little difference in the waveforms between those to items later recognized and those subsequently missed, although some separation between hits and misses is noted beginning about 900 ms after the onset of the first word at Fp1. The correct responses to old items in the Bound and Separate conditions are compared in Figure 3-4. Although collected from two different groups of participants under two different task instructions, the waveforms track each other closely, especially at posterior electrode sites. There appear to be differences, however, between the Separate and Bound groups, and these differences seem to be larger in the left hemisphere than in the right hemisphere and more pronounced at the anterior, than at the posterior, electrode sites.

PAGE 69

60 Statistical analysis of waveforms As in the first experiment, mean differences in EEG amplitudes during the study phase were conducted by running, for each condition (Bound and Separate), an analysis of variance (ANOVA) on the ten “windows” identified in Experiment 1 that comprised the total 6200 ms trial interval. The ANOVAs tested the same 3 factors (Performance: Hit/Miss; Hemisphere: Left/Right; Anterior-Posterior (“AntPos”): as in Experiment 1. In addition, a third set of ANOVAs was run to compare the mean amplitudes produced by correct responses to old items between the Bound and Separate conditions in each window. Thus, this ANOVA was a 3 factor (Condition: Bound/Separate; Hemisphere: Left/Right; Anterior-Posterior: 6 levels of electrode site), 2 x 2 x 6, mixed design. The Greenhouse-Geisser correction ( ) was applied for violations of the assumptions of sphericity for comparisons involving two or more degrees of freedom. Bound condition Although there were no significant main effects of Performance or interactions between Performance and either AntPos or Hemisphere across any of the intervals, marginal effects were observed in the interval immediately preceding the onset of the second word (-400 – 0 ms) (Perf x Hemisphere: F (1,17) = 4.349, p =. 052, = 1.00 and early in the second word interval (0 300 ms), Perf: F (1,17) = 4.326, p = .053, = 1.00. Neither the Perf x. AntPos, nor Perf x. AntPos, x Hemisphere interactions were significant (all p ’s > .05). Separate condition The Separate condition showed no effects related to task performance (Perf, Perf x Hemisphere, Perf x AntPos, Perf x Hemisphere x AntPos: all p ’s > .10).

PAGE 70

61 Figure 3-2. ERPs to two words during study phase during test in Experiment 2, Bound Encoding group. Bars indicate onset and offset times of the words. Black waveform is for first words (items: people, animals, objects) later correctly recognized as studied; grey waveform is for words later missed. FT7 Fp1 Fp2 F3 F4 Cz FT8 TP7 TP8 P3 P4 O2 O1 010002000 -1 1 VTime since onset of first word (ms) 010002000 Time since onset of second word (ms)

PAGE 71

62 Figure 3-3. ERPs to two words during study phase during test in Experiment 2, Separate Encoding group. Bars indicate onset and offset times of the words. Black waveform is for first words (actors and objects) later correctly recognized as studied; grey waveform is for words later missed. FT7 Fp1 Fp2 F3 F4 Cz FT8 TP7 TP8 P3 P4 O2 O1 010002000 -1 1 VTime since onset of first word (ms) 010002000 Time since onset of second word (ms)

PAGE 72

63 Figure 3-4. ERPs to two words during study phase later shown and correctly recognized as intact pairs during test in Experiment 2. Bars indicate onset and offset times of the words. Black waveform is for the Bound Encoding group; Grey waveform is for the Separate Encoding group FT7 Fp1 Fp2 F3 F4 Cz FT8 TP7 TP8 P3 P4 O2 O1 010002000 -1 1 VTime since onset of first word (ms) 010002000 Time since onset of second word (ms)

PAGE 73

64 Bound vs. separate analysis The third analysis consisted of comparing the mean differences between study pairs that yielded subsequent correct recognition of objects, people or animals, at test, in each of the two conditions. Differences were identified near the onset of the first word (0 300 ms) (Task x Hemisphere: (F (1,33) = 7.164, p = .011, Task x Hemisphere x AntPos Table 3-1.Time Intervals in Experiment 2 during which Amplitude Differences were Significant Factor Interval Perf Perf x AntPos Perf x Hem Perf x Ant Pos x Hem [w1] -600 0 0 – 300 H H 300 600 600 1200 1200 -2600 [w2] -400 0 B 0 – 300 B H 300 600 600 1200 1200 – 2600 Note. “B” = marginally significant Dm effects in Bound task, “H” = significant differences in Bound – Separate correct recognition (Hit) comparison. There were no significant Dm effects in Separate task. Bold : .06 > p > .05. For all comparisons, = .05. F (2.271, 74.935) = 3.216, p = .040) and near the onset of the second word (0 -300) (Task x Hemisphere x AntPos: F (2.655, 87.615) = 3.043, p = .039).

PAGE 74

65 Discussion Behavioral measures of performance in the second experiment revealed that single item recognition, whether for the first or second word of a pair, is unaffected by whether those items are the subject of relational processing. The trace that is generated in the encoding phase by either relational or single-item processing is sufficient to produce comparable levels of single item recognition. Levels of item recognition varied, however, by the nature of the target. Location words were less well recognized than object words. Location words, however, were always presented following object words in the study phase so it cannot be determined whether the decrement in location word recognition is attributable solely to the type of stimulus, or whether order effects also contributed to their poorer recognition performance. It might be argued, however, that if order effects were responsible, in part, for the decrement in location recognition, it would suffer less in the Bound condition than in the Separate condition, by virtue of the order being less salient to the encoding. However, the Perf x Item interaction was non-significant, suggesting that its recognition decrement was due primarily to the difficulty in encoding the locations. The pattern of subsequent memory effects was different from, and less extensive than, that found in Experiment 1. Although none of the comparisons reached significance in the Bound condition, the marginally significant comparisons (Table 3-1) are discussed below. The separation of hits and misses in the end of the first word interval (reflected in a marginally significant Task x Hemisphere interaction (p = .053)), is characterized by left, but not right-sided amplitudes for the “miss” responses being larger than “hit” responses. In fact, visual inspection of the waveforms suggests that the differences are driven by a deflection of the “miss” responses. Whether this characterization is accurate

PAGE 75

66 is difficult to determine but it suggests that the separation between hits and misses in the Bound condition is associated with some processing, or failure to process, the first word late in the interval. This interpretation is further supported by the lack of differences in the correct recognition responses to old words presented in the Bound and Separate conditions at those corresponding intervals. Although there were suggestions in the waveforms of the Separate condition of subsequent memory differences, especially over the left hemisphere frontal electrodes at about 1200 ms and again at about 2000 ms at central locations, none of the hit-miss comparisons at any of the intervals reached significance. Although we predicted that the Separate (as well as the Bound) condition would yield subsequent memory effects for item recognition, at least to the first word, there is a possible explanation as to why no effects were observed. First, Dm effects have been shown to be extraordinarily sensitive to task demands. Thus, Paller et al. have shown that, under certain conditions, cued recall produces large Dm effects while item recognition does not. Similarly, recognition responses classified as “Remembered” (according to Tulving’s scheme) are more likely to produce Dm effects than “Know” responses. Thus, correct recognition responses to previously viewed items in the current experiment are likely to have included some proportion of guesses, or at the very least, trials on which the relational encoding failed (and thus would have been “Misses” in the first experiment). Analysis of the confidence ratings that participants gave during the recognition test and sorting of the study phase ERPs into more confident versus less confident responses is more likely to yield Dm effects.

PAGE 76

67 The nature of the item Dm effects observed in the Bound, but not the Separate, condition is open for speculation. Visual inspection of the waveforms suggests that the Bound and Separate hits closely resemble the misses in the Separate condition in the interval (-400 – 0 ms) during which subsequent memory differences were observed. As noted, the misses in the Bound condition show a significant positive deflection. It could be that some aspect of relational processing has a detrimental effect on item recognition. For example, it could be that, in the Bound, but not the Separate, condition, participants shifted their attention from the first word in preparation for the presentation and integration of the second word. If an incomplete trace of the first (object) word was established at the time of the shift, then the miss trials might be associated with effects not observed in either the hit trials in the Bound and Separate conditions or the miss trials in the Separate condition. The subsequent memory differences observed in the next window (0 – 300 ms), again for the Bound but not the Separate condition, may reflect the operation of similar cognitive processes that support (or impair) item recognition but have no impact on paired recognition. Thus, for example, a premature shift of attention to the second (location) word, in the Bound but not the Separate condition, could be reflected in the impaired recognition of the object but not the pair. While the nature of the Dm effects observed for the Bound, but not the Separate, condition can be speculated at, there were clear task related differences in the second experiment, the nature of which seem to be more apparent. Correctly recognized first words were associated with differences in ERPs for the Bound and the Separate conditions at two similar intervals over the course of the trial. That is, at the onset of each

PAGE 77

68 word (600 ms and 3600 ms), the differences between amplitude means for the Task x AntPos x Hem conditions were significant. These differences are likely to reflect strategy differences between the two tasks, given that an object to be integrated with a location may be processed differently than one that will face no such demands. Likewise, upon presentation of a location, there are demand differences for how that location will be processed in the Bound and Separate conditions. The second experiment revealed a unique pattern of subsequent memory effects associated with item recognition, differing from those identified in Experiment 1. In contrast to the Dms that accompany paired recognition, item recognition Dms were restricted to the first word. This makes perfect sense, since the Dms were conditionalized on recognition of the first word. Somewhat unexpectedly, however, the subsequent memory effects were present only in the Bound task, and the relative similarity of the waveforms between the Bound and Separate conditions suggests that those differences may reflect error-related processing in the Bound case that may have been related to the integrative task.

PAGE 78

69 CHAPTER 4 GENERAL DISCUSSION The cognitive and neural processes that underlie successful episodic memory encoding include the creation of a memory trace that encompasses both an item or event and its spatiotemporal context. Little is known, however, about the way in which an item and its context are linked at the time of their presentation. By carefully manipulating the encoding task and memory test type, and sorting, post-hoc, encoding trials by subsequent memory performance the two ERP experiments reported here represent a novel approach to examining the cognitive and neural correlates of episodic memory encoding. Using this approach, we identified what is, to our knowledge, a unique set of ERP subsequent memory effects. Most notably, these include a frontopolar, positive-going, slow-wave potential late after the presentation of the first word of a pair that is more negative for pairs later successfully recognized, following imagistic processing of concrete nouns in an integrative encoding task (Bound condition, Experiment 1). This effect makes clear that relational processing begins even prior to the onset of the second item (here, the spatiotemporal “context”) in a pair, and suggests that prefrontal areas play an important role in this processing. Distinctive Aspects of the Present Approach Many ERP studies of episodic encoding compare the neural activity and behavioral performance associated with one type of task or process with that in a second task or process. Some other ERP studies sort, on a post-hoc basis, encoding trials by subsequent memory performance to compare the neural responses during trials associated

PAGE 79

70 with later successful memory performance with those associated with later unsuccessful memory performance. We have utilized both elements while maintaining tight control over the stimulus materials and tasks. By analyzing encoding ERPs according to subsequent memory performance, we have avoided encoding manipulations such as levels of processing as a proxy for memory performance. Such manipulations putatively generate better or worse memory performance but invariably include errors in the deep condition trials and correct responses in the shallow condition trials. Moreover, by not using such a manipulation, we were able to manipulate, systematically, an encoding strategy that addresses directly the question in which we were interested – are there discernable neural and cognitive processes associated with binding objects and locations in episodic memory? Thus, cognitive and neural processes associated with item-context binding that lead to successful episodic memory were isolated in a pair of carefully controlled experiments. In addition to using a unique paradigm, our experiments carefully controlled both the stimulus materials and task parameters to make comparisons between conditions and experiments valid. So, for example, while one group of participants in the first experiment generated and maintained isolated images of the items and locations, another group generated and maintained integrated images of the same item-location pairs following identical presentation parameters. Moreover, the two groups were tested with identical stimuli, using the same test methods and instructions. Likewise, the second experiment used the same stimulus materials, method and instruction as in the first experiment. The only difference between the two experiments was in the test phase. Moreover, we used unique

PAGE 80

71 item-context pairs throughout, thereby avoiding the stimulus repetition effects that make working memory and source memory paradigms difficult to implement in the study of long-term memory encoding. Although unique face-name (Guo et al., 2005) or faceoccupation (Yovel & Paller, 2004) pairs have been used to study relational encoding, given the controversy surrounding the special cognitive and neural mechanisms of face encoding and recognition, our paradigm is more generalizable than face encoding studies. Moreover, in contrast to the remaining associative encoding studies, our experiments specify the nature of the relational encoding to be performed. Thus, the nature of any processes associated with one task, but not the other, can be described more precisely. Finally, our experiments uniquely yielded the ability to contrast ERPs to the first and second stimulus item in a relational encoding paradigm. This feature enabled us to pose an as-yet unasked question: Are there cognitive and neural processes engaged by the presentation of the first item of a pair that are to be relationally encoded that are preparatory to the presentation of the second item? This question, which seems fundamental to notions of relational encoding, has not been addressed in any ERP study of which we are aware. In sum, no other subsequent memory ERP study has provided the degree of control, or the possibilities for isolating the constituent elements of relational encoding as the current pair of experiments. Memory-Related ERPs and Integrative Episodic Encoding The pair of experiments yielded clear evidence of task and test-dependent ERP effects that were associated with subsequent memory performance. As predicted, these effects differed in timing and topography that depended on both the encoding instructions and the retrieval demands imposed by the type of memory test given. Consistent with our predictions, when memory was queried by paired recognition, the subsequent memory

PAGE 81

72 effects showed significantly different patterns between the two tasks. As expected in the integrative encoding condition, these effects arose at frontal electrode sites; importantly, there were no hemispheric differences. Moreover, the Dm effects occurred in response to each word, suggesting that some kind of item-related processing contributes to relational encoding success. In addition, an unexpected, widespread effect was observed early in the second word interval. This effect may have signaled the allocation of cognitive resources in preparation for subsequent integrative activity. Alternatively, it may be a carry-over of the preparatory Dm observed late in the first-word interval (see Figure 2-1). In contrast to the pattern of activity observed in connection with the integrative instructions, Dm effects associated with the Separate instructions were restricted to (a) a prestimulus hemispheric difference, and (b) a transient, widespread effect immediately before the onset of the second word. The prestimulus Dm, which has not been reported before, may be an important indicator of attentional or other cognitive “set” differences that contributed to paired recognition success. Likewise, the widespread, transient effect just prior to onset of the second word may reflect the allocation of necessary attentional resources that separates later remembered from later-forgotten pairs. Although it is tempting to interpret the long-duration differences to each word in the Bound condition to item processing, their absence in the Separate condition suggests that those effects were not merely indicators of item-only processing. Rather, they likely reflect some degree of processing of the item as, to the first word, a to-be-integrated stimulus feature. The comparable second word Dm may be associated with the integration of the location with the object. Thus, contrary to our strongest predictions about the differences between the two conditions, successful encoding in the Bound task was not simply successful

PAGE 82

73 encoding in the Separate task with an integrative component added on. Inclusion of the integrative component changed the entire pattern of neurocognitive activity associated with subsequent correct paired recognition. Although memory was tested by paired recognition in Experiment 1, the patterns of results could have been due to the contributions of relational processing, item-only processing, or both. Thus, we conducted a second experiment using an item recognition test to discriminate between relational encoding processes that support pair recognition and item encoding processes that support simple recognition. We predicted that there would be little difference between the processes underlying item recognition whether or not a relational encoding strategy was used. Thus, we expected the patterns of subsequent memory differences between the Bound and the Separate conditions to be very similar when memory was tested by item recognition. As in the first experiment, there was no overlap between the Dm effects associated with the Bound encoding and Separate encoding strategies. In fact, there were no significant subsequent memory effects at all in the Separate condition, and only two intervals showed marginal Dm effects in the Bound condition. This finding is consistent with findings that recollection and recall tasks are more likely to produce Dm effects than item recognition and it suggests that the Bound Dm effects represent item-specific encoding processes, rather than relational encoding effects. The timing of the marginal Dm effects in the Bound condition is also consistent with our predictions. Encoding trials were classified on subsequent recognition of the item (person, animal, object) word, which was always presented as the first word of the pair during each study trial. Thus, Dm effects would be expected in response to the first, rather than the second, word interval. The marginally significant effects at the

PAGE 83

74 presentation of the second word may reflect spillover from the sustained processing of the first word at the end of the interval. Overall, then, the study produced, in each condition and experiment, a pattern of ERP differences that were, largely, consistent with our predictions. Comparisons to Previous Findings While there is a scarcity of findings regarding the ERP correlates of item-context encoding, Kounios and colleagues (2001) found that fusion association, in which two concepts are fused together to form a qualitatively distinguishable third concept (e.g., computer + virus = computer virus), has distinct neural correlates from juxtapositional association, in which two concepts are associated by contiguity. Fusion association was distinguishable by activity in right prefrontal cortex following the second word and waveform differences between quickly and slowly retrieved word pair orders at test at bilateral frontotemporal sites from 200 ms to 3000 ms after the onset of the second word. While theoretical and methodological differences between Kounios and colleaguesÂ’ work and the present study make direct comparisons difficult, it is worth noting that Kounios claimed that the difference in topography, timing and polarity between juxtapositional and fusion effects supported the idea that the two different cognitive processes are engaged by the different tasks. Likewise, in the current study, timing and topography differences between the effects found for encoding in the Bound condition and those found for Separate encoding, as classified by paired recognition performance support the idea that the two types of encoding recruit different cognitive processes. This claim is further buttressed by the finding that these effects differ from the Bound and Separate encoding effects that underlies item memory.

PAGE 84

75 Of the ERP findings in the study, perhaps none is more striking than the pair of Dm effects that occur in similar, long (1400 ms) intervals following the presentation of the first and second words in the Bound, but not Separate, encoding task when trials are classified by performance on the paired recognition test. These effects, which have a frontal topography, differ from other observed frontal effects in associative memory encoding in two ways. First, the effects in the interval following the first word ride on a positive-going slow wave beginning around 1400 ms after word onset. While we could not identify any studies other than Kounios and colleaguesÂ’ (2001) that use a sequential S1-S2 word presentation paradigm in recognition memory, the positive-going nature of the slow wave, in contrast to the reversal (negative-going slow wave) at more posterior electrode sites is consistent with sustained positivity at frontopolar sites in other studies (Duarte et al., 2004; Mangels et al., 2001). As noted above, if our task is analogous to an S1-S2 task that typically elicits a negative-going slow wave, it is not surprising, perhaps, that the Fp sites yield positive-going slow change that persists until the onset of the second word. While the first word interval is followed by a frontal, positive-going slow wave, the second word is followed by a widespread negativity (with a notable exception at FT8). The Dm in this interval consists of the more typically observed pattern; subsequently recognized pairs are of greater positivity than subsequently missed items. Second, the Dm for first words at frontopolar sites is of negative polarity (subsequent misses > subsequent hits). We have been able to identify only one other study (Guo et al., 2005) in which, at frontal sites, the amplitude of subsequently unrecognized items was more positive than that of subsequently recognized items.

PAGE 85

76 Although it is unclear to what the negative Dm in Guo and colleagues’ study can be attributed, it, too, was embedded in a sustained positive-going frontal wave (albeit only through the end of the 1s trial interval). Guo and colleagues’ study involved participants intentionally encoding concurrently presented visually presented faces and auditorily presented names. The fact that face recognition Dms were significant in the later part of the interval and the name recognition Dms were significant only in the early interval suggests that the name and face were processed sequentially. Thus, it is possible that the negative Dm effects Guo and colleagues observed for name recognition reflect completion of the name processing and preparation for face name binding or maintenance of the name during face processing. This explanation, of course, is speculative and warrants further investigation. The pattern of the Dms to the first and second word in the long interval is intriguing. Duarte and colleagues (2004), who found distinct subsequent memory effects for pictures subsequently classified as “remembered” or subsequently classified as “known” versus those that were missed, concluded that the sustained bilateral frontal activity associated with “remember” responses were attributable to “more extensive processing” than those later classified as “known.” It could be that, in the Bound condition, participants were mentally “manipulating” or refining their images of the first word object in preparation for the required upcoming integration. No participants reported to us, however, any deliberate strategy in response to the first word. A better understanding of this first word, as well as its second word parallel, effect will be important in using ERPs to elucidate relational encoding.

PAGE 86

77 The second pair of intriguing Dms are those that occurred, in the Bound condition conditionalized on paired recognition performance, to the second word; one a widespread, early (300 ms post-word 2 onset) effect, the other a late (1500 ms post-word 2 onset), frontal effect. In the only experiment that we have identified that sequentially presented successive words for associative processing, and then measured ERPs to the second word, Kounios and colleagues (2001), found that participants who successfully fused word pairs into a unitary concept (e.g., computer + virus = “computer virus”) showed ERP differences according to whether they later quickly or slowly identified the order in which pairs were earlier presented. These differences persisted over the three-second interval following presentation of the second word. Interestingly, the initial differences (200-800 ms) were marked by activation in right prefrontal cortex. From 800 – 2100 ms following the second word, however, activation shifted to a region in left medial superior frontal cortex. In many respects, differences in experimental protocols between our experiments and Kounios and colleagues’ make comparisons between the two difficult. However, the fact that Kounios found a subsequent memory effects that persisted throughout a three second post word interval – and our results point to a pair of Dm effects that lasted nearly two seconds, is striking. Likewise, in addition to an earlier set of subsequent memory effects, Mangels, Picton and Craik (2001) found sustained prefrontal positive and sustained posterior negative subsequent memory effects beginning at about 1000 ms after the onset of the word. They speculated that these effects reflected the interaction of a fronto-posterior network where the posterior portion of the network was responsible for sustained object

PAGE 87

78 representation and the frontal part of the network, particularly at the Fp electrodes, playing a role in the elaborative processes that facilitate subsequent recollection and recognition. It is important to note that, similar to our studies, the Fp electrodes recorded a positive-going wave from about 1000 ms to the end of Mangel and colleagues’ interval (2000 ms). Likewise, we found a positive slow wave at Fp sites from about 1000 ms following the first word until the first 200 ms following presentation of the second word. An important difference between Mangels’ and our findings was that the slow wave in Mangels study was positive-going across most frontal electrodes (Fp, AF, F). The slow wave only became negative-going at posterior sites. In our experiments, the slow wave was positive-going only at Fp sites, and negative-going at other electrode locations. Another important difference, however, was in the polarity of the difference wave between Mangels’ findings (positive at frontal sites, negative at posterior locations), and ours (negative at frontopolar sites). Limitations and Future Directions While the results from the experiments presented are unique and contribute to our understanding of the neurocognitive basis of relational encoding and long-term memory performance, there are aspects of the experimental design that limit the conclusions that can be drawn from them. First, although the design provides a unique amount of control over stimulus and task factors that could otherwise confound the results, the static itemlocation design fails to capture either the temporal or the dynamic aspects of episodic memory. As Craik (1989) notes, episodes, as described by Tulving (1984), consist of a series of events, which in turn consist of item/context pairs. Thus, by limiting the “episodes” here to single pairs, we have excluded participants’ experience of ongoing events and the cognition that accompanies it. An initial foray into the dynamic aspect of

PAGE 88

79 episodic encoding might include having participants generate dynamic images (i.e., visualize an ELEPHANT falling off of a CLIFF). Likewise, encoding activity for the temporal order of location-item pairs could be tested. Second, by their design, the experiments allowed for the possibility of some overlap between the correct and incorrect response classes in that some trials in the correct response category (Experiment 1: Intact Correct; Experiment 2: Old – Old) may have been the result of low confidence guesses. Analysis of the confidence ratings in the test phase of Experiment 2 and resorting encoding trials into High and Medium Confidence correct responses and Low Confidence (Guessing) correct and incorrect trials would point to the degree of overlap and the contribution of guessing in the correct responses ERPs. No such confidence ratings were collected in the first experiment so defining the contribution of guessing trials to the correct responses would be more difficult. Third, what role the ease with which pairs were capable of being imaged might have played is unclear. Although data regarding the ease of imagery were collected, these data have not been analyzed. It could be that, if these data were sorted into Easy and Difficult, they would correspond highly to correct and incorrect subsequent memory performance, suggesting a prominent role for the ease of imagery in encoding related memory effects. On the other hand, it might be that the greater cognitive effort expended in generating and maintaining difficult images would yield better memory performance. Fourth, although low-density localization techniques (e.g., LORETA: PascualMarqui, Michel & Lehmann, 1994) are available, the use of a low-density (16 electrode) array made it difficult to attempt more serious source localization analysis. Nevertheless,

PAGE 89

80 our use of traditional interval analyses and ANOVAs that included hemisphere and anterior-posterior groups as factors enabled us to generally identify regional activity. This regional activity was in accord with at least some previous findings where the source of neural activity has been identified using EEG localization (e.g., Kounios et al., 2001) and fMRI (e.g., Prabhakaran et al., 2000) techniques. Fifth, keeping the encoding instructions and test type as between-subjects factors, one of the strengths of the design, also weakens the cross-task comparisons. The benefits of implementing the instruction and test type manipulation between subjects are clear. Participants are less likely to employ relational encoding strategies, even unintentionally, if they havenÂ’t engaged in them in a preceding study block. Likewise, if the test type were implemented as a within-subjects manipulation, participants would be likely to have received the benefit of item-retrieval in the paired recognition task (if it followed item recognition). Thus, the use of a between-subjects design for these factors largely keeps strategies and memory processes discrete from one another. However, the manipulations introduce a greater degree of variability than would be produced if they were manipulated within-subjects and, thus, tend to weaken the statistical comparison. It is possible that encoding comparisons would have produced more extensive differences than were observed. Moreover, manipulating test type as a within-subjects factor (if a way of keeping it from being confounded with retrieval practice could be ascertained) would facilitate the direct comparisons of the encoding processes that support the two memory retrieval types. Finally, these data could be subjected to a coherence analysis, a technique that is being used increasingly in the study of memory encoding (e.g., Summerfield & Mangels,

PAGE 90

81 2005; Weiss, Muller & Rappelsberger, 2000; Weiss & Rappelsberger, 2000). The essence of this type of analysis is to identify regions in which EEG bands from separate brain regions experience phase synchronization of neural oscillations over discrete temporal windows. This phase locking is a considered a candidate code by which information is shared between spatially distinct brain areas (Summerfield & Mangels, 2005). While coherence analysis has been used to examine feature binding (Summerfield & Mangels, 2005) and perception (Engel, Fries & Singer, 2001), it seems to be a potentially important tool for investigating mechanisms that underlie the relational encoding that constitute item-context binding in episodic memory.

PAGE 91

82 APPENDIX A EXPERIMENT 1, STUDY PHASE WORD PAIRS 2.1 A FIREFLY/SIDEWALK / 1.2 A SURFER/STUDIO / 3.3 O STEREO/MAILBOX / 2.4 O BUTTERFLY/BAKERY / 2.5 I POSSUM/LOFT / 2.6 I FLY/GLACIER / 2.7 O TIGER/BARN / 5.8 A PAINTBRUSH/RANCH / 2.9 O GORILLA/HOUSE / 1.10 A SOLDIER/PARTY / 4.11 A BAYONET/WAREHOUSE / 1.12 A SKATER/STADIUM / 3.13 I DRESSER/MUSEUM / 1.14 I SINGER/ROOF / 3.15 O STOOL/RIDGE / 6.16 O TAMBOURINE/CLIFF / 2.17 I FROG/BILLBOARD / 1.18 O LUMBERJACK/AVALANCH E / 3.19 O CRIB/OFFICE / 2.20 O GRASSHOPPER/PENTHOU SE / 1.21 I FIREMAN/BAR / 2.22 O TURTLE/FIREPLACE / 5.23 I WHEELBARROW/RINK / 2.24 A COCKROACH/CARNIVAL / 1.25 O KING/DUMP / 1.26 O DRUMMER/JUNGLE / 3.27 O LAMP/TREE / 7.28 O TRACTOR/DARKROOM / 2.29 A HIPPOPOTAMUS/GALLER Y / 7.30 A HELICOPTER/PHARMACY / 4.31 A ROCKET/CRATER / 2.32 I HORSE/BOX / 3.33 O TOILET/STEPS / 2.34 A MOUSE/KENNEL / 1.35 A DETECTIVE/CAMP / 8.36 A DOLLHOUSE/DRIVE-IN / 7.37 O SUBMARINE/AIRPORT / 1.38 I HIKER/BANK / 7.39 O AIRPLANE/REEF / 2.40 I OWL/RAVINE / 8.41 O BUBBLES/DISCO / 1.42 O DOORMAN/PLANTATION / 5.43 I PENCIL/MANSION / 2.44 A PIG/WATERFALL / 5.45 A RAKE/FARM / 7.46 O TAXI/ZOO / 1.47 O BANDIT/CASINO / 2.48 O GIRAFFE/CABINET / 1.49 I MECHANIC/THICKET / 2.50 O ANT/CATHEDRAL / 3.51 O COUCH/HILL / 5.52 O MOWER/CORNER / 4.53 I KNIFE/PYRAMID / 2.54 A BIRD/MARKET / 6.55 O VIOLIN/WEDDING / 2.56 O OSTRICH/AMPHITHEATR E / 2.57 O BUFFALO/RESORT / 4.58 O BOMB/MOON / 2.59 O DUCK/HIGHWAY / 1.60 A BARTENDER/VALLEY / 1.61 I DANCER/CAF / 2.62 I COW/MORGUE / 2.63 A SEAL/BATHROOM / 4.64 O ROPE/CONVENTION / 4.65 O CANNON/CABIN / 7.66 I SKATES/TORNADO / 2.67 O

PAGE 92

83 OX/CHAPEL / 1.68 O SWIMMER/HAYLOFT / 1.69 A SLAVE/DESERT / 8.70 O ROBOT/RIVER / 1.71 A EXPLORER/TOWER / 5.72 A SHOVEL/DOORWAY / 8.73 O GLOVE/EARTHQUAKE / 1.74 O BAKER/DAM / 2.75 O CHIMPANZEE/ICEBERG / 2.76 A HEDGEHOG/THUNDERSTO RM / 7.77 A TANK/CEMETERY / 5.78 O DRILL/ISLAND / 7.79 O SCOOTER/LABORATORY / 4.80 A BOOK/GROCERY / 8.81 O HORSESHOE/CLOSET / 2.82 A OCTOPUS/PARK / 4.83 A GUN/FENCE / 2.84 A MANATEE/NURSERY / 1.85 A CLOWN/KITCHEN / 2.86 O LIZARD/MEADOW / 1.87 A MAGICIAN/TEPEE / 1.88 O MAID/MOUNTAIN / 1.89 A NUN/CORRAL / 2.90 O WOLF/BOULDER / 7.91 A STREETCAR/SUNSET / 7.92 I BALLOON/MOSQUE / 5.93 O TOOLBOX/COTTAGE / 2.94 A TOUCAN/DRIVEWAY / 1.95 I SAILOR/ATTIC / 1.96 I GYMNAST/PATH / 1.97 O DENTIST/TOMB / 2.98 A LEOPARD/SKYSCRAPER / 2.99 A BEAR/BEDROOM / 1.100 A CARPENTER/DUSK / 5.101 O PLANE/CELLAR / 4.102 A ARROW/PLAYHOUSE / 1.103 I INMATE/CHIMNEY / 1.104 O MILKMAN/COFFIN / 2.105 A PORCUPINE/OUTHOUSE / 1.106 O PROSTITUTE/MALL / 4.107 A FORK/TUNNEL / 7.108 A BICYCLE/BALCONY / 2.109 O LADYBUG/FORT / 8.110 A CRAYONS/SEWER / 2.111 A GOAT/BASEMENT / 1.112 A ASTRONAUT/WELL / 3.113 A STOVE/DECK / 7.114 A TRUCK/QUARRY / 1.115 A GIRL/VOLCANO / 2.116 A PANDA/LAUNDRY / 1.117 O BRIDE/SHOWER / 7.118 O RICKSHAW/UNIVERSITY / 3.119 A VASE/ESCALATOR / 6.120 O FLUTE/FOREST / 8.121 O DOLL/DORMITORY / 1.122 O BOY/SWAMP / 1.123 O SKIER/APARTMENT / 5.124 A LADDER/SNOWSTORM / 2.125 A ANTELOPE/PLAYGROUND / 2.126 O CHICKEN/CANYON / 3.127 O CHAIR/CITY / 1.128 A CONDUCTOR/CLOUD / 2.129 O CRAB/ALLEY / 2.130 A RACCOON/DAYBREAK / 5.131 O VISE/HAILSTORM / 4.132 O PILLOW/PRAIRIE / 2.133 O SQUIRREL/IGLOO / 8.134 A PUPPET/TENT / 5.135 O WRENCH/PORCH / 8.136 I BOOMERANG/CREVICE / 7.137 O SURFBOARD/WINDOW / 8.138 A SOFTBALL/RAMP / 1.139 A JUGGLER/HOTEL / 3.140 O RADIO/ELEVATOR / 2.141 A FERRET/LIBRARY / 6.142 O CLARINET/GYMNASIUM / 2.143 A SPIDER/CAGE / 3.144 O BED/FLOOD / 1.145 A JUDGE/HOSPITAL / 1.146 O GROOM/JAIL / 1.147 O BRICKLAYER/VILLAGE / 2.148 A

PAGE 93

84 LION/HUT / 6.149 A GUITAR/CREEK / 4.150 O SPEAR/FOG / 5.151 O SCREWS/CIRCUS / 2.152 A WHALE/SLAUGHTERHOUS E / 6.153 O HARP/GARAGE / 6.154 A ACCORDION/GEYSER / 5.155 O SANDPAPER/OCEAN / 6.156 A CYMBALS/LAKE / 3.157 A PICTURE/SCHOOL / 4.158 O SWORD/CAVE / 1.159 O COWBOY/HARBOR / 2.160 O RAT/BEACH /

PAGE 94

85 APPENDIX B EXPERIMENT 1, PAIRED RECOGNITION TEST WORD PAIRS 2.1 A O FIREFLY/SIDEWALK / 1.2 A O SURFER/STUDIO / 3.3 O O STEREO/MAILBOX / 2.4 O O BUTTERFLY/BAKERY / 2.5 I O OPOSSUM/LOFT / 2.6 I O FLY/GLACIER / 2.7 O O TIGER/BARN / 5.8 A O PAINTBRUSH/RANCH / 2.9 O O GORILLA/HOUSE / 1.10 A O SOLDIER/PARTY / 4.11 A O BAYONET/WAREHOUSE / 1.12 A O SKATER/STADIUM / 3.13 I O DRESSER/MUSEUM / 1.14 I O SINGER/ROOF / 3.15 O O STOOL/RIDGE / 6.16 O O TAMBOURINE/CLIFF / 2.17 I O FROG/BILLBOARD / 1.18 O O LUMBERJACK/AVALANCH E / 3.19 O O CRIB/OFFICE / 2.20 O O GRASSHOPPER/PENTHOU SE / 1.21 I O FIREMAN/BAR / 2.22 O O TURTLE/FIREPLACE / 5.23 I O WHEELBARROW/RINK / 2.24 A O COCKROACH/CARNIVAL / 1.25 O O KING/DUMP / 1.26 O O DRUMMER/JUNGLE / 3.27 O O LAMP/TREE / 7.28 O O TRACTOR/DARKROOM / 2.29 A O HIPPOPOTAMUS/GALLER Y / 7.30 A O HELICOPTER/PHARMACY / 4.31 A O ROCKET/CRATER / 2.32 I O HORSE/BOX / 3.33 O O TOILET/STEPS / 2.34 A O MOUSE/KENNEL / 1.35 A O DETECTIVE/CAMP / 8.36 A O DOLLHOUSE/DRIVE-IN / 7.37 O O SUBMARINE/AIRPORT / 1.38 I O HIKER/BANK / 7.39 O O AIRPLANE/REEF / 2.40 I O OWL/RAVINE / 8.41 O O BUBBLES/DISCO / 1.42 O O DOORMAN/PLANTATION / 5.43 I O PENCIL/MANSION / 2.44 A O PIG/WATERFALL / 5.45 A O RAKE/FARM / 7.46 O O TAXI/ZOO / 1.47 O O BANDIT/CASINO / 2.48 O O GIRAFFE/CABINET / 1.49 I O MECHANIC/THICKET / 2.50 O O ANT/CATHEDRAL / 3.51 O O COUCH/HILL / 5.52 O O MOWER/CORNER / 4.53 I O KNIFE/PYRAMID / 2.54 A O BIRD/MARKET / 6.55 O O VIOLIN/WEDDING / 2.56 O O OSTRICH/AMPHITHEATR E / 2.57 O O BUFFALO/RESORT / 4.58 O O BOMB/MOON / 2.59 O O DUCK/HIGHWAY / 1.60 A O BARTENDER/VALLEY / 1.61 I O DANCER/CAFE / 2.62 I O COW/MORGUE / 2.63 A O SEAL/BATHROOM /

PAGE 95

86 4.64 O O ROPE/CONVENTION / 4.65 O O CANNON/CABIN / 7.66 I O SKATES/TORNADO / 2.67 O O OX/CHAPEL / 1.68 O O SWIMMER/HAYLOFT / 1.69 A O SLAVE/DESERT / 8.70 O O ROBOT/RIVER / 1.71 A O EXPLORER/TOWER / 5.72 A O SHOVEL/DOORWAY / 8.73 O O GLOVE/EARTHQUAKE / 1.74 O O BAKER/DAM / 2.75 O O CHIMPANZEE/ICEBERG / 2.76 A O HEDGEHOG/THUNDERSTO RM / 7.77 A O TANK/CEMETERY / 5.78 O O DRILL/ISLAND / 7.79 O O SCOOTER/LABORATORY / 4.80 A O BOOK/GROCERY / 8.81 A N HORSESHOE/LAKE / 2.82 I N OCTOPUS/ATTIC / 4.83 A N GUN/SCHOOL / 2.84 A N MANATEE/SNOWSTORM / 1.85 O N CLOWN/COFFIN / 2.86 O N LIZARD/MOUNTAIN / 1.87 A N MAGICIAN/SKYSCRAPER / 1.88 A N MAID/SEWER / 1.89 A N NUN/HOTEL / 2.90 O N WOLF/DORMITORY / 7.91 O N STREETCAR/ELEVATOR / 7.92 O N BALLOON/IGLOO / 1.93 O N LADYBUG/SUNSET / 5.94 A N TOOLBOX/HOSPITAL / 2.95 O N TOUCAN/CITY / 7.96 A N RICKSHAW/ESCALATOR / 1.97 O N SAILOR/CELLAR / 1.98 A N GYMNAST/VOLCANO / 1.99 A N DENTIST/PLAYGROUND / 2.100 O N LEOPARD/FLOOD / 2.101 A N BEAR/OUTHOUSE / 1.102 O N CARPENTER/FORT / 5.103 A N PLANE/CREEK / 4.104 I N ARROW/CREVICE / 1.105 O N INMATE/BOULDER / 1.106 O N MILKMAN/APARTMENT / 5.107 O N BED/FOG / 2.108 O N PORCUPINE/CIRCUS / 1.109 O N PROSTITUTE/GARAGE / 4.110 A N FORK/TENT / 7.111 A N BICYCLE/LAUNDRY / 8.112 A N CRAYONS/BASEMENT / 2.113 O N GOAT/SWAMP / 3.114 A N STOVE/TEPEE / 2.115 O N CHICKEN/HAILSTORM / 7.116 O N TRUCK/ALLEY / 1.117 O N GIRL/BEACH / 2.118 O N PANDA/VILLAGE / 1.119 A N BRIDE/RAMP / 3.120 O N VASE/MALL / 6.121 A N FLUTE/KITCHEN / 8.122 O N DOLL/JAIL / 1.123 A N BOY/LIBRARY / 1.124 O N SKIER/PORCH / 5.125 O N LADDER/OCEAN / 2.126 A N ANTELOPE/NURSERY / 1.127 O N CONDUCTOR/CAVE / 2.128 O N CRAB/FOREST / 2.129 A N RACCOON/DRIVEWAY / 5.130 A N VISE/DUSK / 4.131 A N PILLOW/QUARRY / 3.132 A N CHAIR/FENCE / 8.133 A N PUPPET/SLAUGHTERHOU SE / 5.134 A N WRENCH/TUNNEL / 8.135 I N BOOMERANG/MOSQUE / 7.136 O N SURFBOARD/MEADOW / 8.137 O N SOFTBALL/WINDOW / 1.138 O N JUGGLER/CLOSET / 3.139 A N RADIO/CAGE / 2.140 A N FERRET/CORRAL / 6.141 O N CLARINET/CANYON / 2.142 A N SPIDER/BALCONY / 3.143 A N ASTRONAUT/PLAYHOUSE / 1.144 O N JUDGE/COTTAGE /

PAGE 96

87 1.145 A N GROOM/CLOUD / 1.146 A N BRICKLAYER/PARK / 2.147 A N LION/GEYSER / 6.148 O N GUITAR/HARBOR / 2.149 O N SQUIRREL/PRAIRIE / 4.150 I N SPEAR/CHIMNEY / 5.151 A N SCREWS/BEDROOM / 2.152 O N WHALE/SHOWER / 6.153 O N HARP/UNIVERSITY / 6.154 A N ACCORDION/HUT / 5.155 A N SANDPAPER/DAYBREAK / 6.156 O N CYMBALS/GYMNASIUM / 3.157 A N PICTURE/WELL / 4.158 O N SWORD/TOMB / 1.159 I N COWBOY/PATH / 2.160 A N RAT/DECK /

PAGE 97

88 APPENDIX C EXPERIMENT 2, STUDY PHASE WORD PAIRS 2.1 A FIREFLY/SIDEWALK / 1.2 A SURFER/STUDIO / 3.3 O STEREO/MAILBOX / 2.4 O BUTTERFLY/BAKERY / 2.5 I POSSUM/LOFT / 2.6 I FLY/GLACIER / 2.7 O TIGER/BARN / 5.8 A PAINTBRUSH/RANCH / 2.9 O GORILLA/HOUSE / 1.10 A SOLDIER/PARTY / 4.11 A BAYONET/WAREHOUSE / 1.12 A SKATER/STADIUM / 3.13 I DRESSER/MUSEUM / 1.14 I SINGER/ROOF / 3.15 O STOOL/RIDGE / 6.16 O TAMBOURINE/CLIFF / 2.17 I FROG/BILLBOARD / 1.18 O LUMBERJACK/AVALANCH E / 3.19 O CRIB/OFFICE / 2.20 O GRASSHOPPER/PENTHOU SE / 1.21 I FIREMAN/BAR / 2.22 O TURTLE/FIREPLACE / 5.23 I WHEELBARROW/RINK / 2.24 A COCKROACH/CARNIVAL / 1.25 O KING/DUMP / 1.26 O DRUMMER/JUNGLE / 3.27 O LAMP/TREE / 7.28 O TRACTOR/DARKROOM / 2.29 A HIPPOPOTAMUS/GALLER Y / 7.30 A HELICOPTER/PHARMACY / 4.31 A ROCKET/CRATER / 2.32 I HORSE/BOX / 3.33 O TOILET/STEPS / 2.34 A MOUSE/KENNEL / 1.35 A DETECTIVE/CAMP / 8.36 A DOLLHOUSE/DRIVE-IN / 7.37 O SUBMARINE/AIRPORT / 1.38 I HIKER/BANK / 7.39 O AIRPLANE/REEF / 2.40 I OWL/RAVINE / 8.41 O BUBBLES/DISCO / 1.42 O DOORMAN/PLANTATION / 5.43 I PENCIL/MANSION / 2.44 A PIG/WATERFALL / 5.45 A RAKE/FARM / 7.46 O TAXI/ZOO / 1.47 O BANDIT/CASINO / 2.48 O GIRAFFE/CABINET / 1.49 I MECHANIC/THICKET / 2.50 O ANT/CATHEDRAL / 3.51 O COUCH/HILL / 5.52 O MOWER/CORNER / 4.53 I KNIFE/PYRAMID / 2.54 A BIRD/MARKET / 6.55 O VIOLIN/WEDDING / 2.56 O OSTRICH/AMPHITHEATR E / 2.57 O BUFFALO/RESORT / 4.58 O BOMB/MOON / 2.59 O DUCK/HIGHWAY / 1.60 A BARTENDER/VALLEY / 1.61 I DANCER/CAF / 2.62 I COW/MORGUE / 2.63 A SEAL/BATHROOM / 4.64 O ROPE/CONVENTION / 4.65 O CANNON/CABIN / 7.66 I SKATES/TORNADO / 1.68 O

PAGE 98

89 SWIMMER/HAYLOFT / 1.69 A SLAVE/DESERT / 8.70 O ROBOT/RIVER / 1.71 A EXPLORER/TOWER / 5.72 A SHOVEL/DOORWAY / 8.73 O GLOVE/EARTHQUAKE / 1.74 O BAKER/DAM / 7.77 A TANK/CEMETERY / 5.78 O DRILL/ISLAND / 8.81 O HORSESHOE/CLOSET / 1.85 A CLOWN/KITCHEN / 1.87 A MAGICIAN/TEPEE / 6.120 O FLUTE/FOREST / 6.142 O CLARINET/GYMNASIUM /

PAGE 99

90 APPENDIX D EXPERIMENT 2, OBJECT WORD TEST ITEMS 2.1 A O FIREFLY / 1.2 A O SURFER / 3.3 O O STEREO / 2.4 O O BUTTERFLY / 2.5 I O POSSUM / 2.6 I O FLY / 2.7 O O TIGER / 5.8 A O PAINTBRUSH / 2.9 O O GORILLA / 1.10 A O SOLDIER / 4.11 A O BAYONET / 1.12 A O SKATER / 3.13 I O DRESSER / 1.14 I O SINGER / 3.15 O O STOOL / 6.16 O O TAMBOURINE / 2.17 I O FROG / 1.18 O O LUMBERJACK / 3.19 O O CRIB / 2.20 O O GRASSHOPPER / 1.21 I O FIREMAN / 2.22 O O TURTLE / 5.23 I O WHEELBARROW / 2.24 A O COCKROACH / 1.25 O O KING / 1.26 O O DRUMMER / 3.27 O O LAMP / 7.28 O O TRACTOR / 2.29 A O HIPPOPOTAMUS / 7.30 A O HELICOPTER / 4.31 A O ROCKET / 2.32 I O HORSE / 3.33 O O TOILET / 2.34 A O MOUSE / 1.35 A O DETECTIVE / 8.36 A O DOLLHOUSE / 7.37 O O SUBMARINE / 1.38 I O HIKER / 7.39 O O AIRPLANE / 2.40 I O OWL / 8.41 O O BUBBLES / 1.42 O O DOORMAN / 5.43 I O PENCIL / 2.44 A O PIG / 5.45 A O RAKE / 7.46 O O TAXI / 1.47 O O BANDIT / 2.48 O O GIRAFFE / 1.49 I O MECHANIC / 2.50 O O ANT / 3.51 O O COUCH / 5.52 O O MOWER / 4.53 I O KNIFE / 2.54 A O BIRD / 6.55 O O VIOLIN / 2.56 O O OSTRICH / 2.57 O O BUFFALO / 4.58 O O BOMB / 2.59 O O DUCK / 1.60 A O BARTENDER / 1.61 I O DANCER / 2.62 I O COW / 2.63 A O SEAL / 4.64 O O ROPE / 4.65 O O CANNON / 7.66 I O SKATES / 1.67 O O SWIMMER / 1.68 A O SLAVE / 8.69 O O ROBOT / 1.70 A O EXPLORER / 5.71 A O

PAGE 100

91 SHOVEL / 8.72 O O GLOVE / 1.73 O O BAKER / 7.74 A O TANK / 5.75 O O DRILL / 8.76 O O HORSESHOE / 1.77 A O CLOWN / 1.78 A O MAGICIAN / 6.79 O O FLUTE / 6.80 O O CLARINET / 2.81 O N OX / 2.82 O N CHIMPANZEE / 2.83 A N HEDGEHOG / 7.84 O N SCOOTER / 4.85 A N BOOK / 2.86 A N OCTOPUS / 4.87 A N GUN / 2.88 A N MANATEE / 2.89 O N LIZARD / 1.90 O N MAID / 1.91 A N NUN / 2.92 O N WOLF / 7.93 A N STREETCAR / 7.94 I N BALLOON / 5.95 O N TOOLBOX / 2.96 A N TOUCAN / 1.97 I N SAILOR / 1.98 I N GYMNAST / 1.99 O N DENTIST / 2.100 A N LEOPARD / 2.101 A N BEAR / 1.102 A N CARPENTER / 5.103 O N PLANE / 4.104 A N ARROW / 1.105 I N INMATE / 1.106 O N MILKMAN / 2.107 A N PORCUPINE / 1.108 O N PROSTITUTE / 4.109 A N FORK / 7.110 A N BICYCLE / 2.111 O N LADYBUG / 8.112 A N CRAYONS / 2.113 A N GOAT / 1.114 A N ASTRONAUT / 3.115 A N STOVE / 7.116 A N TRUCK / 1.117 A N GIRL / 2.118 A N PANDA / 1.119 O N BRIDE / 7.120 O N RICKSHAW / 3.121 A N VASE / 8.122 O N DOLL / 1.123 O N BOY / 1.124 O N SKIER / 5.125 A N LADDER / 2.126 A N ANTELOPE / 2.127 O N CHICKEN / 3.128 O N CHAIR / 1.129 A N CONDUCTOR / 2.130 O N CRAB / 2.131 A N RACCOON / 5.132 O N VISE / 4.133 O N PILLOW / 2.134 O N SQUIRREL / 8.135 A N PUPPET / 5.136 O N WRENCH / 8.137 I N BOOMERANG / 7.138 O N SURFBOARD / 8.139 A N SOFTBALL / 1.140 A N JUGGLER / 3.141 O N RADIO / 2.142 A N FERRET / 2.143 A N SPIDER / 3.144 O N BED / 1.145 A N JUDGE / 1.146 O N GROOM / 1.147 O N BRICKLAYER / 2.148 A N LION / 6.149 A N GUITAR / 4.150 O N SPEAR / 5.151 O N SCREWS / 2.152 A N WHALE / 6.153 O N HARP / 6.154 A N ACCORDION / 5.155 O N SANDPAPER / 6.156 A N CYMBALS /

PAGE 101

92 3.157 A N PICTURE / 4.158 O N SWORD / 1.159 O N COWBOY / 2.160 O N RAT /

PAGE 102

93 APPENDIX E EXPERIMENT 2, LOCATION WORD TEST ITEMS 2.1 A O SIDEWALK / 1.2 A O STUDIO / 3.3 O O MAILBOX / 2.4 O O BAKERY / 2.5 I O LOFT / 2.6 I O GLACIER / 2.7 O O BARN / 5.8 A O RANCH / 2.9 O O HOUSE / 1.1 A O PARTY / 4.11 A O WAREHOUSE / 1.12 A O STADIUM / 3.13 I O MUSEUM / 1.14 I O ROOF / 3.15 O O RIDGE / 6.16 O O CLIFF / 2.17 I O BILLBOARD / 1.18 O O AVALANCHE / 3.19 O O OFFICE / 2.2 O O PENTHOUSE / 1.21 I O BAR / 2.22 O O FIREPLACE / 5.23 I O RINK / 2.24 A O CARNIVAL / 1.25 O O DUMP / 1.26 O O JUNGLE / 3.27 O O TREE / 7.28 O O DARKROOM / 2.29 A O GALLERY / 7.3 A O PHARMACY / 4.31 A O CRATER / 2.32 I O BOX / 3.33 O O STEPS / 2.34 A O KENNEL / 1.35 A O CAMP / 8.36 A O DRIVE-IN / 7.37 O O AIRPORT / 1.38 I O BANK / 7.39 O O REEF / 2.4 I O RAVINE / 8.41 O O DISCO / 1.42 O O PLANTATION / 5.43 I O MANSION / 2.44 A O WATERFALL / 5.45 A O FARM / 7.46 O O ZOO / 1.47 O O CASINO / 2.48 O O CABINET / 1.49 I O THICKET / 2.5 O O CATHEDRAL / 3.51 O O HILL / 5.52 O O CORNER / 4.53 I O PYRAMID / 2.54 A O MARKET / 6.55 O O WEDDING / 2.56 O O AMPHITHEATRE / 2.57 O O RESORT / 4.58 O O MOON / 2.59 O O HIGHWAY / 1.6 A O VALLEY / 1.61 I O CAF / 2.62 I O MORGUE / 2.63 A O BATHROOM / 4.64 O O CONVENTION / 4.65 O O CABIN / 7.66 I O TORNADO / 1.67 O O HAYLOFT / 1.68 A O DESERT / 8.69 O O RIVER / 1.7 A O TOWER / 5.71 A O

PAGE 103

94 DOORWAY / 8.72 O O EARTHQUAKE / 1.73 O O DAM / 7.74 A O CEMETERY / 5.75 O O ISLAND / 8.76 O O CLOSET / 1.77 A O KITCHEN / 1.78 A O TEPEE / 6.79 O O FOREST / 6.8 O O GYMNASIUM / 2.81 O N CHAPEL / 2.82 O N ICEBERG / 2.83 A N THUNDERSTORM / 7.84 O N LABORATORY / 4.85 A N GROCERY / 2.86 A N PARK / 4.87 A N FENCE / 2.88 A N NURSERY / 2.89 O N MEADOW / 1.9 O N MOUNTAIN / 1.91 A N CORRAL / 2.92 O N BOULDER / 7.93 A N SUNSET / 7.94 I N MOSQUE / 5.95 O N COTTAGE / 2.96 A N DRIVEWAY / 1.97 I N ATTIC / 1.98 I N PATH / 1.99 O N TOMB / 2.1 A N SKYSCRAPER / 2.101 A N BEDROOM / 1.102 A N DUSK / 5.103 O N CELLAR / 4.104 A N PLAYHOUSE / 1.105 I N CHIMNEY / 1.106 O N COFFIN / 2.107 A N OUTHOUSE / 1.108 O N MALL / 4.109 A N TUNNEL / 7.11 A N BALCONY / 2.111 O N FORT / 8.112 A N SEWER / 2.113 A N BASEMENT / 1.114 A N WELL / 3.115 A N DECK / 7.116 A N QUARRY / 1.117 A N VOLCANO / 2.118 A N LAUNDRY / 1.119 O N SHOWER / 7.12 O N UNIVERSITY / 3.121 A N ESCALATOR / 8.122 O N DORMITORY / 1.123 O N SWAMP / 1.124 O N APARTMENT / 5.125 A N SNOWSTORM / 2.126 A N PLAYGROUND / 2.127 O N CANYON / 3.128 O N CITY / 1.129 A N CLOUD / 2.13 O N ALLEY / 2.131 A N DAYBREAK / 5.132 O N HAILSTORM / 4.133 O N PRAIRIE / 2.134 O N IGLOO / 8.135 A N TENT / 5.136 O N PORCH / 8.137 I N CREVICE / 7.138 O N WINDOW / 8.139 A N RAMP / 1.14 A N HOTEL / 3.141 O N ELEVATOR / 2.142 A N LIBRARY / 2.143 A N CAGE / 3.144 O N FLOOD / 1.145 A N HOSPITAL / 1.146 O N JAIL / 1.147 O N VILLAGE / 2.148 A N HUT / 6.149 A N CREEK / 4.15 O N FOG / 5.151 O N CIRCUS / 2.152 A N SLAUGHTERHOUSE / 6.153 O N GARAGE / 6.154 A N GEYSER / 5.155 O N OCEAN / 6.156 A N LAKE /

PAGE 104

95 3.157 A N SCHOOL / 4.158 O N CAVE / 1.159 O N HARBOR / 2.16 O N BEACH /

PAGE 105

96 LIST OF REFERENCES Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 11, 417-423. Baker, J. T., Sanders, A. L., Maccotta, L., & Buckner, R. L. (2001). Neural correlates of verbal memory encoding during semantic and structural processing tasks. NeuroReport, 12, 1251-1256. Bar, M., & Aminoff, E. (2003). Cortical analysis of visual context. Neuron, 38, 347-358. Battig, W. F., & Montague, W. E. (1969). Category norms for verbal items in 56 categories: A replication and extension of the Connecticut category norms. Journal of Experimental Psychology, 80, (3, pt. 2), 1-46. Bor, D., Duncan, J., Wiseman, R. J., & Owen, A. M. (2003). Encoding strategies dissociate prefrontal activity from working memory demand. Neuron, 37, 361-367. Bower, G.H. (1970). Imagery as a relational organizer in associative memory. Journal of Verbal Learning and Verbal Behavior, 9 529-533. Brewer, J. B., Zhao, Z., Desmond, J. E., Glover, G. H., & Gabrieli, J. D. E. (1998). Making memories: Brain activity that predicts how well visual experience will be remembered. Science, 281, 1185-1187. Buckner, R. L., Wheeler, M. E., & Sheridan, M. (2001). Encoding processes during retrieval tasks. Journal of Cognitive Neuroscience, 13, 406-415. Cabeza, R., & Nyberg, L. (2000). Imaging cognition II: An empirical review of 275 PET and fMRI studies. Journal of Cognitive Neuroscience, 12, 1-47. Cansino, S., Maquet, P., Dolan, R. J., & Rugg, M. D. (2002). Brain activity underlying encoding and retrieval of source memory. Cerebral Cortex, 12, 1048-1056. Casasanto, D. J., Killgore, W. D. S., Maldjian, J. A., Glosser, G., Alsop, D. C., Cooke, A. M., Grossman, M., & Detre, J. A. (2002). Neural correlates of successful and unsuccessful verbal memory encoding. Brain and Language, 80, 287-295.

PAGE 106

97 Chalfonte, B. L., & Johnson, M. K. (1996). Feature memory and binding in young and older adults. Memory & Cognition, 24, 403-416. Clark, D., & Wagner, A. D. (2003). Assembling and encoding word representations: fMRI subsequent memory effects implicate a role for phonological control. Neuropsychologia, 41, 304-317. Craik, F. I. M. (1989). On the making of episodes. In H.L.Roediger III & F. I. M. Craik (Eds.), Varieties of memory and consciousness: Essays in honour of Endel Tulving (pp. 43-57). Hillsdale, NJ: Lawrence Erlbaum Associates. Daselaar, S. M., Veltman, D. J., Rombouts, S. A. R. B., Raaijmakers, J. G. W., & Jonker, C. (2003). Neuroanatomical correlates of episodic encoding and retrieval in young and elderly subjects. Brain, 126, 43-56. Davachi, L., Maril, A., & Wagner, A. D. (2001). When keeping in mind supports later bringing to mind: Neural markers of phonological rehearsal predict subsequent remembering. Journal of Cognitive Neuroscience, 13, 10591070. Davachi, L., Mitchell, J. P., & Wagner, A. D. (2003). Multiple routes to memory: Distinct medial temporal lobe processes build item and source memories. Proceedings of the National Academy of Sciences, 100, 2157-2162. Davachi, L., & Wagner, A. D. (2002). Hippocampal contributions to episodic encoding: Insights from relational and item-based learning. Journal of Neurophysiology, 88, 982-990. Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134 9-21. Duarte, A., Ranganath, C., Winward, L., Hayward, D., & Knight, R. T. (2004). Dissociable neural correlates in familiarity and recollection during the encoding and retrieval of pictures. Cognitive Brain Research, 18, 255-272. Engel, A. K., Fries, P., & Singer, W. (2001). Dynamic predictions: Oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2, 704-716. Fabiani, M., & Donchin, E. (1995). Encoding process and memory organization: A model of the van Restorff effect. Journal of Experimental Psychology: Learning, Memory and Cognition, 21, 224-240. Fabiani, M., Karis & Donchin, E. (1990). Effects of mnemonic strategy manipulation in a Von Restorff paradigm. Electroencephalograhpy and Clinical Neurophysiology, 75, 22-35.

PAGE 107

98 Fabiani, M., Karis, D., & Donchin, E. (1986). P300 and recall in an incidental memory paradigm. Psychophysiology, 23, 298-308. Fernandez, G., Effern, A., Grunwald, T., Pezer, N., Lehnertz, K., Kampelmann, M., Van Roost, D., & Elger, C. E. (1999). Real-time tracking of memory formation in the human rhinal cortex and hippocampus. Science, 285, 1582-1585. Fernandez, G., Weyerts, H., Tendolkar, I., Smid, H. G. O. M., Scholz, M., & Heinze, H.-J. (1998). Event-related potentials of verbal encoding into episodic memory: Dissociation between the effects of subsequent memory performance and distinctiveness. Psychophysiology, 35, 709-720. Fletcher, P. C., Frith, C. D., & Rugg, M. D. (1997). The functional neuroanatomy of episodic memory. Trends in Neurosciences, 20, 213-218. Fletcher, P. C., & Henson, R. N. A. (2001). Frontal lobes and human memory: Insights from functional neuroimaging. Brain, 124, 849-881. Friedman, D. (1990). ERPs during continuous recognition for words. Biological Psychology, 30, 61-87. Friedman, D., Ritter, W., & Snodgrass, J. G. (1996). ERPs during study as a function of subsequent direct and indirect memory testing in young and old adults. Cognitive Brain Research, 4, 1-13. Friedman, D., & Trott, C. (2000). An event-related potential study of encoding in young and older adults. Neuropsychologia, 38, 542-557. Gonsalves, B., & Paller, K. A. (2000). Neural events that underlie remembering something that never happened. Nature Neuroscience, 3, 1316-1321. Guo, C., Voss, J. L., & Paller, K. A. (2005). Electrophysiological correlates of forming memories for faces, names, and face-name associations. Cognitive Brain Research, 22, 153-164. Guo, C., Zhu, Y., Ding, J., Fan, S., & Paller, K. A. (2004). An electrophysiological investigation of memory encoding, depth of processing, and word frequency in humans. Neuroscience Letters, 356, 7982. Habib, R., Nyberg, L., & Tulving, E. (2003). Hemispheric asymmetries of memory: The HERA model revisited. Trends in Cognitive Sciences, 7, 241-245. Hasher, L., & Zacks, R. T. (1979). Automatic and effortful processes in memory. Journal of Experimental Psychology: General, 108, 356-388.

PAGE 108

99 Henkel, L. A., Johnson, M. K., & DeLeonardis, D. M. (1998). Aging and source monitoring: Cognitive processes and neuropsychological correlates. Journal of Experimental Psychology: General, 127, 251-268. Henson, R. N. A., Rugg, M. D., Shallice, T., Josephs, O., & Dolan, R. J. (1999). Recollection and familiarity in recognition memory: An event-related functional magnetic resonance imaging study. Journal of Neuroscience, 19, 3962-3972. Hunt, R. R., & Einstein, G. O. (1981). Relational and item specific information in memory. Journal of Verbal Learning and Verbal Behavior, 20, 497-514. Jackson, III, O., & Schacter, D. L. ( 2003). Encoding activity in anterior medial temporal lobe supports subsequent associative recognition. Neuroimage, 21, 456-462. Janata, P. (2001). Brain electrical activity evoked by mental formation of auditory expectations and images. Brain Topography, 13, 169-193. Johnson, M. K. (1992). MEM: Mechanisms of recollection. Journal of Cognitive Neuroscience, 4, 268-280. Johnson, M. K., & Hirst, W. (1993). MEM: Memory subsystems as processes. In A.Collins, S. Gathercole, M. Conway, & P. Morris (Eds.), Theories of Memory (pp. 241-286). Hillsdale, NJ: Erlbaum. Johnson, Jr., R. (1995). Event-related potential insights into the neurobiology of memory systems. In F. Boller & J. Grafman. Handbook of Neuropsychology (pp. 135-164). Amsterdam: Elsevier. Karis, D., Fabiani, M., & Donchin, E. (1984). P300 and memory: Individual differences in the Von Restorff effect. Cognitive Psychology, 16, 177-216. Kensinger, E. A., Clarke, R. J., & Corkin, S. (2003). What neural correlates underlie successful encoding and retrieval? A functional magnetic resonance imaging study using a divided attention paradigm. Journal of Neuroscience, 23, 2407-2415. Kirchoff, B. A., Wagner, A. D., Maril, A., & Stern, C. E. (2000). Prefrontaltemporal circuitry for episodic encoding and subsequent memory. Journal of Neuroscience, 20, 6173-6180. Kounios, J., Smith, R., Yang, W., Bachman, P., & D'Esposito, M. (2001). Cognitive association formation in human memory revealed by spatiotemporal brain imaging. Neuron, 29, 297-306. Lian, J., Goldstein, A., Donchin, E., & He, B. (2002). Cortical potential imaging of episodic memory encoding. Brain Topography, 15, 29-36.

PAGE 109

100 Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279-281. Mangels, J. A., Picton, T. W., & Craik, F. I. M. (2001). Attention and successful episodic encoding: an event-related potential study. Cognitive Brain Research, 11, 77-95. Mayes, A. R. & Montaldi, D. (1999). The neuroimaging of long-term memory encoding processes. Memory, 7, 613-659. Mecklinger, A., & Muller, N. (1996). Dissociations in the processing of "what" and "where" information in working memory: An event-related potential analysis. Journal of Cognitive Neuroscience, 8, 453-473. Mitchell, K. J., Johnson, M. K., Raye, C. L., & D'Esposito, M. (2000a). fMRI evidence of age-related hippocampal dysfunction in feature binding in working memory. Cognitive Brain Research, 10, 197-206. Mitchell, K. J., Johnson, M. K., Raye, C. L., Mather, M., & D'Esposito, M. (2000b). Aging and reflective processes of working memory: Binding and test load deficits. Psychology and Aging, 15, 527-541. Neville, H. J., Kutas, M., Chesney, G., & Schmidt, A. L. (1986). Event-related brain potentials during initial encoding and recognition memory of congruous and incongruous words. Journal of Memory and Language, 25, 75-92. Nyberg, L. (2002). Levels of processing: A view from functional brain imaging. Memory, 10, 345-348. Nyberg, L., Cabeza, R., & Tulving, E. (1996). PET studies of encoding and retrieval: The HERA model. Psychonomic Bulletin and Review, 3, 135148. Otten, L. J., Henson, R. N. A., & Rugg, M. D. (2001). Depth of processing effects on neural correlates of memory encoding: Relationship between findings from acrossand within-task comparisons. Brain, 124, 399-412. Otten, L. J. & Rugg, M. D. (2001). Task-dependency of the neural correlates of episodic encoding as measured by fMRI. Cerebral Cortex, 11, 1150-1160. Paller, K. A., & Wagner, A. D. (2002). Observing the transformation of experience into memory. Trends in Cognitive Sciences, 6, 93-102. Pascual-Marqui, R. D., Michel, C. M. & Lehmann, D. (1994). Low resolution electromagnetic tomography: A new method for localizing electrical activity in the brain. International Journal of Psychophysiology, 18, 49-65.

PAGE 110

101 Prabhakaran, V., Narayanan, K., Zhao, Z., & Gabrieli, J. D. E. (2000). Integration of diverse information in working memory within the frontal lobe. Nature Neuroscience, 3, 85-90. Ranganath, C., Johnson, M. K., & D'Esposito, M. (2003). Prefrontal activity associated with working memory and episodic long-term memory. Neuropsychologia, 41, 378-389. Ranganath, C., Yonelinas, A. P., Cohen, M. X., Dy, C. J., Tom, S. M. DÂ’ Esposito, M. (2004). Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia, 42, 2-13. Raye, C. L., Johnson, M. K., Mitchell, K. J., Reeder, J. A., & Greene, E. J. (2002). Neuroimaging a single thought: dorsolateral PFC activity associated with refreshing just-activated information. Neuroimage, 15, 447-453. Reber, P. J., Siwiec, R. M., Gitelman, D. R., Parrish, T. B., Mesulam, M. M., & Paller, K. A. (2002). Neural correlates of successful encoding identified using functional magnetic resonance imaging. Journal of Neuroscience, 22, 9541-9548. Rossi, S., Cappa, S. F., Babiloni, C., Pasqualetti, P., Miniussi, C., Carducci, F., Babiloni, F., & Rossini, P. M. (2001). Prefontal cortex in long-term memory: an "interference" approach using magnetic stimulation. Nature Neuroscience, 4, 948-952. Rubin, D. C., & Friendly, M. (1986). Predicting which words get recalled: Measures of free recall, availability, goodness, emotionality and pronouncability for 925 nouns. Memory and Cognition, 14, 79-94. Rugg, M. D., & Allan, K. (2000). Event-related potential studies of memory. In E. Tulving & F. I. M. Craik (Eds.) Oxford Handbook of Memory (pp. 521537). London: Oxford University Press Rypma, B., & D'Esposito, M. (2003). A subsequent-memory effect in dorsolateral prefrontal cortex. Cognitive Brain Research, 16, 162-166. Sanquist, T. F., Rohrbaugh, J. W., Syndulko, K., & Lindsley, D. B. (1980) Electrocortical signs of levels of processing: Perceptual analysis and recognition memory. Psychophysiology, 17, 568-576. Schacter, D. L (2001). The Seven Sins of Memory: How the Mind Forgets and Remembers Boston, Houghton-Mifflin Schacter, D. L., & Wagner, A. D. (1999). Medial temporal lobe activations in fMRI and PET studies of encoding and retrieval. Hippocampus, 9, 7-24.

PAGE 111

102 Schott, B., Richardson-Klavehn, A., Heinze, H.-J., & Duzel, E. (2002). Perceptual priming versus explicit memory: Dissociable neural correlates at encoding. Journal of Cognitive Neuroscience, 14, 578-592. Smith, M. E. (1993). Neurophysiological manifestations of recollective experience during recognition memory judgments. Journal of Cognitive Neuroscience, 5, 1-13. Smith, M. E., & Halgren, E. (1989). Dissociation of recognition memory components following temporal lobe lesions. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 50-60. Sommer, T., Rose, M., Weiller, C., & Buchel, C. (2005). Contributions of occipital, parietal and parahippocampal cortex to encoding of objectlocation associations. Neuropsychologia, 43, 732-743. Sperling, R., Chua, E., Cocchiarella, A., Rand-Giovanetti, E., Poldrack, R., Schacter, D. L., & Albert, M. (2003). Putting names to faces: Successful encoding of associative memories activates the anterior hippocampal formation. Neuroimage, 20, 1400-1410. Stark, C. E. L., & Okado, Y. (2003). Making memories without trying: medial temporal lobe activity associated with incidental memory formation during recognition. Journal of Neuroscience, 23, 6748-6753. Strange, B. A., Otten, L. J., Josephs, O., Rugg, M. D., & Dolan, R. J. (2002). Dissociable human perirhinal, hippocampal, and parahippocampal roles during verbal encoding. Journal of Neuroscience, 22, 523-528. Summerfield, C., & Mangels, J. A. (2005). Coherent theta-band EEG activity predicts item-context binding during encoding. Neuroimage, 24, 692-703. Tulving, E. (1984). Precis of elements of episodic memory. Behavioral and Brain Sciences, 7, 223-238. Tulving, E. (1985). Memory and consciousness. Canadian Psychology, 26, 1-12. Tulving, E., Kapur, S., Craik, F. I. M., Moscovitch, M., & Houle, S. (1994). Hemispheric encoding/retrieval asymmetry in episodic memory: Positron emission tomography findings. Proceedings of the National Academy of Science, 91, 2016-2020. Turriziani, P., Fadda, L., Caltagirone, C., & Carlesimo, G. A. (2004). Recognition memory for single items and for associations in amnesic patients. Neuropsychologia, 42, 426-433.

PAGE 112

103 Van Petten, C., & Senkfor, A. J. (1996). Memory for words and novel visual patterns: Repetition, recognition and encoding effects in the event-related brain potential. Psychophysiology, 33, 491-506. Wagner, A. D. (1999). Working memory contributions to human learning and remembering. Neuron, 22, 19-22. Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., Rosen, B. R., & Buckner, R. L. (1998). Building memories: Remembering and forgetting of verbal experiences as predicted by brain activity. Science, 281, 1188-1191. Weyerts, H., Tendolkar, I., Smid, H. G. O. M., & Heinze, H.-J. (1997). ERPs to encoding and recognition in two different inter-item association tasks. NeuroReport, 8, 1583-1588. Yonelinas, A. P. (2002). The nature of recollection and familiarity: A review of 30 years of research. Journal of Memory and Language, 46, 441-517. Yovel, G., & Paller, K. A. (2004). The neural basis of the butcher-on-the-bus phenomenon: when a face seems familiar but is not remembered. Neuroimage, 21, 789-800.

PAGE 113

104 BIOGRAPHICAL SKETCH Brian Howland was born in Washington, D.C., on June 13, 1961. He attended high school at James Madison High School in Vienna, Virginia, and graduated in 1979. He attended the College of Wooster in Wooster, Ohio, and graduated with a B.A. in economics in 1983. Upon graduation, he attended Washington and Lee University School of Law from which he graduated in 1986. He was engaged in the private practice of law in Virginia, and subsequently, Pennsylvania for almost five years. In 1993, he enrolled in graduate school in the Department of Psychology at the University of Florida, concentrating in cognitive psychology. He received the Master of Science degree in 1997 and will receive the Doctor of Philosophy degree in August 2005.