1 SEMANTIC -MOTOR REPRESENTATIONS: EFFECTS ON LANGUAGE AND MOTOR PRODUCTION By AMY D. RODRIGUEZ A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2010
2 2010 Amy D. Rodriguez
3 In loving memory of my abuela, Carmen Rodriguez
4 ACKNOWLEDGMENTS I owe a debt of gratitude to my colleagues, family and friends who shared this journey with me I thank my chair and mentor, Jay Rosenbek, for his unwavering support and for teaching me about research, life, and the importance of keeping the human element in all I do. I thank Jamie Reilly, my co-chair, for countless hours of guidance in the developmen t of this dissertation and for his sense of humor that redeemed me many times I thank Leslie Gonzalez Rothi for creating opportunities for me to learn and grow as a researcher and for her loving support in all of my endeavors I thank Lori Altmann for her enthusiasm for my research and for the invaluable cont ributions she made to t his dissertation I thank Stacie Raymer for expanding my knowledge of rehabilitation research and for being wise and gentle mentor ever y step of the way. To my family, I love you. I am the person I am because of the influence that each of you has had on my life. Thank you for believing in me and for sharing in my joys and sorrows along the way T hank you most of all, for loving me even when I didnt call or write enough. I couldnt have done this w ithout your prayers and support. I thank my dog, Pancho, a loyal companion and li ving proof that where there is great love there are always miracles. F inally, to my friends who made me laugh, let me cry, and let me sing. Words seem so in adequate to describe the love and gratitude I feel for each of you My heart is full.
5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ...................................................................................................... 4 LIST OF TABLES ................................................................................................................ 8 LIST OF FIGURES .............................................................................................................. 9 ABSTRACT ........................................................................................................................ 10 CHAPTER 1 INTRODUCTION ........................................................................................................ 12 Overview ..................................................................................................................... 13 Review of the Literature .............................................................................................. 14 Theories of Semantic Representation ................................................................. 14 Evidence for Sensorimotor Theories Perceptual Representation ..................... 16 Evidence for Sensorimotor Theories Motor Representation ............................. 17 Neurophysiological evidence from perception studies ................................. 17 Neurophysiological evidence from production studies ................................. 18 Neuropsychological evidence ....................................................................... 19 Embodied Cognition ............................................................................................. 22 Functional Interaction of Language and Motor Systems .................................... 23 Language effects on the motor system ........................................................ 23 Motor effects on the language system .......................................................... 25 Neuropsychological evidence ....................................................................... 26 Reciprocal Activation of Language and Motor Systems ............................................ 27 Languag e Production ........................................................................................... 28 Motor Production .................................................................................................. 28 Purpose of the Investigation ....................................................................................... 29 2 STIMULI STANDARDIZATION .................................................................................. 32 Participants ................................................................................................................. 32 Semantic Category Selection ..................................................................................... 32 Procedure ............................................................................................................. 32 Results .................................................................................................................. 33 Word Selection ............................................................................................................ 33 Proced ure ............................................................................................................. 34 Results .................................................................................................................. 35 3 PARTICIPANTS FOR EXPERIMENTS 1 3 ............................................................... 37 Inclusion and Exclusion Criteria ................................................................................. 37 Randomization ............................................................................................................ 37
6 4 EXPERIMENT1: EFFECT OF SEMANTIC -MOTOR REPRESENTATIONS ON CONCURRENT, CONTINUOUS L ANGUAGE AND MOTOR PRODUCTION ........ 39 Procedure .................................................................................................................... 39 Data Acquisition and Processing ............................................................................... 40 Statistical Analyses and Results ................................................................................ 42 Time on Task Analyses ........................................................................................ 43 Total Task Output Analyses ................................................................................. 44 Interim Discussion ...................................................................................................... 45 Finger -Tapping Rate ............................................................................................ 45 Word Production Rate .......................................................................................... 46 5 EXPRERIMENT 2: EFFECTOF SEMANTIC -MOTOR REPRESENTATIONS ON SEQUENTIAL LANGUAGE TO MOTOR PRODUCTION ......................................... 50 Experimental Procedure ............................................................................................. 50 Experimental Trial Structure ....................................................................................... 51 Control Condition and Trial Structure ......................................................................... 52 Data A cquisition and Processing ............................................................................... 53 Statistical Analysis and Results ................................................................................. 53 Interim Discussion ...................................................................................................... 54 6 EXPERIMENT 3: EFFECT OF SEMANTIC -MOTOR REPRESENTATIONS ON SEQUENTIAL MOTOR TO LANGUAGE PRODUCTION ......................................... 58 Experimental Procedure ............................................................................................. 58 Experimental Trial Structure ....................................................................................... 59 Control Condition and Trial Structure ......................................................................... 60 Data Acquisition and Processing ............................................................................... 60 Statistical Analysis and Results ................................................................................. 61 Interim Discussion ...................................................................................................... 62 7 DISCUSSION .............................................................................................................. 65 Resource Allocation Theory ....................................................................................... 65 Single Resource Theories ................................................................................... 66 Multiple Resource Theories ................................................................................. 67 Findings from Experiment 1 ................................................................................. 68 PictureWord Interference Effects .............................................................................. 69 Findings from Experiment 2 ................................................................................. 70 Findings from Experiment 3 ................................................................................. 71 Summary of Findings from Ex periments 13 ............................................................. 72 Implications for Theories of Semantic Memory ......................................................... 74 Implications for Individuals with Neurologic Disease ................................................ 75 Considerations for Future Research .......................................................................... 77 Conclusion .................................................................................................................. 78
7 APPENDIX A INSTRUCTIONS FOR PARTICIPANT RATINGS OF WORD STIMULI ................... 80 B PSYCHOLINGUISTIC CHARACTERISTICS OF STIMULI IN SETS 1 and 2 .......... 81 C DESCRIPTION OF NONWORD STIMULI SELECTION ........................................... 84 LIST OF REFERENCES ................................................................................................... 85 BIOGRAPHICAL SKETCH ................................................................................................ 93
8 LIST OF TABLES Tab le page 2 -1 Results for verbal fluency subcategories ............................................................... 36 2 -2 Mean concreteness and familiarity ratings for Sets 1 an d 2 ................................. 36 4 -1 Five conditions that comprise Experiment 1 ......................................................... 48 4 -2 Word production and finger tapping means by semantic category ...................... 48 4 -3 Time on Task ratios by Condition and Task .......................................................... 49 4 -4 Total Task Output ratios by Condition and Task ................................................... 49 5 -1 Reaction time means for pointing to a fixed target by ISI and Condition ............. 57 6 -1 Response time means for word production by ISI and Condition ........................ 64
9 LIST OF FIGURES Figure page 1 -1 The direction of effect investigated in Experiments 13. ....................................... 31 4 -1 Depiction of dual task effects on tapping and word production during time on task by semantic condition. ..................................................................................... 49 5 -1 Schematic depiction of the trial structure for Experiment 2. ................................. 57 5 -2 Depiction of MR and VR word primes on pointing by ISI. ..................................... 57 6 -1 Schematic depiction of the trial structure for Experi ment 3. ................................. 64 6 -2 Depiction of the effects of motor priming on MR and VR words by ISI. ............... 64 7 -1 Schematic depiction of multiple re source pools used for word production. ......... 79 7 -2 Schematic depiction of multiple resource pools used for finger tapping. ............. 79
10 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy SEMANTIC -MOTOR REPRESENTATIONS: EFFECTS ON LANGUAGE AND MOTOR PRODUCTION By Amy D. Rodriguez May 2010 Chair: Name: John C. Rosenbek Co chair : James J. Reilly Major: Rehabilitation Science Semantic memory contains knowledge of the meaning of objects, concepts, and words. Sensorimotor theories of semantic memory suggest that perceptual and/or motor features of co ncepts are distributed across brain regions responsible for processing those features. Embodied cognition, a framework for understanding sensorimotor representations, proposes that concepts are grounded in ones interaction with the world. For example, mot or features of the concept write are instantiated in the motor cortex by observin g or enact ing the task of writing and t his motor representation is linked to language areas that store the lexical -semantic representation of write. This suggests a funct ional interaction between language and motor systems for concepts involving human motor action (i.e., semantic -motor representations). This interaction is well -documented in the literature, but to our knowledge, no studies have investigated the effect of s emantic motor representations on continuous, concurrent language and motor production. Additionally, the reciprocality of these effects has not been investigated in sequential paradigms in which there is no overlap in task production.
11 The purpose of our st udy wa s to determine the effects of shared semantic -motor representations on concurrent and sequential language and motor production. Forty healthy adults, age 1822 years, participated in three experiments comparing the effects of motor -related (MR) words (e.g., write) and visually -related (VR) words (e.g., bloom) on language and motor production. Experiment 1 investigated the effect of concurrent continuous language and motor production during a generative naming and finger tapping task under two semanti c conditions Experiment 2 investigated the effect of MR and VR word production on pointing to a target. Experiment 3 investigated the effect of finger tapping on MR and VR word production. Experiment 1 demonstrated that dual task tapping and word producti on facilitated performance in the motor related s emantic condition. Experiment 2 demonstrated that MR words interfered with poi nting to a target at a short delay and facilitated pointing at a longer delay Experiment 3 demonstrated that a tapping interfered with MR word production at a short delay and facilitated MR word production at a longer delay. Th ese results suggest reciprocal activation of language and motor systems providing support for sensorimotor theories and more generally for embodied cognition
12 CHAPTER 1 INTRODUCTION Semantic memory contains knowledge of the meaning of objects, concepts, and words. It is where our k nowledge about the world is stored and allows us to express and comprehend communication, as well as use objects in our environm ent to perform daily tasks (Hart, et al., 2007) Because semantic memory is an integral part of daily life, the study of semantic representation has long been a part of cogn itive -linguistic research. While much of this research has been dedicated to understanding the contribution of the perceptual system ( e.g., visual features) to semantic representation, there is increasing interest in contributions of the motor system. In p art, the motivation for this line of research arises from a long-standing debate over how semantic representations are processed and stored in the brain. One class of theories suggests that semantic memory consists of abstract propositional representations that are mediated by central brain regions (i.e., amodal theories). Another class of theories suggests that object knowledge is comprised of perceptual and/or motor features distributed across modality -specific brain regions (i.e., sensorimotor theories). Embodied cognition is an emerging framework for understanding sensorimotor representations in the human brain. Theories of embodied cognition propose that concepts are grounded in ones interaction with the world. For example, an embodied approach to conc ept representation holds that a word such as write has features instantiated in the motor cortex as a result of t he experience of observing or enacting the use of the hand to manipulate an implement during the process of writing. T his motor representation is linked to language areas that store the lexical -semantic representation of write. This suggests there is a functional interaction between
13 language and motor systems for concepts involving human motor action (i.e., semantic motor representations). A vast amount of evidence from neurophysiological and neuropsychological studies supports sensorimotor theories of semantic representation as well as theories of embodied cognition. And while many of these studies have demonstrated involvement of the motor cortex in semantic representation, research into how semantic motor representations affect motor and language production remains limited in scope. That is, much of the research focuses on activation of brain regions in response to perception or production of motor related words and sentences with less attention to the nature of the interaction between language and motor systems during behavioral production tasks. To our knowledge, the effects of semantic -motor representations on continuous language and moto r production tasks performed simultaneously have not been investigated. Additionally, the reciprocality of these effects has not been investigated in sequential paradigms in which there is no overlap in task production. Thus, the purpose of our study wa s t o determine the reciprocal effects of shared semantic -motor representations on concurrent and sequential language and motor production in healthy young adults Overview The remainder of this c hapter provides a review and analysis of the literature related to theories of semantic representation, as well as evidence for functional interaction of language and motor systems. The notion of reciprocal activation of language and motor systems is introduced leading into the purpose of the study and related predicti ons. Chapter 2 provides a description of the stimuli standardization for our study. Chapter 3 provides information on the individuals who participated in our study. Chapters 46 outline the methods and results, and provide an interim discussion
14 of the find ings, for Experiments 13, respectively. Finally, Chapter 7 provides a general discussion of the findings, highlights implications of the results, and proposes considerations for future research. Review of the Literature Theories of Semantic Representation There exists a longstanding debate about the nature of semantic memor y with respect to the structure of semantic representations and their interdependence on sensorimotor brain regions (Gainotti, 2006) The relationship between sensorimotor processes and language is the prim ary focus of our study As such, arguments about the relationship between concepts and words, and theories regarding the role of words in creating a conceptual repertoire, are beyond the scope of th e current s tudy and the empirical studies reviewed in this chapter Thus, we use t he term semantic representation to denote the mental representations underlying both concepts and words Currently, two broad classes of theories about semantic representation predominate. One theory holds that semantic representations are amodal (Coccia, Bartolini, Luzzi, Provinciali, & Lambon Ralph, 2004) Amodal t heories hold that concepts are represented as abstract propositional representations whose storage is not critically dependent upon sensory regions involved in perception ( e.g. ones knowledge of a dog does not necessarily demand access to visual cortices ). Perhaps the most prominent amodal theory has proposed that s emantic knowledge organized in a central hub located in the anterior temporal pole (Patterson, Nestor, & Rogers, 2007; Rogers, Hocking, et al., 2006) These authors suggest that the anterior temporal lobe is responsible for similarity relations between all categories of knowledge. Another theory
15 by Damasio and colleagues suggests that the anterior temporal lobe is a convergence zone critical for processing unique entities (H. Damasio, Grabowski, Tranel, Hichwa, & Damasio, 1996) According to amodal theories activation of semantic representations during cognitive tasks is not dependent on the storage and retrieval of all attributes, facts and propositions related to the concept. Rather, it is only necessary that the information relevant to the task at hand b e available for the required response (Rogers, et al., 2004) As such, in a picture-naming task the visual attributes not attributes associated with other modalities (e.g., auditory attributes), would be integrated with abstract knowledg e for successful naming. Evidence in support of amodal theories is derived from computation models (Rogers, et al., 2004) as well as neuropsychological profiles associated with Alzheimers disease and semantic dementia (Coccia, et al., 2004; Patterson, et al., 2006; Rogers, Hocking, et al., 2006; Rogers, Ivanoiu, Patterson, & Hodges, 2006) However, a number of arguments have been raised against the existence of abstract representations of object knowledge in the brain. One of the strongest argu ments against amodal theories is the finding that perception and action interact with cognition (Barsalo u, 2008; Barsalou, Simmons, Barbey, & Wilson, 2003) An alternative class of theories, referred to as nonpropositional sensorimotor theori es seek to account for this interaction. Sensorimotor theories hold that semantic representations are stored accordin g to their sensorimotor features. That is, the sensory and motor features of concepts are stored in modality -specific regions distributed across the brain. For example, when you see the word dog, it activates a distributed semantic representation rich wi th
16 information about how a dog smells (olfactory), the sound of its bark (auditory), common physical attributes (visual), how it may act in particular situations (predictive) and factual knowledge (semantic), as well as personal feelings about dogs (affect ive). Critically, fully distributed views of semantic memory hold that the brain represents object knowledge through co activation of many such distributed features without the inherent need for a central organizing region. The following sections review ev idence from neurophysiological and neuropsychological studies supporting nonpropositional sensorimotor theories Evidence for Sensorimotor Theories Perceptual Representation Investigations of various features related to objects (nouns) and actions (verbs) have contributed a great deal to the understanding of the distribution of sensorimotor features across cortex. For example, visual features of objects (i.e., shape, color, and motion) are represented in distinct cortical regions. Several studies on the generation of words associated with color have shown activation in the mid -fusiform gyrus, a region associated with the perception of size and color (Kellenbach, Brett, & Patterson, 2001; Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995; Simmons, et al., 2007) Visual motion also has distinct cortical representation as evidenced by activation of the left ventral premotor cortex and middle temporal gyrus (area MT+/V5) during observation and naming of tools and other m otion-related words (Beauchamp, Lee, Haxby, & Martin, 2002, 2003; Chao, Haxby, & Martin, 1999; Chao & Martin, 2000; H. Damasio, et al., 2001; Grafton, Fadiga, Arbib, & Rizzolatti, 1997; Kable, Lease-Spellmeyer, & Cha tterjee, 2002; Martin, et al., 1995; Perani, et al., 1995; Tyler, et al., 2003) Activation in these regions is presumed to reflect the retrieval of action knowledge related to objects associated with motion and manipulation.
17 In addition to visual feature s, other perceptual features of objects, such as taste, smell, and sound appear to have distinct cortical representation. For example, it has been demonstrated that gustatory processing areas are not only active when food is consumed but also when pi ctures of appetizing food are presented, s uggesting that gustatory processing occurs in modality -specific cortical regions (Simmons, Martin, & Barsalou, 2005) Activation of the primary olfactory cortex, responsible for odor perception, has been demonstrated in studies of odor imagery (Bensafi, Sobel, & Khan, 2007; Djordjevic, Zatorre, Petrides, Boyle, & Jones -Gotman, 2005) and processing of linguistic odor -related stimuli (Gonzalez, et al., 2006) Activation of left posterior superior temporal gyrus and adjacent parietal cortex has been associated with making judgments of sounds related to picture stimuli, suggesting that visual stimuli may recruit modality -specific regions for auditory processing (Kellenbach, et al., 2001) Evidence for Sensorimotor Theories Motor Representation Increasing interest in semantic motor concept representation has yielded a large body of evidence from neurophysiological investigations of action verbs (i.e., actions or events). This evidence is supported by neuropsychological studies of individuals with focal brain damage. The next two sections review c onverging evidence from perception and production studies demonstrating that semantic -motor concepts are at least partially represented in motor cortices. Neurophysiological evidence from perception studies A line of researc h that provides tremendous support for the existence of semantic motor representations is the work of Pulvermuller and colleagues investigating the cortical representation of action words. Results of two electroencephalogram (EEG) studies suggest that lexi cal decision for arm -, leg-, and face -related verbs results in
18 somatotopic activation of corresponding regions along the motor strip at approximately 250 milliseconds (ms ) after word onset (Hauk & Pulvermuller, 2004; Pulvermuller, Harle, & Hummel, 2001) A subsequent magnetoencephalography (MEG) study also demonstrated activation of somatotopic representations in response to acoustic stimuli (Pulvermuller, Shtyrov, & Ilmoniemi, 2005) Similarly, fu nctional magnetic resonance imaging (fMRI) studies have shown that semantic judgments about hitting (e.g., knock, pound, strike) and cutting ( e.g. slash mince, slic e ) verbs activate the arm and hand areas of the primary motor cortex respectively (Kemmerer, Castillo, Talavage, Patterson, & Wiley, 2008) Other studies have extended this research beyond the category of verbs to investigate semantic representation in other grammatical classes. I n an event -related potential (ERP) study compa ring lexical decision for action nouns and visual nouns with action verbs in the German language, Pulvermuller and colleagues showed that visual nouns and action nouns differed topographically, while action nouns and verbs demonstrated a similar topography (Pulvermuller, Mohr, & Schleichert, 1999) Similarly, Vigliocco and colleagues used positron emission tomography (PET) to show that processing of motor -related words activated left precentral gyrus, while processing of visual words activated left inferior temporal and inferior frontal regions (Vigliocco, et al., 2006) These results suggest that cortical activation from conceptual processing is related to the sensorimotor features of a word (and the degree to which the word is sensor y or motor -related). Neurophysiological evidence from production studies A number of studies have also investigated semantic motor representation in motor cortices using word production paradigms. Hauk and colleagues demonstrated activation of primary mot or cortex for production of action words related to the arms and
19 legs, as well as activation of premotor cortex for words related to the arms and face, providing support for somatotopic organization of motor -related words (Hauk, Johnsrude, & Pulvermuller, 2004) Additionally, fMRI has been utilized to investigate activation of motor areas during generative naming tasks. Vitali and colleagues demonstrated activation in inferior prefrontal and premotor cortex during covert production of tools relative to animals in a word fluency paradigm (Vitali, et al., 2005) Recently, Esopenko and colleagues demonstrated a similar pattern of activation in a nounverb association task requiring overt production of verbs (Esopenko, Borowsky, Cummine, & Sarty, 2008) Several neurophysiological studies have also provided support for the importance of sensorimotor features, rather than grammatical class, in semantic representation. Oliveri and colleagues used TMS to demonstrate that left motor cortex was activated in a spoken word production task involving action -related nouns and verbs (Oliveri, et al., 2004) A subsequent study by Saccumen and colleagues provided further evidence with fMRI results demonstrating activation of similar cortical networks during naming of manipulable objects and act ions (Saccuman, et al., 2006) Together these neurophysiological studies show that primary motor cortex and premotor cortex, which work together during planning and execution of actions, are active during covert semantic processing and overt generation of motor -related words. This modality -spe cific activation in motor cortices lends further support to sensorimotor theories of semantic representation. Neuropsychological e vidence In addition to the evidence provided by neurophysiological studies, there is support for sensorimotor theories from neuropsychological studies of individuals with acquired
20 brain damage. Because stroke results in focal damage, the profile of cognitive-linguistic deficits associated with aphasia following circumscribed damage of left hemisphere and the association with the se deficits to corresponding brain regions, provide valuable insights into the organization of semantic representation. Perhaps the most convincing evidence in this respect comes from nounverb dissociations and category -specific naming deficits. The corti cal representation of nouns (i.e., objects) and verbs (i.e., actions or events) has long been investigated in studies that use aphasia classification and/or lesion localization to look at functional brainbehavior relationships Early classification studies showed that individuals with nonfluent aphasia exhibit verb impairments, while individuals with fluent aphasia have intact verb processing (or the impairment is observed to a lesser degree than nouns). A number of studies confirm the finding of verb impairments in agrammatic Brocas aphasia (Kim & Thompson, 2000; Miceli, Silveri, Villa, & Caramazza, 1984) Importantly, Arevalo and colleagues showed that verb naming in Brocas aphasia was significantly more difficul t than noun naming, but when the stimuli were categorized according to manipulability, performance was worse for manipulable objects (Arevalo, et al., 2007) This manipulability effect suggests that some verb naming deficits may be attributed to the sensorimotor component of words rather than grammatical class or noun verb differences in semantic complexity However there is some evidence suggesting that individuals with fluent aphasia exhibit verb impairments (Berndt, Mitchum, Haendiges, & Sandson, 1997) but it has been demonstrated that the degree of production and comprehension impairment may be dependent on th e task and stimuli (McCann & Edwards, 2002)
21 Lesion mapping studies have shown that damage to a number of regions can affect verb retrieval. Damasio and Tranel (1993) reported an individual with a lesion in the left premotor cortex who exhibited impaired verb production with relative preservation of noun processing This finding was later supported by other studies demonstrating that left premotor/prefrontal/frontal cortex damage results in naming deficits wo rse for verbs than for nouns while temporal lesions result in naming deficits that are worse for nouns than for verbs (Daniele, Giustolisi, Silveri, Colosimo, & Gainotti, 1994; Miozzo, Soardi, & Cappa, 1994; Shapir o & Caramazza, 2003; Tranel, Adolphs, Damasio, & Damasio, 2001; Tranel, Kemmerer, Adolphs, Damasio, & Damasio, 2003) However, studies have also shown contrasting results (Caramazza & Hillis, 1991; Shapiro, Shelton, & Caramazza, 2000) Thus, a clean dissociation between frontal versus temporal lobe damage and nounverb dissociations has remained elusive. Category -specific deficits found in neuropsychological populations also provide e vidence for storage of semantic features in anatomically distinct brain regions. This distinction is often made along the lines of living and nonliving entities. For example, damage to visual areas and motor areas may cause a deficit for recognition and naming of animals and tools, resp ectively (H. Damasio, et al., 1996) Presumably, these deficits result from the damag e to regions that are dominant in the processing of objects within these categories. That is, the visual system is the dominant modality for processing animals and other biological natural kinds, wh e reas the motor system is the dominant modality for proces sing tools and other manufactured artifacts (Gainotti, Silveri, Daniele, & Giustolisi, 1995; Humphreys & Forde, 2001; Warrington & Shallice, 1984) Additionally, there is evidence that motion properties are further fractionated such that
22 biological motion is functionally segregated from mechanical motion (Ma rtin, 2007) This suggests that sensorimotor experiences involving human motor action are differentiated from other sensorimotor experiences. Embodied Cognition A framework for understanding how sensorimotor features are instantiated in corresponding brai n regions is provided by theories of embodied cognition. The notion of embodied cognition was first described by Allport (1985) who stated: The essential idea is that the same neural elements that are involved in coding the sensory attributes of a (possible unknown) object presented to eye or hand or ear also make up the elements of the autoassociated activity -patterns that represent familiar object -concepts in semantic memory. This model is, of course, in radical opposition to the view, appare ntly held by many psychologists, that semantic memory is represented in some abstract, modality independent, conceptual domain remote from the mechanisms of perception and motor organization (p. 53) Thus, Allport proposed that conceptual knowledge was based sensory and motor modalities and that information in these modalities was processed in distinct ways As such, t he overarching theme across embodied cognition theories is that concepts are grounded in sensorimotor systems through ones bodily intera ctions with the environment. However, there are varying degrees to which these theories implicate sensorimotor systems in semantic representation. For example, Gallese and Lakoff (2005) propose the extreme point of view that sensory -motor representations are at the core of all cognitive operations, and as such, the ability to recognize and understand the use of objects depends on ones ability to produce the objects associated action A more moderate view proposed by Barsalou and colleagues is that individuals sele ctively attend to specific perceptual features (i.e., shape, color, sound, smell, movement, emotion) during real wor l d interactions with objects, resulting in patterns of activation in
23 the sensorimotor cortex. These perceptual representations can later be partially reactivated during cognitive tasks through attention and memory integration (Barsalou, 1999, 2008; Barsalou, et al., 2003) Others have suggested a similar theory in which the sensory motor system is the basis for organization and representation of knowledge of actions and objects but motor production processes are not required for successful recognition and comprehension of objects and their use (Mahon & Caramazza, 2005; Martin, 2007; Martin & Chao, 2001) Functional Inte raction of Language and Motor Systems The functional interaction between language and motor systems can be seen in neurophysiological as well as neuropsychological studies of perception, production, observation and imagery. The relationship between languag e and motor production tasks has, for the most part, been approached from the language to motor perspective (i.e., when a language task precedes a motor task) with a great deal of studies focusing on the effects of language perception on motor production However, less is known about the effects of semantic motor representations in language to motor tasks requiring overt language production. R elatively fewer studies have investigated the relationship between these two systems from a motor to language p erspective where a motor task precedes a language task The available evidence related to effects of language processing on motor systems and motor processing on language systems are reviewed in turn. Language effects on the motor s ystem Boulenger and col leagues (2006) used a behavioral paradigm to investigate the effects of verb processing on a reaching task. They found i nterference during the first 160-180 ms after word onset when the language and motor task s were concurrent and
24 facilitation when the wor d appeared before the reaching task (Boulenger, et al., 2006) A later study involving EEG and kinematic an alysis demonstr ated that when action verbs were displayed too quickly to be consciously perceived during movement preparation it interfered with the subsequent reaching movement (Boulenger, et al., 2008) These results provide further evidence that brain regions active during motor planning and execution are also active during language processing Pulvermuller and colleagues used transcranial magnetic stimulation (TMS) to investigate the nature of semantic -motor representations during lexical decision tasks. TMS applied somatotopically to the left motor cortex 150 ms after word onset resulted in faster reaction times on a button press task for words related to the stimulated region than words unrelated to the stimulated region (Pulvermuller, Hauk, Nikulin, & Ilmoniemi, 2005) A similar result was obtained in a TMS study demonstrating that hand motor cortex excitability was modulated by spoken production of action -related nouns and verbs relative to nonaction words (Oliveri, et al., 2004) With respect to effects of language on motor production across grammatical boundaries, Oliveri et als results confirm findings in a study demonstrating that production of concrete motor -related nouns interfered with motor production tasks (Morsella, 2002) Collectively, these studies provide evidence for language effects on the motor system, suggesting that areas responsible for motor prod uction are involved in language processing. Moreover, they show that these effects can be facilitative or inhibitory in nature, depending on the content and timing of verbal production. In sum, when the verbal cue is congruent with the action, and precedes motor production, facilitation occurs. Conversely, when the verbal cue is incongruent with the action, or is
25 presented simultaneously with the motor cue, interference may occur. However, the time course for facilitation or interference in sequential langu age to motor production tasks warrants further investigation. Motor effects on the language s ystem A fewer number of studies have studied motor effects on the language system. Setola and Reilly (2005) compared the influence of performed and observed movements on a lexical decision for verbs with visual associations vs. motor associations. Results showed faster reaction times for general action words (e.g., those with visual associations) and slower reaction times for hand act ion words (e.g., those with motor associations) indicating that observed or performed hand movements interfere with processing of hand-related words when these tasks were performed simultaneously. Nazir and colleagues (2008) also demonstrated that perception of action -related verbs interfered with a reaching task when words were presented 50 ms and 250 ms after initiation of the movement. Morsella (2002) investigated motor effects on the language system during simultaneous production tasks and found that motor production slowed production of concrete motor -related nouns relative to abstract words This evidence provides support for motor effects on language perception and production, which supports the notion that areas involved in motor pr oduction are also engage by language processing. Furthermore, it appears the influence is inhibitory for motor movements when tasks are completed concurrently and when words are p resented after movement onset. However, additional studies are warranted to e lucidate the effect of semantic -motor representations on production tasks, as well as time course for cue presentation.
26 Neuropsychological e vidence I ndividuals with aphasia and limb apraxia provide evidence for the interaction of language and motor systems as well. Limb apraxi a is a neuropsychological disorder of learned movement not resulting from primary motor or sensory deficits (Rothi & Heilman, 1997) More specifically, ideomotor limb apraxia, which can affect both pantomime and actual tool use, is thought to result from damage to parietal and prefrontal cortices that results in a disruption of gesture information and stored knowledge related to tool use that is required for successful motor output. Ideomotor limb apraxia and aphasia are common consequences of stroke, and it is reported that these disorders often cooccur (Kertesz & Hooper, 1982) Studies have s hown that individuals with posterior lesions demonstrate intact verbal and nonverbal action intention and action plans; however, their language is characterized by paraphasias and word retrieval deficits, which are analogous to deficits that individuals wi th ideomotor limb apraxia demonstrate (i.e., executing the wrong grip for a hand tool ) (Buxbaum, Sirigu, Schwartz, & Klatzky, 2003; Leiguarda & Marsden, 2000) In patients with frontal lesions, deficits in action plan generation affect the sequencing of words and actions, which some argue is analogous to grammatical deficits and difficulty performing actions that require sequential movement (van Schie, Toni, & Bekkering, 2006) Additionally, it has been suggested that the action impairment associated with limb apraxia prohibits successful use of gesture as a facilitative or compensatory strategy in individuals with aphasia (Kertesz & Hooper, 1982) H owever, evidence against this suggestion has been demonstrated in neuropsychological studies of individuals with limb apraxia (Haaland, 198 4; Rosci, Chiesa, Laiacona, & Capitani, 2003) and treatment studies of individuals with limb apraxia and aphasia (Raymer, et al., 2006; Rodriguez,
27 Raymer, & Rothi, 2006) suggesting the re is not a direct relationsh ip between deficits in action and language perception and production. Nonetheless there is evidence that disruption in language or action systems due to brain damage can result in negative functional consequences supporting the existence of some degree of anatomical overlap betw een language and motor systems. Reciprocal Activation of Language and Motor Systems A vast amount of evidence exists in support of a functional interaction between language and motor systems. Based on the review of evidence for sensorimotor theories and the interaction of language and motor systems, two conclusions can be drawn. First, the semantic representations of action -related words are functionally linked to the motor system. Second, similar functional networks support languag e and motor production. Specifically, production of motor -related words and manual goal directed behavior are served by portions of a common neural network in addition to the areas that process associated sensory features If actions and motor -related conc epts share some neural real estate, two predictions can be made with regard to the interaction between motor and language systems. First, activation of semantic -motor representations may reciprocally a ffect language and motor production. That is, activatio n of semantic motor representations may be sufficient to activate associated words and actions For example, when the tool used for pounding a nail is viewed the lexical representation hammer as well as the corresponding motor regions involved in the ac tion associated with hammer (i.e., hand and arm motor cortex for grasping and striking) are activated. The unique characteristic of semantic -motor representations, as opposed to semantic representations lacking a human motor component, is that the sensorim otor information can translate into
28 a ctivation of the motor system Thus, language and motor activation should co occur when the brain processes these semantic -motor representations or when actions are executed Language Production Evidence reviewed in thi s chapter suggests that word production tasks activate a wide range of cortical areas depending on the sensorimotor features of the underlying concept. As such, motor -related concepts have some degree of representation in the motor system. Evidence also su ggests that language production can influence motor production by way of facilitati on or interference. These effects are dependent on temporal aspects of production and the congruency of modality of target and response (e.g., hand actions affect hand words ). Motor Production This chapter also reviewed evidence regarding the effects of semantic -motor representations on motor to language production. The evidence in this direction is less substantive, suggesting that this phenomenon is more difficult to invest igate experimentally or that it may be more complex than the effects of semantic motor representations on language t o motor production. It can be argued that motor production activates semantic motor representations associated with the motor region creatin g competition for selection Evidence also suggests that language production can influence motor production by way of facilitati on or interference. Again, these effects are dependent on temporal aspects of production and the congruency of modality (e.g., h and actions affect hand words).
29 Purpose of the Investigation Based on the review of the literature, several hypotheses can be generated with regard to the interaction of language and motor system in the context of semantic motor representations: Motor engagement is sufficient to affect language production Language production is sufficient to engage the motor system The nature of language and motor system interaction is moderated by task and temporal dynamics While there is evidence for the interaction of l anguage and motor systems under varying conditions, the effects of semantic motor representations on concurrent, continuous language and motor production have not been investigated. Additionally, the reciprocality of language and motor effects in a sequent ial production paradigm has not been investigated. The investigation of language motor interactions during production is important for a number of reasons. First, understanding how language and motor systems interact during perception only accounts for par t of how we relate to others and our environment. Secondly, language and motor systems are dynamic and are likely engaged in unique ways during production (versus perception) tasks. Finally, a greater understanding of language motor interaction during production has implications for functioning of healthy individuals and those with neurologic disease. Thus, t he purpose of our study was to determine the reciprocal effects of shared semantic -motor representations on concurrent and sequential language and moto r production in healthy young adults We chose to conduct our study on healthy young adults because of age -related slowing in reaction time for motor responses (Falkenstein, Yordanova, & Kolev, 2006) differences in finger tapping abilities between young and old (Aoki & Fuk uoka, 2010) as well age -related psychomotor slowing
30 reported to affect verbal fluency and word production (Rodriguez Aranda, Waterloo, Sparr, & Sundet, 2006) The direction of effect was tested in three orthogonal experiments (see Figure 11) that tested t h e following predictions : Experiment 1 (Motor + Language) : Concurrent and continuous motor and language production will be differentially aff ected by semantic motor representations such that interference will occur when motor production is simultaneously a ccompanied by a language task that requires production of motor -related words relative to visually related words. Experiment 2 (Language : Sequential language to motor production will be differential ly a ffect ed by semantic motor representations such that production of m otor -related words relative to visually -related words, will interfere with a subsequent motor production task. Exp eriment 3 (Motor Language) : Sequential motor to language production will be differentially affected by semantic motor representations such that a motor production task will interfere with subsequent production of motor -related words.
31 Figure 11 The direction of effect investigated in Experiments 1 -3
32 CHAPTER 2 STIMULI STANDARDIZAT ION Participant s Ten healthy young adults ( n=2 males, n=8 females), age 1925 years, from the University of Florida participated in the stimuli standardization study. B y se lf -report, all participants were right handed, native speakers of English with no history of developmental or acquired cognitive or motor impairment. Approval from the Institutional Review Board was obtained and all participants signed an informed consent form prior to the initiation of study procedures. Semantic Category Selection Experiment 1 required selection of verbal fluency categories to examine the effect of semantic -motor representations on concurrent, continuous language and motor production. To e nsure relative equivalency among the categories a series of verbal fluency tasks was administered within two broad semantic conditions : Semantic -Motor (categories highly associated with human manipulation or human motor action) and Semantic -Other ( categori es not associated with human manipulation or human motor action ). Procedure Participants were given one minute and asked to name as many different words as possible that belonged to a given category. Four subcategories were tested in the Semantic Motor con dition: a) things you do; b) school and office supplies ; c) garage tools; d) musical instruments. Three subcategories were tested in the Semantic -Other condition: e) animals ; f) cities ; g) fruits and vegetables. Responses were digitally
33 recorded using a la ptop computer with an external microphone. Responses were later transcribed and scored offline. Results The recordings for each participant were reviewed to code errors in word production for each verbal fluency subcategory. The t wo types of errors we code d were category errors and perseverations. The mean for each subcategory was calculated based on the total number of correct words for all participants (see Table 2-1). Participants demonstrated heterogeneity in the number of words they produced across Semantic -Motor subcategories. The subcategories things you do and school and office supplies resulted in production of words not associated with human motor manipulation or motor action ( i.e. learning, desk). Additionally, the subcategory garage tools resulted in production of fewer words than the other two categories. Performance in all three subcategories under the Semantic -Other condition was equivalent. To improve performance and response consistency in the Semantic Motor subcategories, things you do was modified to things you do with your hands and the subcategories school and office supplies and garage tools were replaced with the category objects that require the use of your hands. A total of six subcategories were selected for use in Ex periment 1. The two modified subcategories, as well as musical instruments comprised the Semantic Motor condition. The three original subcategories (animals, cities, and fruits and vegetables) comprised the Semantic -Other condition. Word Selection As discu ssed previously, motor related nouns and action verbs have been shown to activate motor cortices in neurophysiological perception and production studies (Oliveri, et al., 2004; Pulvermuller, et al., 1999; Saccuman, e t al., 2006; Vigliocco, et al.,
34 2006) and language-motor overlap has also been demonstrated in behavioral studies investigating either motor related nouns (Morsella, 2002) or action verbs (Boulenger, et al., 2006; Boulenger, et al., 2008; Nazir, et al., 2008; Setola & Reilly, 2005) Motor related nouns and action verbs, h owever, have been inves tigated in separate experiments rather than within experiments where the psycholinguistic variables of the stimuli are held c onstant across word type. In order to extend the current research, two stimuli sets, comprised of motor -related (MR) nouns (e.g., p encil), MR verbs (e.g., write), visually -related (VR) nouns (e.g., flower) and VR verbs (e.g., bloom), were developed for Experiments 2 and 3. Procedure Nouns were extracted from the Wisconsin Perceptual Attribute Ratings Database which contains 1,402 nouns and their mean salience ratings in four perceptual attribute domains (sound, color, manipulability, and motion) (Medler, Arnoldussen, Binder, & Seidenberg, 2005) Nouns were selected according to the following criteria: MR and VR nouns we re 3-6 letters in length and no more than two syllables MR nouns were objects that required interaction with hands for use and had manipulability or motion ratings of greater than 2.5 (out of 6) VR nouns were objects not associated with human interaction and had color ratings of 3.0 or higher and manipulability ratings of 1.5 or lower (out of 6) The verb corpus was extracted from an English verb lexicon categorized by classes (Levin, 1993) MR and VR verbs in their base form (e.g., run, write) were selected according to the following criteria: MR and VR verbs were 36 letters in length and no more than two syllables MR verbs were from classes that require movement of the hands VR verbs were from classes not directly related to human body movement or human psychological states
35 The n ouns and verbs (n=165) were combined into randomized rating sheets. Participants were asked to rate each word on concreteness, familiar ity and dominant grammatical class Concreteness, defined as ability to see, touch, or manipulate, was rated on a 17 scale (1= not at all able, 7= extremely able). Familiarity was rated on a 17 scale (1= not at all familiar, 7= extremely familiar). Judgm ent of dominant grammatical class was based on whether the word was more strongly associated with use as a noun (i.e., object) or verb (i.e., action or event). See Appendix A for instructions provided to participants, as well as examples of each rating sca le. Results The mean ratings for familiarity, concreteness and grammatical class for each word were analyzed separately. The stimuli chosen for Experiments 2 and 3 fit the following criteria: Concreteness average above 3.5 for nouns and 2.0 for verbs Famil iarity average above 3.5 for all nouns and verbs Dominant grammatical class agreement in 9 of 10 rater s A total of 96 words were selected and subsequently divided into two balanced sets Set s 1 and 2 each contain 24 nouns (n=12 motor, n=12 visual) and 24 verbs (n=12 motor, n=12 visual) for a total of 48 words (see Table 2-2 ). These sets were counterbalanced across participants for use in Experiments 2 and 3. See Appendix B for the concreteness, familiarity, and grammatical class ratings of all 96 words in Sets 1 and 2.
36 Table 2 1. Results for verbal fluency subcategories Range Mean (SD) Semantic -Motor Things you do 1123 17.5 (3.4) School and office supplies 8 25 16.5 (5.2) Garage tools 3 10 7.0 (2.4) Musical instruments 1320 15.7 (2.4) Semantic -Other Animals 1429 21.6 (5.0) Cities 9 38 21.8 (8.5) Fruits and vegetables 1530 21.7 (3.8) Table 2 2 Mean c oncreteness and f amiliarity r atings for S ets 1 and 2 Set 1 Set 2 Concreteness Familiarit y Concreteness Familiarity Motor Nouns (n=12) 6.82 6.47 6.83 6.48 Verbs (n=12) 4.57 6.45 4.55 6.40 Visual Nouns (n=12) 6.81 6.59 6.82 6. 59 Verbs (n=12) 3.97 6.23 3.98 6. 23
37 CHAPTER 3 PARTICIPANTS FOR EXPERIMENTS 1 3 A convenience sample of 40 healthy young adults (n=5 males, n=35 females), age 1822 years from the University of Florida participated in our study .1 Approval from the Institutional Review Board was obtained and all participants signed an informed consent form prior to the i nitiation of study procedures. Inclusion and Exclusion Criteria Two screening procedures were administered to determine eligibility to participate: A brief questionnaire that included questions about native language and history of developmental or acquired cognitive or motor impairment was administered. Individuals with a history of developmental or acquired disorders affecting cognitive and/or motor function were not included in the study due to possible differences in performance on the language and motor tasks. The Edinburgh Handedness Inventory (Oldfield, 1971) was administered to determine dominance of the right or left hand in everyday tasks. Individuals who were left hand dominant or ambidextrous, as indicated by scores of 40 or below or between 40 and +40, respectively, were excluded from participation. Onl y those who were right -hand dominant, as indicated by a score of +40, were included. Randomization All 40 participants completed three experiments individually, (n=20 with the left hand, n=20 with the right hand) Experiment 1 was the first experiment adm inistered to each participant. The order of administration for Experiments 2 and 3 was coun terbalanced across participants (n=20 completed Expe riments 1, 2, then 3, n=20 1 We acknowledge the underrepresentation of males in our sample. However, it is unclear whether an unequal gender distribution affect s our findings. Results of studies on gender differences in finger tapping are mixed. Studies investigating single task tapping have demonstrated that males tap with greater speed and regularity ( Schmidt, Oliveira, Krahe, & Filgueiras, 2000) while studies investigating concurrent tapping and verbal production report no difference in number of taps between males and females (Clark, Guitar, & Hoffman, 1985) Similarly, significant gender differences in strategies for completing verbal fluency tasks have demonstrated superiority in females for phonemic fluency tasks only (Weiss, et al., 2006) Additionally, the categories selected for Ex periment 1 were gender neutral.
38 completed Experiments 1, 3, then 2). Control conditions for Experiments 2 and 3 were a lso conducted in a counterbalanced fashion across participants (n=20 completed the control trials prior to Experiments 2 and 3, n=20 completed the control trials after the E xperiments 2 and 3).
39 CHAPTER 4 EXPERIMENT1: EFFECT OF SEMANTIC -MOTOR REPRESENTATIO NS ON CONCURRENT, CONTINUO US LANGUAGE AND MOTO R PRODUCTION The effect of semantic motor representations on concurrent language and motor production were examined in the context of continuous production tasks. Although verbal manual dual task paradigms are common, they have not been utilized in the investigation of language motor interactions Our verbal manual dual task paradigm required rapid, continuous, internally generated language production (i.e., verbal fluency) and rapid, continuous motor production (i.e., finger tapping) Because both language and motor systems we re engaged simultaneously, this paradigm allow ed for investigation of the direct effects of semantic motor representations on language and motor production. The two variables measured in Ex periment 1 were number of taps and number of words produced. Procedure Experiment 1 was conducted on a laptop computer with a USB enabled button box and a Marantz digital recorder with a headset microphone. Participants were asked to rest their hand comfo rtably on the button box. The experiment included five conditions: one Baseline Motor condition, two Baseline Language conditions and two Motor + Language conditions (see Table 4 1). The Baseline Motor and Baseline Language conditions allowed for comparison of tapping and word production under single and concurrent (Motor + Language) task conditions, allowing for more precise measurement of the direct effects of semantic motor representations on concurrent language and motor production. In the Baseline Moto r c ondition, participants were asked to press one button on a button box as quickly and steadily as possible for one minute In the Baseline Language conditions,
40 participants were asked to generate as many different words as possible for a given category i n one minute. In the Motor + Language conditions participants were asked to generate as many different words as possible for a given category in one minute while also tapping as quickly and steadily as possible. Experiment 1 began with the Baseline Motor condition to give participants an opportunity to become familiar with finger tapping. The Baseline Language and Motor + Language c onditions were pseudorandomized across participants to rule out order effects and reduce the possibility of motor fatigue. Add itionally, the three subcategories under Semantic -Motor (i.e., things you do with your hands, objects that require the use of your hands, musical instruments) and Semantic -Other (i.e., fruits and vegetables, animals, cities) were pseudorandomized across th e Baseline Language and Language + Motor conditions to rule out experimental bias. Data Acquisition and Processing The variables of interest in this experiment were tapping rate and number of words produced per category. The button box captured tapping dat a every 100 ms Word production was recorded via the Marantz digital recorder in order to complete offline coding and to calculate number of responses. The recordings for each participant were transcribed and each category was coded in 100 ms intervals. Th e coding scheme was as follows: 0= silence, 1= semantic processing (500 ms prior to word production), 2= speaking. Responses were also coded for accuracy with errors labeled as category errors or perseverations. Tapping data per 100 ms were time-locked to word production data for the Motor + Language conditions. The method for processing the d ata for the five conditions is described below.
41 Baseline Motor (Condition 1) : The t otal number of taps in 60 seconds as well as taps per second, was calculated for ea ch participant. Baseline Language (Conditions 2 and 3) : The t otal number of words in 60 seconds was calculated separately for the Semantic Motor and Semantic -Other baseline categories for each participant. Time on task (in seconds), based on time spent sem antic processing (code 1) and speaking (code 2), was also calculated. Motor + Language (Conditions 4 and 5) : The t otal number of taps and words in 60 seconds was calculated separately for the Semantic -Motor and Semantic -Other subcategories for each par ticipant Time on task (in seconds) was also calculated for each subcategory. Means for tapping and word production were calculated for each part icipant by averaging the totals for both subcategories under Semantic -Motor and Semantic -Other conditions. Base d on these calculations, the following proportions were derived separately for Semantic Motor and Semantic -Other conditions These proportions were the dependent variables in the statistical analyses for Experiment 1: T apping proportions for Time on T ask: These proportions were calculated for each participant by dividing the mean number of taps for time on task during the Motor + Language conditions by the mean number of taps durin g the Baseline Motor condition. Word production proportions for Time on T ask: These proportions were calculated for each participant by dividing the mean number of words in time on task during the Motor + Language conditions by the mean number of word s during the Baseline Language condition.
42 Tapping proportions for T otal T ask Output: These proportions were calculated for each participant by dividing the mean number of taps produced in 60 seconds during the Motor + Language conditions by the mean number of taps produced in 60 seconds durin g the Baseline Motor condition. Word production proportions for T otal T ask Output : These proportions were calculated for each participant by dividing the mean number of words produced in 60 seconds during the Motor + Language conditions by the mean number of words produced in 60 seconds durin g the Ba seline Motor condition. Calculation of the se proportions allowed for a standardized comparison of change in tapping and word production from Baseline conditions to Motor + Language conditions eliminating any differences related to task difficulty (e.g., greater difficulty thinking of words for the Semantic Motor categories) Additionally, the proportions made it possible to determine the effects of dual task performance on finger tapping and word production in the Semantic -Motor and Semantic -Other conditio ns. Statistical Analyses and Results Errors accounted for less than 2% of total responses across semantic categories in the Baseline Language and Motor + Language conditions. The Semantic Motor condition had a higher error rate (2.4%) than the Semantic -Oth er condition (1.2%). However, most of the errors across conditions were perseverations (1.3%) rather than category errors, suggesting that very few were due to errors in semantic processing. Additionally, because the proportion of errors was relatively small it did not influence the results of our statistical analyses. As such, all words produced by participants were included in the word production proportions. This posed less of a threat than eliminating the errors, as there was not an equivalent method fo r systematic elimination of
43 corresponding taps. See Table 4-2 for the mean number of taps and words produced for each of the three subcategor ies that comprise Semantic -Motor and Semantic -Other conditions Time on Task Analyses To examine the dual task effe cts of tapping and word production under the two semantic conditions, a 2 C ondition (Semantic Motor, Semantic -Other) by 2 Task (taps, w ords) repeated measures multivariate analysis of variance ( M ANOVA) was conducted. The MANOVA re vealed a significant main effect of C ondition [F(1, 3 9 ) = 5.18, p = .028, 117] indicating th at dual task effects in the Semantic -Other condition were greater than dual task effects in the S emantic Motor condition when considering time spent speaking and tapping (silences excluded) No main effect of Task was found [F(1, 3 9 ) = .372, p = .546009] nor was there a Condition by Task interaction [F(1, 3 9 ) = .865, p = 358 0 22 ] Time on task r atio means and standard deviations are reported in Table 4-3 A one sample t test was used to determine if the variables were significantly greater than or less than one, which woul d indicate a significant effect of dual task performance for tapping or word production in each semantic condition The t -test wa s conducted at alpha level .01 based on a Bonferroni correction for multiple comparisons. The t -test revealed that task performance was not significantly different from baseline in the Semantic -Motor condition for tapping [ t (3 9 )= 1.33 p =. 191] or word production [ t (3 9 )= .872, p =. 389 ] However, the Semantic -Other condition revealed a marginally significant difference from baseline performance for tapping [ t (3 9 )= 2.48, p =.018] and a significant difference from baseline performance for word production [ t (3 9 )= 3.83, p < 00 01 ] This indicates there was a greater dual task cost in the Semantic -Other
44 condition relative to the Semantic Mot o r condition in both finger tapping and word production (see Figure 41) Total Task Output Analyses To examine the dual task effects of tapping and word production under the two semantic conditions, a 2 Condition (Semantic Motor, Semantic -Other) by 2 Task (taps, words) repeated measures multivariate analysis of variance (MANOVA) was conducted. The MANOVA re vealed a significant main effe ct of Task [F(1, 39 ) = 5.83 p = .021, 130] indicating that when considering change in total output participants demonstrated a dual task effect only o n tapping performance. No main effect of Condition was found [F (1, 39) = 1.92, p = 174 47], nor was there a Condi tion by Task interaction [F(1, 39) = 311, p = 580 08]. In summary, in dual task conditions, tapping dropped regardless of word type being produced, but the number of word produced was unaffected by the dual task manipulation. Total Task Ou tput ratio means and standard deviations are reported in Table 44. A one sample t test was used to determine if changes in tapping and word production were significantly greater than or less than one, which would indicate a significant effect of dual task performance for tapping or word production in each semantic condition. The t -test was conducted at alpha level .01 based on a Bonferroni correction for multiple comparisons. T apping performance was affected in the Semantic Other condition [ t (39) = 4.14, p < .0001] but not in the Semantic -Motor condition[ t (39) = 2.32, p= .026] indicating there was significant drop in tapping rate only in the Semantic Other condition Changes in word production were not significant in either the Semantic Other condition [ t (3 9) = .062, p = .951], or the Semantic Motor condition [ t (39) = 1.32,
45 p= .196] In sum, there were significant dual task effects on tapping which were greatest in the Semantic -Other condition, but no significant dual task effect on word production. Interim Discussion This experiment investigated the effect of semantic -motor representations on concurrent, continuous language and motor production. Participants generated words to Semantic Motor and Semantic -Other categories while finger -tapping. We found that c ontinuous, concurrent tapping and word generation interfered with the time spent tapping and producing words in the Semantic -Other condition relative to Semantic Motor condition. In addition, when looking at change in the actual output (taps and words) onl y tapping showed a significant dual task effect. Verbal manual dual task effects are widely reported in the literature. A common finding when pairing verbal output (e.g., word reading) with manual output (e.g., finger tapping) is a reduction in performance in one or both tasks (Van Hoof & Van Strien, 1997) As such, a reduction in word production and or tapping performance was expected in both conditions. However, we predicted there would be significantly more interference when tapping was accompanied by production of words in Semantic Motor categories which was not supported by our findings. We discuss the finger tapping and word production results in turn. Finger -Tapping Rate We found significant effects of dual task performance on time spent tapping (i.e., more pauses) in the Semantic -Other condition and on overall tapping output (i.e., fewer taps), which was most pronounced in the Semantic -Other condition. The Semantic Other condition was included to demonstrate the effects of tapping when words unrelated to motor performance were generated; thus, these results are consistent with
46 the literature that word production interferes with motor performance. In contrast, and contrary to our prediction, time spent tapping in the Semantic -Motor Condition was preserved relative to the Semantic -Other condition, and the number of taps produced showed no dual task effect in the Semantic -Motor cond ition. These findings suggest that producing Semantic -Motor words facilitated tapping relative t o the Semantic -Other condition. A potential alternate explanation for these findings is that participants chose to prioritize the cognitive task (i.e., focused more on word production rather than tapping) in the Semantic -Other condition resulting in a significant decrease in tapping performance; however, this possibility is unlikely because there was a significant decrease in time spent in word production during time on task in the Semantic -Other condition. The other alternative is that producing a larger number of words in the Semantic -Other condition relative to the Semantic -Motor condition resulted in the significant decrease in finger tapping in the former; however, differences in difficulty between conditions were accounted for in the tapping and word production proportions used in our analyses. Thus, it appears more likely that there was facilitation of finger tapping in the Semantic -Motor condition rather th an interference in the Semantic -Other condition. This explanation will be further explored in Chapter 7. Word Production Rate We found a significant effect of dual task performance on time spent in word production in the Semantic -Other condition, consistent with previous findings of verbal manual interference in the literature. However, contrary to our prediction, there was no significant effect of dual task performance on time spent in word production in the Semantic Motor Condition. Indeed, the dual task effect in the Semantic -Motor condition
47 was significantly less than the dual task effect in the Semantic -Other condition This provides additional evidence that there was facilitation of language and motor systems in the Semantic Motor condition. Again, i t might be suggested that the Semantic -Motor condition required greater cognitive effort, so participants chose to focus more on word production in that condition. However, increased cognitive demand resulting in greater attention to word production would h ave likely resulted in decreased tapping performance and/or decreased word production in the Semantic Motor condition, whic h is not what we found Thus, we believe the most parsimonious explanation of these findings is that the Semantic Motor condition fac ilitated performance in this dual task experiment. In summary, al though the findings of interference in the Semantic -Other condition for T ime on T ask and T otal T ask Output we re not consistent with our prediction, semantic -motor representations appear to have influenced language and motor production. Specifically, the Semantic -Other condition interfered with production, while the Semantic -Motor condition did not T herefore, t h e findings from the Time on Task and Total Task Output analyses suggest that concur rent activation of semantic -motor representations from tapping and word production facilitated p erformance in the Semantic Motor condition. This finding provides support for the hypotheses that language production is sufficient to engage motor production and vice versa. T he implications of this experiment will be discussed further Chapter 7.
48 Table 4 1 Five c onditions that c omprise Experiment 1 Baseline Motor Baseline Language Motor + Language Tapping Semantic Motor (1 category) Tapping + Semantic Motor (2 categories) Semantic -Other (1category) Tapping + Semantic -Other (2 categories) Table 4 2. Word production and f inger tapping means by s emantic c ategory Baseline Motor + Language Mean SD Mean SD Taps 218.93 35.72 Semantic -Motor Things you do with your hands 213.04 43.27 Musical instruments 206.6 41.59 Objects 213.83 30.68 Total Semantic -Motor 210.86 38.51 Semantic -Other Animals 191.96 48.34 Cities 206.15 30.83 Fruits and vegetables 206.81 51.74 Total Semantic -Other 201.40 39.28 Words Semantic -Motor Things you do with your hands 14.57 3.55 14.65 3.90 Musical instruments 14.40 5.50 16.5 4.64 Objects require use of hands 18.25 5.31 17.04 5.09 Total Semantic -Motor 16.00 5.05 16.06 3.88 Semantic -Other Animals 24.75 5.83 25.04 6.43 Cities 26.14 10.44 21.08 6.32 Fruits and vegetables 21.71 5.76 20.77 5.52 Total Semantic -Other 24.18 7.79 22.36 5.32
49 Table 4 3 Time on T ask ratios by Condition and Task Taps Words Mean SD Mean SD Semantic Motor .977 10 7 .979 156 Semantic -Other .953 120 .923 128 Table 4 4 Total Task Output ratios by C ondition and T ask Taps Words Mean SD Mean SD Semantic Motor .9 64 0 98 1 11 509 Semantic -Other .9 22 118 1.00 367 Figure 41. Depiction of dual task effects on tapping and word production during t ime on task by semantic condition.
50 CHAPTER 5 EXPRERIMENT 2: EF FECTOF SEMANTIC -MOTOR REPRESENTATION S ON SEQUENTIAL LANGUAGE TO MOTOR PRODUCTION In this experiment, t he effect of semantic -motor representation on language to motor production was examined using a semantic prime to influence motor production. Semantic pri ming is a phenomenon in which exposure to information that is semantically related to a target influences the speed and/or accuracy of a target response Recently studies have demonstrated that motor features of objects can serve as semantic prime s For ex ample, Myung and colleagues demonstrated that a word such as typewriter could be used to prime the word piano based on similar manner of manipulability (i.e., both require similar manual manipulation) (Myung, Blumstein, & Sedivy, 2006) The p aradigm used in this experiment t ook a slightly different but related approach in which semantic -motor word prime s were used to prime motor production. Th is paradigm, also used by M orsella (2002) is based on th e underlying assumption that word primes associated with hand motor function w ould act ivate the motor system and influence motor production. This experiment investigated the effect of MR and VR words on a goal directed manual task (i.e., pointing to a fixed target ) The dependent variable in Experiment 2 was reaction time (RT) for pointing. Experimental Procedure Experiment 2 was conducted on a desktop computer running E Prime stimulus delivery software with a touch screen monitor. Participants were seated in front of the computer monitor and asked to place their hand on a secured mouse pad located within comfortable reach. The participants were instructed to read each word aloud as quickly and accurately as possible and upon presentation of the cue (green light) to touch the circle on the screen with their index finger as quickly as possible then return their hand
51 to the mouse pad. Prior to the experiment, participants were informed that they may see a word appear in purple at the end of some trials. If the word appeared, they were to indicate whether that word was associated with the previou s target word. The purpose of th e word association task was to promote deeper semantic processing of target words, rather than simple grapheme to phoneme conversion. After the instructions were provided, participants were given practice trials to familiari ze themselves with initiating responses on cue as quickly as possible and making adequate contact with the touch screen monitor to record responses. Feedback was provided on trials in which participants did not complete the task correctly. The structure of the practice trials was analogous to the experimental trials; however, the words used in the practice trials were not part of the experimental stimuli. Experimental Trial Structure After the 500 ms fixation cross, t he semantic prime cue appeared for 300 m s. The ISI for the cue (green circle) to execute the finger pointing task occurred at fixed, random intervals of 250 ms, 500 ms, and 70 0 ms. The cue remained on the screen until the participant executed the pointing task. On trials that contained the word association task, a word appeared 10 00 ms after completion of the pointing task. E -Prime presented all trials automatically with an intertrial interval of 1500 ms. See Figure 5 1 for a depiction of the experimental trial structure. Experiment 2 consisted o f 96 trials (n= 24 MR words, n=24 VR words n=48 nonwords) See Appendix D for a description of nonword stimuli. The 250 ms and 500 ms ISIs were blocked by word type such that 20 participants received MR words at 250 ms and VR words at 500 ms and 20 partic ipants received VR words at 250 ms and MR words at 500 ms. The MR words, VR words, and nonwords were presented randomly
52 during the experiment Each word was only presented once. Additionally, t here were 16 word association trials (n=8 for MR words, n=8 for VR words) four of the words were associated with the target word and four of the words were not. Previous studies by Morsella (2002) showed no priming effects beyond 600 ms after the onset of a semantic word prime, but these results were based on comparis on of words with different psycholinguistic properties (words varied by frequency and concreteness ). Additionally, there was overlap in language and motor production in those experiments. In this experiment, we were interested in investigating the effect o f sequential language to motor production with no overlap in task production. As such, we chose the 250 ms and 500 ms ISIs, which allowed for completion of word production and more precise investigation of the time course of our semantic primes. The purpo se of the 700 ms trials was to include a longer ISI to help prevent participants from initiating an anticipatory motor response (i.e., pointing to the target prior to the cue). As such, only nonwords were presented in the 700 ms trials. Control Condition a nd Trial Structure Since the dependent variable in Experiment 2 was RT for pointing, a control task was completed to obtain RT data for pointing in the absence of a prime. The control trials were conducted on a desktop computer running E -Prime stimulus del ivery software with a touch screen monitor Participants were presented with 60 trials and asked to point to the green circle as quickly as possible. The timing and duration of the fixation cross, green circle cue, and interstimulus interval was the same i n the control trials as the experimental trials.
53 Data Acquisition and Processing E-Prime recorded RT in milliseconds via the touch screen computer monitor. Word production was recorded via a Marantz digital audio recorder in order to complete offline scori ng of responses. Since semantic processing of target words was critical to this experiment, recordings were reviewed and the trials in which participants made errors in word production were removed. N onword trials were also removed. The RT mean and standar d deviation in the remaining control and experimental trials was used to calculate a z -score for each trial. We eliminated reaction time outliers using a z -score cutoff of + 1.96 (corresponding to an alpha of .05) We also eliminated trials in which partic ipants initiated pointing prior to the cue and trials in which responses were not recorded on the first contact (i.e., participants had to touch the screen twice). Mean RTs for the control and experimental condition were calculated for each participant on the remaining trials. The effect of MR words and VR words on pointing was calculated by subtracting the mean RT of control pointing from the mean RT of pointing after MR and VR word primes. Statistical Analysis and Results Combined word production errors (e.g., wrong word produced) and motor production errors (e.g., RT not recorded on first contact) accounted for 17% of responses across all participants. To determine the effect of semantic -motor primes on pointing to a fixed target, a 2 x 2 mixed model rep eated measures a nalysis of variance (ANOVA), with W ord T ype (MR, VR) as a within subjects factor and ISI (250 ms, 500 ms) as a between subjects factor was conducted. The ANOVA revealed a significant main effect of ISI [F(1, 3 8 ) = 8.18 p = .007177] qualified by a W ord T ype by ISI interaction [F(1, 3 8 ) = 75.42 p 665]. The interaction
54 indicat ed that RT for pointing to a fixed target was differentially affected by production of MR words relative to VR words as well as the duration of the delay (i.e., ISI) See Table 5 -1 for means and standard error means. No significant main effect of Word Type was found [F(1, 3 8 ) = 3.39 p = .0 74082]. An independent samples t test was conducted to further expl ore the Word Type by ISI interaction revealed by the ANOVA. The t test revealed a significant priming effect for MR words [ t (38)=5.8 7 p< .0001] but no significant priming effect for VR words [ t (38)=.057, p= .955 ] as shown in Figure 52. Furthermore, at 2 50 ms ISI, there was an increase in RT of 74 ms for MR words and an increase in RT of 38 ms for VR words, indicating that MR and VR word production interfered with pointing to a target but the interference was significantly greater for MR words. At the 500 ms ISI, there was a decrease in RT of 17 ms for the MR words and an increase in RT of 37 ms for the VR words Th us, s emantic -motor primes (i.e., MR words) interfered with pointing to a fixed target at a short delay but facilitated pointing to a fixed targ et at a longer delay Interim Discussion This experiment investigated the effect of semantic -motor representations on sequential language to motor production. Participants produced MR and VR word primes prior to initiating a goal -directed manual task (i.e. finger pointing). It was predicted that MR word primes and VR word primes would differentially affect motor production such that MR words primes w ould interfere with motor production. Our finding that MR primes increased RT for pointing to a target at 25 0 ms ISI is consistent with our prediction, as well as with the semantic priming literature in which interference is consistently reported at short SOAs. Our finding that MR primes decreased RT for pointing at 500 ms ISI, al though not specifically predict ed, is also
55 consistent with the semantic priming literature, in which it is reported that facilitation occurs at longer SOAs. Performance with VR primes demonstrates the general effects of switching between a word production task and a pointing task. The relatively equivalent increase in RT at 250 ms and 500 ms ISI for VR word primes illustrates that these words had little effect on motor performance, and f urther supports the interpretation that the effects obtained with MR primes were the result of an int eraction between MR wo rd semantics and motor production. With respect to other production studies our results support those of Morsella (2002) who found interference at 100 ms SOA for high frequency words. However, a direct comparison of results is diffic ult because of the differences in the psycholinguistic properties of our stimuli sets and primes, as well as the fact that Morsellas paradigm investigated overlapping production of language and motor tasks w hereas our study investigated sequential produc tion. With respect to perception findings, our results are similar to those of Boulenger and colleagues who showed that processing of action verbs interfered with a reaching task within 200 ms of cue onset (Boulenger, et al., 2008) Because o f the short SOA in that experiment, there was likely overlap in the verb perception and motor production tasks. Nonetheless, t here appear to be similarities in how language perception and language production affect motor production. In summary, the findings of Experiment 2 support the use of cross modal semantic priming to investigate the effect of sequential language to motor production. The finding of interference and facilitation in only the MR condition (not the VR condition) suggests that the priming e ffect obtained was related to activation of semantic motor representations via semantic -motor primes and not other factors. As such, the
56 hypothesis that language production is sufficient to engage the motor system is supported. Additionally, use of semanti c motor primes interfered wi th motor production at a short delay and facilitated motor production at a longer delay su pporting the hypothesis that temporal dynamics largely determine the nature of language motor interactions. Th ese findings will be discus sed further in Chapter 7.
57 Figure 5 1. Schematic depiction of the trial structure for Experiment 2. Table 5 1. Reaction t ime means for pointing to a fixed target by ISI and C ondition 250 ms 500 ms Mean SEM Mean SEM Control (ms) 513.65 14.50 501.81 18.70 MR words (ms) 588.16 13.1 484.57 16.05 VR words (ms) 552.42 12.29 539.52 19.59 Figure 52 Depiction of MR and VR word primes on pointing by ISI.
58 CHAPTER 6 EXPERIMENT 3: EFFECT OF SEMANTIC -MOTOR REPRESENTATION S ON SEQUENT IAL MOTOR TO LANGUAGE PRODUCTION In this experiment, t he effect of semantic -motor representations on motor to language production was examined using motor production as a prime. This paradigm was based on t he hypothesis that engaging in a motor task activates motor programs which can, in turn, influence production of words associated with human motor movement Previous experiments have utilized manual grasp and finger extension as motor primes (Morsella, 2002) These motor primes resulted in different results presumably based on difficulty in production. In this experiment, finger tapping served as the motor prime. Finger tapping was se lected because it is a relatively simple motor task that can be carried out with some consistency and maintained over time. This experiment investigated the effect of finger tapping on subsequent MR and VR word production. The dependent variable in Experim ent 3 was RT for word production. Experimental Procedure Experiment 3 was conducted on a desktop computer running E Prime stimulus delivery software and a voice key coupled to a sensitive microphone relay. Participants were seated i n front of the computer and a computer mouse was placed on a secured pad within comfortable reach. Participants were instructed to watch the computer screen and, upon presentation of the cue (green square), begin tapping the button on the mouse as quickly and steadily as possible until a word appeared inside a red box at which time they should stop tapping and read the word aloud as quickly and accurately as possible. The participants were asked to keep their eyes on the computer screen and their index finger on the mouse at all t imes.
59 After the instructions were provided, participants were given practice trials to familiarize themselves with stopping and starting the finger tapping task on cue. Feedback was provided on trials in which participants did not complete the task correct ly. The structure of the practice trials was analogous to the experimental trials; however, the words used in the practice trials were not part of the experimental stimuli. Experimental Trial Structure After the 250 ms fixation cross, t he motor prime cue appeared for 300 ms. The w ord cue (red square with the word) was randomized across trials to occur at 500 ms, 750 ms, and 1000 ms after the appearance of the green square of the motor prime and participants tapped until the red square appeared with the word. The word cue remained on the screen until the participant initiated word production. On trials that contained the word association task, a word appeared 1 5 00 ms after word production. E-Prime presented the trials automatically with an intertrial interval of 1 5 00 ms. See Figure 6 1 for a depiction of the experimental trial structure Experiment 3 consisted of 96 trials (n= 24 MR words, n=24 VR words, n=48 nonwords) The 500 ms and 750 ms ISIs were blocked by word type such that 20 participants received MR words at 500 ms and VR words at 750 ms and 20 participants received VR words at 750 ms and MR words at 500 ms. The MR words, VR words, and nonwords were presented randomly during the experiment Eac h word was only presented once. The word association task used in Experiment 2 was also used in this experiment to assure semantic processing of words In this experiment, we were interested in investigating the effect of sequential motor to language production with no overlap in production tasks. Additionally, w e were interested in the effect of motor production rather than motor planning or response
60 preparation. As such, we chose the 500 ms and 750 ms ISIs to allow time for initiation of the motor task. The purpose of the 1000 ms trials was to include a longer SOA to help prevent participants from extinguishing the motor task in anticipation of the word. As such, only nonwords were presented in the 700 ms trials. Control Condition and Trial Structure Since the dependent variable in Experiment 3 was RT for word production, a control task was completed to obtain data on RT for word production in the absence of a prime. The control task was conducted on a desktop computer running E -Prime stimulus delivery software and a voice key coupled to a sensitive microphone relay. Participants were asked to read aloud each word presented on the screen as quickly and accurately as possible. There were 96 trials (n=48 MR words, n= VR words) presented randomly. The control words were the same words used in the experimental trials. The timing and duration of the fixation cross, word cue, and interstimulus interval was the same in the control trials as the experimental trials. Data Acquisition and Processing In the experimental trials E -Prime recorded data on finger -tapping as well as RT for words in milliseconds via the voice key and microphone relay. In the control trials E Prime recorded RT for word production. In both conditions, word production was also recorded via a Marantz digital audio recorder in order to complete offline sco ring of responses. Since the motor component preceding language was critical to this experiment, experimental trials in which participants did not initiate finger tapping were removed. Semantic processing of target words was also critical to the experiment so recordings were reviewed and the trials in which participants made errors in word production were removed. N onword trials were also removed.
61 The RT mean and standard deviation for the remaining control and experimental trials was used to calculate a z -score for each trial. We eliminated reaction time outliers using a z -score cutoff of + 1.96 (corresponding to an alpha of .05). This eliminated trials in which responses were not captured on the first attempt (i.e., participants had to say the word twice) Mean RTs were calculated for MR and VR words in the control condition and experimental conditions. The effect of motor prim es on MR words and VR words was calculated by subtracting the mean RT of the control words from the mean RT of the primed words in each condition. Statistical Analysis and Results Combined word production and motor production errors accounted for 13% of responses across all participants. A 2 x 2 mixed model repeated measures ANOVA with Word T ype (MR, VR) as a within subjects factor an d ISI (500 ms, 750 ms) as a between subjects factor was conducted. The ANOVA revealed a significant W ord T ype by ISI interaction [F(1, 3 8 ) = 24.87 p 396 ], indicating that finger tapping differentially affected production of MR words and VR words and that production of these words was also differentially affected at different delays See Table 6-1 for means and standard error means. No main effect of Word Type [F(1, 3 8 ) = 3.32 p = 568, 009] or ISI [F(1, 3 8 ) = 3.27, p = 079422 ] was found. An i ndependent -samp les t test was conducted to further explore the Word Type by ISI interaction revealed by the ANO VA. The t test revealed that finger tapping si gnificantly affected MR word production [ t (3 8)= 2.79, p =. 008 ] but not VR word production [ t (38)=. 6 7 2 p =. 505 ] At the 500 ms ISI, there was an increase in RT of 1 39 ms for MR words and 121 ms for VR words At th e 750 ms ISI, there was an increase in RT of 97 ms for the MR words and 111 ms for the VR words.
62 A p aired samples t -test was conducted to compare the priming effects obtained for MR words relative to VR words at the two ISIs. The t test s were conducted at alpha level .025 based on a Bonferroni correction for multiple comparisons The t test s revealed a significant difference between the priming effects f or MR words relative to VR words at 500 ms ISI [paired t (18)= -3.96, p= .001] and 750 ms ISI [paired t (20) =3.12, p= .005] Importantly, as shown in Figure 6-2, t hese results suggest that the motor prime caused significant interference of MR word production at 500 ms relative to VR word production, but facilitation of MR word production relative to VR word production at 750 ms ISI Interim Discussion This experiment investigated the effect of semantic -motor representations on sequential motor to language production. Participants engaged in a finger -tapping task prior to production of MR and VR words. It was predi cted that sequential motor to language production would have differential effects on MR words and VR words such that motor production w ould interfere with subsequent production of MR words. Our finding that finger -tapping increased RT for MR word production relative to VR word production at 500 ms ISI is consistent with our prediction, as well as reports in the semantic priming literature of interference at short SOAs Our finding that finger -tapping resulted in a significant decrease in RT for MR word production at 750 ms, al though not specifically predicted, is also consistent with the semantic priming literature, in which it is reported that facilitation occurs at longer SOAs. The relatively equivalent priming effects obtained in RT at 500 ms and 750 ms ISI for VR word production further
63 supports the interpretation that the effects obtained were the result of an interaction between the motor prime and MR word production. To our knowledge, no studies have investigated the effect of motor primes on sequent ial motor to language production tasks. In a concurrent task, Morsella (2002) found that concreteness determined whe ther motor primes facilitated or interfered with single word production. However, a direct comparison of results with that study is difficult because the task was con current rather than sequential. With respect to previous findings in the perception literature, our results are similar to those of Setola and Reilly (2005) who demonstrated that performed movements resulted in sl ower RTs for making lexical decisions about hand action words Again, there appear to be similarities in how motor production affects language perception and language production. In summary, the findings of Experiment 3 support the use of cross modal semantic priming to investigate the effect of sequential language to motor production. The finding of sign ificant interference and facilitation in only the MR condition (not the VR condition) suggests that the priming effect obtained was related to activation of semantic -motor representations via the motor prime and not other factors. As such, the hypothesis t hat motor engagement is sufficient to influence language production is supported. Additionally, use of motor primes interfered with MR word production at a short delay and facilitated MR word production at a longer delay, again supporting the hypothesis th at temporal dynamics largely determine the nature of language motor interactions. The se findings will be discussed further in Chapter 7.
64 Figure 6 1 Schematic depiction of the trial structure for Experiment 3. Table 6 1. Response time means for word production by ISI and C ondition 500 ms 750 ms Mean SEM Mean SEM Control MR words (ms) 461. 6 8 12.22 475.36 10.32 Experimental MR words (ms) 600.84 14.90 572.67 12.05 Control VR words (ms) 460.58 11.73 471.49 10.78 Experimental VR words (ms) 581.68 13.09 583.10 11.77 Figure 62 Depiction of the effects of motor priming on MR and VR words by ISI.
65 CHAPTER 7 DISCUSSION The three experiments that comprised our study investigated the effect of semantic -motor representations on lang uage and motor production tasks. While some evidence has been reported in the literature supporting language motor interaction, our study was the first to investigate this interaction in a task requiring continuous, concurrent language and motor production. Additionally, it was the first to test the reciprocality of language and motor effects in a sequential production paradigm. The results obtained in Experiment 1-3 support the existence of language motor interactions in concurrent and sequential tasks. While these language and motor effects were dependent on temporal aspects (i.e., we found both facilitation and interference under different conditions), we demonstrated that semantic motor representations can affect both language and motor production. In th e following sections, we provide a discussion of our results in the context of resource allocation theory (Experiment 1) and pictureword interference effects (Experiments 2 and 3). We conclude with implications for sensorimotor theory and embodied cogniti on, implications for individuals with neurologic disease, and considerations for future research. Resource Allocation Theory Attention a cognitive process that involves selecti ve allocation of resources has long been a topic of great interest in cognitiv e and clinical sciences. Similarly, dual task performance, or the ability to execute two tasks simultaneously, has been central to attention research for over a century (Pashler, 1994) Aside from the re levance of attention and dual task performance to everyday life (i.e., talk ing on a cell phone while driving a vehicle), dual task performance provides a means for understanding the
66 functional interaction of brain systems. A number of theories have been pr oposed to explain findings related to dual task performance (for a review see Pashler, 1994) We will focus on resource allocation theories which hold that performance o f tasks requires utilization of a modality neutral single resource pool or a discrete series of smaller resource pools dedicated to attention allocation for specific tasks The emergence of resource theories was born out of the inability of other theories to account for tasks that require the ability to respond simultaneously to multiple tasks or multiple task demands (e.g., divided attention) and the ability to maintain a behavioral set despite distracting or competing stimuli (e.g., selective attention). Differing resource theories were proposed by Kahneman (1973) and later modified by Norman and Bobrow (1975) Navon and Gopher (1979) and Wickens (1984) Although resources have not been structurally well defined, they can be conceptualized in terms of supply and demand. As performance on one or mor e tasks increases the demand for resources, the supply of resources is diminished for use in other tasks The relationship between resource supply and demand is what determines task performance. In some cases, the resource demands do not exceed the resourc e supply. In such cases, task performance is not affected. More commonly though, resource demands do exceed resource supplies and performance on one or more tasks suffers. Single Resource Theories One of the early resource theories, put forth by Kahneman (1973) proposed that the brain has a single pool of resources available for the performance of tasks. According to his theory, task performance is dependent on how resources within this limited -capacity pool a re allocated. That is, two tasks cannot be completed simultaneously at the same rate, so the duration and difficulty of the tasks being
67 performed predicts the presence and degree of interference (Hazeltine, Ruthruff, & Remington, 2006; McLeod, 1977) As such, single resource theories are classified as content independent, because interference is the result of generic limitations of a system in performing two tasks, rather than the content of information being processed. Multiple Resource Theories An alternative to this single resource view are multiple resource theories first pr oposed by Allport and colleagues (1972) and Navon and Gopher (1979) Multiple resource theories were developed due to weaknesses in single resource theories, such as the inability to explain perfect time -sharing between tasks and different degrees of interference that are unrelated to manipulations of task difficulty and task modalities (Wickens, 1984) In contrast to a general resource pool approach, m ultiple resource theories hold that the brain is comprised of multiple processors that function more or less independently (i.e., each processor has its own resources that can be shared by several tasks ). A later version of this theory, proposed by Wickens (1984) took that idea one step further and suggested that there are separate processors for modalities (i.e., visual and auditory), codes (i.e., spatial and verbal), s tages (i.e., encoding/processing and responding) and responses (i.e., manual and vocal), all served by different brain regions. According to multiple resource theories, performing two tasks that utilize different processors will result in less interference than two tasks that utilize the same processors. As such, task similarity predicts the degree of interference (McLeod, 1977) Multiple resource theories are classified as content -dependent because interference is the result of the structure, modality, and content of inf ormation being processed, rather than generic limitations of a system in performing two tasks. Multiple resource theory
68 offers an explanation for the findings in Experiment 1, which required simultaneous finger tapping and word generation. Findings from Ex periment 1 In Experiment 1 we found dual task costs (i.e., decreases in time on task and decreased total task output) during both the Semantic Motor and the Semantic -Other conditions, although the costs were significantly greater in the latter. This interf erence is predicted by utilization of resources for word production and tapping (see Figures 71 and 7 -2). That is, both word production and finger tapping share resources from stage processors due to the requirement of a response and from response pro cessors due to utilization of verbal and manual responding, resulting in a demand for sharing resources when these two tasks are performed concurrently This resource s haring readily d escribes the dual task effects found in the Semantic -Other condition; however, this does not account for the finding of the significant difference in dual task costs between Semantic -Motor and Semantic -Other conditions. The finding that Semantic Motor and Semantic -Other conditions were differentially affected under the dual task condition suggests that continuous tapping and motor word production were facilitative in the Semantic Motor condition. W e propose that the dual task effects observed in the Semantic -Other condition represents baseline dual task effects of generating words while tapping and that the performance in the Semantic Motor condition was facilitated by semantic priming between finger tapping and semantic -motor word production. As such, th e primary difference between Semantic Other and Semantic -Motor conditions w as that MR words activate d semantic motor representations in the brain. Multiple resource theories do not take into account effects of semantic priming, or the influence of the meaning of one word on subsequent
69 retrieval of a semantically related word. In this case, both hand movement and hand w ord production were facilitated relative to the unrelated Semantic -Other cond i tion, suggesting that facilitation was due to semantic priming. This interpretation is further supported by the findings of Experiments 2 and 3 in which finger tapping and MR word production served as primes for language and motor production, respectively. In summary, we demonstrated that continuous, concurrent tapping and word production activated semantic motor representations which facili tated performance in b oth tasks The dual task cost observed in both the Semantic -Motor and Semantic Other conditions is predicted by multiple resource theory but the advantage for tapping and word production in the Semantic Motor condition is predicted b y semantic priming. These findings are consistent with the hypothesis that language and motor systems are capable of reciprocal activation. Picture Word I nterference Effects A common method for investigating interactions between attention, semantic represe ntations, and language production is the picture word interference paradigm in which individuals name pictures while ignoring written word distractors. It has been demonstrated that when word distractors are presented along with a picture, RT for naming the picture is increased. It has also been demonstrated that the degree of interference depends on the degree to which the target (i.e., picture) and distractor (i.e., word) are related (for a review see MacLeod, 1991) That is, when the tar get and distractor are semantically related (e.g., dog-cat) there is greater interference than when the target and distractor are unrelated (e.g., dog-cloud). In Experiments 2 and 3 we investigated the reciprocality of semantic motor representations on language and motor production tasks. We employed variations of
70 the picture word paradigm in both experiments. In Experiment 2 we used semantic primes to investigate the effect of semantic motor representations on motor production. In this experiment, pointing to a circle on a computer screen was the target and MR and VR word primes served as related and unrelated distractors, respectively. In Experiment 3, we used a motor prime to investigate the effect of motor production on semantic motor word representations. In this experiment, MR words and VR words served as targets and finger tapping (i.e., the motor prime) served as the distractor. The motor prime was treated as a semantic prime capable of activating categorically related semantic -motor representations. Findings from Experiment 2 In Experiment 2 we demonstrated that MR word primes behaved like categorically related distractors. Engagement of the motor system via these primes resulted in significant interference with pointing to a fixed target at a short d elay and significant facilitation at a longer delay Additionally, VR word primes behaved like unrelated distractors that did not engage the motor system and thus did not significantly affect RT for pointing to a fixed target. We proposed that the observed increase in RT for VR words relative to baseline resulted from switching from a verbal to motor task rather than engagement of the motor system T hese findings are consistent with semantic interference effects observed in the pictureword interference par adigm. Additionally, the finding that MR words influenced pointing to a target suggests that activation of a specific semantic motor representation may not be necessary to influence motor production. That is, the semantic word prime yielded low -level activ ation of the motor system sufficient to influence motor production. We interpret the findings of Experiment
71 2 as supporting evidence for our prediction that activation of semantic -motor representations through language can affect motor production Findings from Experiment 3 In Experiment 3 we demonstrated that finger -tapping as a motor prime behaved like a categorically related semantic prime. In the VR word condition, the motor prime was an unrelated distractor, and thus represents the simple effect of hav ing to stop tapping and produce any word. However, relative to performance in the VR word condition, we found interference at a short delay and facilitation at a longer delay for MR word production. These findings are also consistent with semantic interfer ence effects observed in the picture word paradigm. Additionally, the finding that tapping influenced production of MR words suggests that engagement of the motor system need not be symbolic to facilitate MR word production. That is, the motor prime yielded low -level activation of the motor system sufficient to influence word production. We interpret the findings of Experiment 3 as supporting evidence that engagement of the motor system can affect motor -related language production. In summary, we demonstrat ed the existence of reciprocal interactions between semantic and motor representations on sequential languageand motor production tasks. Additionally, we confirmed that finger tapping and MR words can be used as categorically -related primes capable of inf luencing language and motor systems, respectively. The standardization and counterbalancing of our two stimuli sets suggests that our effects were not attributable to psycholinguistic variables known to influence RT for naming, such as concreteness (Strain, Patterson, & Seidenberg, 1995) and familiarity (Connine, Mullennix, Shernoff, & Yelen, 1990) Rather, they were based on the sensorimotor properties of our words.
72 Summary of Findings from Experiments 13 While the three experiments that comprised our study were orthogonal, the effects obtained are complimentary and contribute evidence in support of our hypotheses. Experiment 1 supported language motor interactions in a continuous, concurrent paradigm. Dual task effects of tapping and word production to motor related categories were not as great as dual task effects of tapping and word production to nonmotor related categories. These findings suggest it is possible to simultaneous ly engage semantic -motor representations via language and motor production and that this simultaneous engagement can be facilitative in nature. Experiments 2 and 3 supported language motor interactions in sequential production paradigms. Experiment 2 showed that activation of semantic -motor representations via MR word primes can lead to interference or facilitation of a motor production task depending on the delay. Similarly, Experiment 3 showed that activation of semantic motor representations via a motor prime can lead to interference or facilitation of MR word production depending on the delay. Since these effects were not observed in the VR condition, we interpret this as evidence that languagemotor interaction was unique when semantic -motor repres e ntat ions were activated. It is interesting to note the nature of the language-motor interactions observed in Experiments 1 3. Experiments 2 and 3 consistently yielded interference at short delays and facilitation at long delays for motor and language tasks in the MR word conditions, respectively. However, Experiment 1 yielded only facilitation for both the motor and language task s in the Semantic -Motor condition. It may be that Experiment 1 mimicked the longer delay condition of Experiment 3 and that tapping fo r longer periods of time allow ed for a utomatization of the motor behavior (i.e., selection of the response had
73 occurred, the motor program was executed, and the pattern only need ed to be maintained). Thus, tapping became an automatic process that result ed in low level activation of the motor system sufficient to facilitate motor -related words. However, a similar explanation cannot account for facilitation of tapping performance To explain the finding of facilitation for tapping, we propose a variation of t he redundancy signal effect. The redundancy signal effect is a phenomenon whereby RTs and response force are faster in the presence of redundant cues relative to single cues (Giray & Ul rich, 1993) It is possible that tapping was speeded in the Semantic Motor condition as a result of redundant cues. That is, continuous tapping served as one cue and continuous generation of motor related words served as another cue, and that this redunda nt cueing of the motor system facilitated tapping performance. This type of advantage is similar to cross-talk facilitation, which has been demonstrated when semantic cues are presented simultaneously during a task requiring verbal and manual responding (Logan & Schulkind, 2000) This explanation of our results is supported by the finding that semantic -motor word primes and motor primes were capable of facilitating performance in Experiments 2 and 3. Howe ver, redundancy signal effects and cross -talk facilitation have only been investigated in experimental paradigms utilizing RT, response force and accuracy as dependent variables. As such, further investigation would be necessary to obtain empirical support for this explanation of facilitation of tapping in the Semantic Motor condition of Experiment 1. Nonetheless, Experiments 1-3 support our hypotheses that language production is sufficient to engage the motor system and that motor engagement is sufficient to in fluence language production. They also support our hypothesis that the nature of
74 language motor interaction is determined by the temporal dynamics of the task. As such, this contributes support for an overlap in anatomical regions that support language pro duction and motor production. Implications for Theories of Semantic Memory Our findings are difficult to reconcile with amodal theories of semantic representation, which suggest that semantic memory consists of abstract representations According to this c lass of theories, concepts are not stored in sensorimotor regions. As such, amodal theories in their current state cannot account for the facilitation of tapping and word generation in the Semantic -Motor condition of Experiment 1, nor can it account for th e finding of a significant influence of MR word primes on pointing in Experiment 2 or the significant influence of a motor prime on MR word production in Experiment 3. Instead, our findings regarding the effect of semantic -motor representations on language and motor production extend the current research in support of sensorimotor theories (e.g., semantic representations are stored according to their sensorimotor features ) and theories of embodied cognition (e.g., concepts are grounded in ones interaction with the world ). Specifically, we were able to demonstrate that semantic representations maintain their sensorimotor states and that these sensorimotor states are active during language and motor production. In our study, the use of semantic primes such as pencil and write, were shown to differentially engage the motor system relative to semantic primes such as flower and bloom, which do not have a human motor component. Additionally, the use of finger -tapping as a motor prime was shown to different ially affect production of words associated with human motor
75 performance (e.g., pencil and write) and generation of words to human motor -related categories (e.g., things you do with your hands) relative to words with a higher weighting of visual salience ( e.g., flower and bloom) and categories not related to the human motor system (e.g., cities). As such, our results demonstrate reciprocality of effects between language and motor systems during concurrent and sequential task production tasks, which can be a ccounted for by sensorimotor theories and embodied cognition. One caveat is that our experiments were not designed to elucidate the underlying nature of activation in the motor system during semantic processing (i.e., we cannot state that motor activation was due to mental simulation associated with the action relevant to the target word). Our study also does not allow us to answer the critical question of whether activation of semantic -motor representations is both necessary and sufficient for production. However, our results are consistent with neurophysiological studies demonstrating a strong association between language and motor systems in perception and production tasks (Boulenger, et al., 2006; Esopenko, et al., 2008; Hauk, et al., 2004; Hauk & Pulvermuller, 2004; Nazir, et al., 2008) While we can argue that semantic activation of motor -related concepts was sufficient to engage the motor system, and that motor activ ation was sufficient to engage the language sy stem we are not able to make claims about whether this activation is necessary for motor producti on or word production in the context of semantic -motor representations Implications for Individuals with Neurologic Disease Language motor interactions like the ones demonstrated in our study have implications for language and motor deficits in neurologic populations T he inherent link
76 between language and motor systems can be used to develop novel rehabilitation approaches for individuals with aphasia and oth er neurologic disease affecting language and motor production. Indeed, some researchers have already begun to exploit language motor interac tion in their rehabilitation approaches and findings suggest that facilitation of language via the motor system is n ot restricted to use of symbolic hand gestures Hanlon and colleagues (1990) demonstrated that activation of the motor system via nonsymbolic gestures p roduced with the right hand influenced word retri eval in individuals with aphasia However, some patients with aphasia have co occurring hemiplegia restricting use of the right hand to initiate any type of g esture. Importantly, Experiment 1 of our study showed that activation of language via the motor sy stem via finger tapping was not dependent on use of the right hand, suggesting that individuals with aphasia and right hemiplegia may be able to use the left hand to stimulate language. This has been demonstrated in several studies that employed the use of a nonsymbolic gesture performed with the left hand to improve naming ability in patients with nonfluent aphasia (Crosson, 2008; Crosson, et al., 2007) Although the approach developed by Crosson and colleagues sought to engage right hemisphere attentional mechanisms to improve naming it is possible that activation of the motor system provided an added benefit to retrieval of words associated with human motor action. Additionally, coupling of action and language production during intensive language therapy has yielded positive results, highlighting the importance of practicing language in action contexts (Pulvermuller & Berthier, 2008)
77 In summary, our study yielded two findings that may be relevant to rehabilitation of deficits in individuals with ap hasia and limb apraxia. First, we demonstrated that both right and left hand finger tapping activated semantic -motor representations influencing the production of motor -related words. Secondly, we demonstrated that motor production need not be symbolic to facilitate MR word production under some conditions (i.e., at longer delays and during continuous, concurrent production tasks). Further investigation into the applicability of these findings for treatment of verb production deficits and action production deficits is warranted. Additionally, continued investigation into the nature of language motor facilitation in healthy individuals and those with neurologic disease is necessary to gain a better understanding of this phenomenon and realize its full potenti al in rehabilitation. Considerations for Future Research The results of this study provide support for the interaction of language and motor systems during various semantic -motor tasks showing that these systems can facilitate or interfere with production depending on temporal dynamics However, there is much to be learned about the nature of this interaction (i.e., the flow of information between these two systems). Research addressing the following questions would extend the findings of our study and cont ribute to current knowledge regarding the functional links between language and motor s ystems: What is the time course of languagemotor interference and facilitation in production tasks? This question can be addressed by utilizing a variety of SOAs withi n the range of those employed in our study to define where interference ends and facilitation begins. Do semantic primes and motor primes result in spreading activation throughout the relevant systems or is activation specific to the effector? This questio n can be addressed by including different effectors for the motor prime (i.e., feet) and different motor -related word categories (e.g., words associated with foot actions).
78 Do the language -motor effects observed in this study occur beyond the word producti on level? This question has been addressed in perception studies (Buccino, et al., 2005; Tettamanti, et al., 2005) but use of sentence level stimuli in a production paradigm would extend the research and broaden the applicability of the findings to functional language production. Do the language -motor effects observed in our study generalize to older adults? We chose to include only younger adults in our sample due to changes in motor performance and psychomotor perf ormance with age. However, investigation of the effects of semantic -motor representations on language and motor production in older may provide valuable on how language and motor system interaction changes with age. Do the language motor effects observed in our study generalize to individuals with language and motor system deficits (e.g., stroke Parkinsons disease)? A better understanding of this interaction in compromised systems would contribute valuable information to current theories of semantic memor y and potentially inform the development of new treatment approaches. Conclusion The three experiments that comprised our study demonstrated that language and motor systems interact during continuous, concurrent as well as sequential production tasks in th e context of semantic motor representations. Thus, ou r results demonstrate that semantic representations maintain their sensorimotor states, and that activation of these sensorimotor representations can be primed via both the language and motor system. Thus, our study supports the hypothesis of reciprocality in language motor system interaction and contributes to the evidence in favor of sensorimotor theories and theories of embodied cognition.
79 Figure 71. Schematic depiction of multiple resource pools used for word production. Figure 72. Schematic depiction of multiple resource pools used for finger tapping.
80 APPENDIX A INSTRUCTIONS FOR PARTICIPANT RATINGS OF WORD STIMULI Familiarity You are asked to use the rating scale (17) found next to e ach word to indicate how familiar you are with the word, where 1=not at all familiar with the word and 7=extremely familiar with the word. Please circle one number to indicate your judgment of the words familiarity. Example: 1 2 3 4 5 6 7 Not at Extremely all familiar familiar Appear 1 2 3 4 5 6 7 Cup 1 2 3 4 5 6 7 Grass 1 2 3 4 5 6 7 Concreteness You are asked to use the rating scale (17) found next to each word to indicate the concreteness of the word. Concreteness is defined as how well you are able to see, touch, or manipulate what the word represents, where 1= not at all able to see, touch, or manipulate and 7=extremely able to see, touch or manipulate. Please circle one number to indicate your judgment of the words concreteness Example: 1 2 3 4 5 6 7 Not at Extremely all able able Appear 1 2 3 4 5 6 7 Cup 1 2 3 4 5 6 7 Grass 1 2 3 4 5 6 7 Dominant Grammatical Class You are asked to circle either noun or verb depending on whether you associate the word more strongly as a noun (object) or a verb (action/event). Please circle only one grammatical class for each word. Example: Appear : Noun Verb Cup: Noun Verb Grass : Noun Verb
81 APPENDIX B PSYCHOLINGUISTIC CHA RACTERISTICS OF STIM ULI IN SETS 1 AND 2 Concreteness* Familiarity* Grammatical Class** SET 1 Motor Nouns n eedle 6.5 6.3 10 l id 6.6 6.6 10 a rrow 6.8 5.9 10 p liers 6.7 5.7 10 r azor 6.7 6.8 10 k nob 6.8 6.5 10 f lute 6.9 5.9 10 f ork 6.9 7.0 9 p en 6.9 7.0 10 w rench 7.0 6.4 10 b all 7.0 6.7 10 k nife 7.0 6.8 10 SET 2 Motor Nouns b lade 6.6 6.7 10 k ey 6.6 7.0 10 d art 6.7 5.8 10 c omb 6.7 6.9 9 a xe 6.8 5.8 10 h ammer 6.8 6.4 10 s poon 6.8 7.0 10 s w ord 6.9 6.0 10 g un 7.0 5.8 10 z ipper 7.0 6.8 10 c up 7.0 6.9 10 p encil 7.0 7.0 10 SET 1 Motor Verbs k nock 4.1 6.7 10 t urn 4.2 6.7 10 t ap 4.3 5.8 9 e rase 4.4 6.7 10 s ew 4.5 6.0 10 p ush 4.5 6.6 10 w rap 4.6 6.3 10 f old 4.6 6.5 10 p our 4.7 6.9 10 c arve 4.8 6.0 10 c hop 4.8 6.5 10 p oint 5.3 6.7 9 SET 2 Motor Verbs c ut 4.2 6.7 10 d ig 4.2 7.0 9 p oke 4.4 3.6 10 g rab 4.4 6.6 10 l ift 4.5 6.6 10
82 Concreteness* Familiarity* Grammatical Class** p ull 4.5 6.8 10 s have 4.6 6.1 10 r ub 4.6 6.9 10 s tir 4.7 6.2 10 t ug 4.7 6.5 9 t ype 4.7 6.9 10 w ipe 5.1 6.9 10 SET 1 Visual Nouns l ion 6.6 6.1 10 l a ke 6.6 6.8 10 b each 6.6 6.8 10 n est 6.7 6.0 9 t iger 6.8 6.1 10 l eaf 6.8 6.9 10 p ig 6.9 6.4 10 b ee 6.9 6.5 10 t ree 6.9 7.0 10 o live 6.9 6.8 9 p ea 7.0 6.7 10 a pple 7.0 7.0 10 SET 2 Visual Nouns l awn 6.6 6.6 10 z ebra 6.7 6.2 10 p ool 6.7 6.7 10 o cean 6.7 6.9 10 f rog 6.8 6.5 10 f lower 6.8 6.6 9 d oor 6.8 6.9 10 p lum 6.9 6.4 10 t urtle 6.9 6.5 10 p arrot 6.9 6.5 10 l ime 7.0 6.3 10 g rass 7.0 7.0 10 SET 1 Visual Verbs w arp 2.9 5.6 10 f ade 3.2 6.3 10 l aunch 3.3 6.0 10 r ot 3.4 6.2 9 g low 3.7 6.2 9 s izzle 4.0 5.6 10 p eck 4.0 6.0 10 g raze 4.0 6.1 10 f reeze 4.0 6.6 9 b lee d 5.1 6.6 9 d ry 4.1 6.7 9 d rip 5.9 6.9 10 SET 2 Visual Verbs s eep 3.1 5.3 10 b urst 3.3 5.9 10
83 Concreteness* Familiarity* Grammatical Class** e rupt 3.6 5.7 10 b loom 3.6 6.3 9 t haw 3.7 6.3 9 g row 3.8 6.5 10 s immer 4.0 6.4 10 w ag 4.1 6.3 10 b oil 4.5 7.0 10 m elt 4.6 6.4 9 b urn 4.6 6.7 9 b rew 4.8 6.4 9 *Mean rating of 10 individuals using a 7point rating scale **Number of individuals out of 10 who ra ted grammatical class as listed
84 APPENDIX D DESCRIPTION OF NONWO RD STIMULI SELECTION Experiments 2 and 3 required the inclusion of nonwords in the stimuli sets. While the selection of these words was not part of the standardization study described above, the selection process was systematic. Nonwords were extracted from the English Lexicon Project (ELP) online database which contains 40,481 nonwords (Balota, et al., 2007) The ELP database was queried for nonwords that were three to six letters in length. A list of over 11,600 nonwords was returned along with mean accuracy for naming. A total of 96 nonword stimuli were chosen that fit the following criteria: Mean accuracy for na ming above 97% Phonologically plausible in the English language No more than two syllables in length No suffixes indicating plurality or tense
85 LIST OF REFERENCES Allport, D. A. (1985). Distributed memory, modular subsystems and dysphas ia. New York: Churchill Livingstone. Allport, D. A., Antonis, B., & Reynolds, P. (1972). On the division of attention: A disproof of the single channel hypothesis. Quarterly Journal of Experimental Psychology, 24(2), 225235. Aoki, T., & Fukuoka, Y. (2010) Finger tapping ability in healthy elderly and young adults. Medicine & Science in Sports & Exercise, 42(3), 449455. Arevalo, A., Perani, D., Cappa, S. F., Butler, A., Bates, E., & Dronkers, N. (2007). Action and object processing in aphasia: From nouns and verbs to the effect of manipulability. Brain and Language, 100 (1), 79 94. Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., et al. (2007). The english lexicon project. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt= Citation&list_uids=17958156 Barsalou, L. W. (1999). Perceptual symobols systems. Behavioral and Brain Sciences, 22, 577660. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617645. Barsalou, L. W., Simmons, W. K., Barbey, A. K., & Wilson, C. D. (2003). Grounding conceptual knowledge in modality -specific systems. Trends in Cognitive Sci ences, 7(2), 84-91. Beauchamp, M. S., Lee, K. E., Haxby, J. V., & Martin, A. (2002). Parallel visual motion processing streams for manipulable objects and human movements. Neuron, 34(1), 149159. Beauchamp, M. S., Lee, K. E., Haxby, J. V., & Martin, A. (20 03). FMRI responses to video and point light displays of moving humans and manipulable objects. Journal of Cognitive Neuroscience, 15(7), 9911001. Bensafi, M., Sobel, N., & Khan, R. M. (2007). Hedonic -specific activity in piriform cortex during odor imagery mimics that during odor perception. Journal of Neurophysiology, 98(6), 32543262. Berndt, R. S., Mitchum, C. C., Haendiges, A. N., & Sandson, J. (1997). Verb retrieval in aphasia. 1. Characterizing single word impairments. Brain and Language, 56(1), 68106.
86 Boulenger, V., Roy, A. C., Paulignan, Y., Deprez, V., Jeannerod, M., & Nazir, T. A. (2006). Cross -talk between language processes and overt motor behavior in the first 200 msec of processing. Journal of Cognitive Neuroscience, 18(10), 16071615. Boule nger, V., Silber, B. Y., Roy, A. C., Paulignan, Y., Jeannerod, M., & Nazir, T. A. (2008). Subliminal display of action words interferes with motor planning: A combined EEG and kinematic study. Journal of Physiology Paris, 102(1 3), 130136. Buccino, G., R iggio, L., Melli, G., Binkofski, F., Gallese, V., & Rizzolatti, G. (2005). Listening to action-related sentences modulates the activity of the motor system: A combined TMS and behavioral study. Brain Research. Cognitive Brain Research, 24(3), 355363. Buxb aum, L. J., Sirigu, A., Schwartz, M. F., & Klatzky, R. (2003). Cognitive representations of hand posture in ideomotor apraxia. Neuropsychologia, 41 (8), 10911113. Caramazza, A., & Hillis, A. E. (1991). Lexical organization of nouns and verbs in the brain. Nature, 349(6312), 788790. Chao, L. L., Haxby, J. V., & Martin, A. (1999). Attribute based neural substrates in temporal cortex for perceiving and knowing about objects. Nature Neuroscience, 2 (10), 913919. Chao, L. L., & Martin, A. (2000). Representation of manipulable manmade objects in the dorsal stream. NeuroImage, 12 (4), 478-484. Clark, D. G., Guitar, B., & Hoffman, P. R. (1985). Reliability of verbal manual interference. Brain and Cognition, 4 (4), 486491. Coccia, M., Bartolini, M., Luzzi, S., Provi nciali, L., & Lambon Ralph, M. A. (2004). Semantic memory is an amodal, dynamic system: Evidence from the interaction of naming and object use in semantic dementia. Cognitive Neuropsychology, 21(5), 513527. Connine, C. M., Mullennix, J., Shernoff, E., & Y elen, J. (1990). Word familiarity and frequency in visual and auditory word recognition. Journal of Experimental Psychology. Learning, Memory, and Cognition, 16 (6), 1084-1096. Crosson, B. (2008). An intention manipulation to change lateralization of word p roduction in nonfluent aphasia: Current status. Seminars in Speech Language Pathology, 29(3), 188 -200. Crosson, B., Fabrizio, K. S., Singletary, F., Cato, M. A., Wierenga, C. E., Parkinson, R. B., et al. (2007). Treatment of naming in nonfluent aphasia thr ough manipulation
87 of intention and attention: A phase 1 comparison of two novel treatments. Journal of the International Neuropsychological Society, 13(4), 582594. Damasio, A. R., & Tranel, D. (1993). Nouns and verbs are retrieved with differently distributed neural systems. Proceedings of the National Academy of Sciences of the United States of America, 90(11), 4957 -4960. Damasio, H., Grabowski, T. J., Tranel, D., Hichwa, R. D., & Damasio, A. R. (1996). A neural basis for lexical retrieval. Nature, 380(65 74), 499-505. Damasio, H., Grabowski, T. J., Tranel, D., Ponto, L. L., Hichwa, R. D., & Damasio, A. R. (2001). Neural correlates of naming actions and of naming spatial relations. NeuroImage, 13 (6), 10531064. Daniele, A., Giustolisi, L., Silveri, M. C., C olosimo, C., & Gainotti, G. (1994). Evidence for a possible neuroanatomical basis for lexical processing of nouns and verbs. Neuropsychologia, 32(11), 13251341. Djordjevic, J., Zatorre, R. J., Petrides, M., Boyle, J. A., & Jones -Gotman, M. (2005). Functional neuroimaging of odor imagery. NeuroImage, 24(3), 791801. Esopenko, C., Borowsky, R., Cummine, J., & Sarty, G. (2008). Mapping the semantic homunculus: A functional and behavioural analysis of overt semantic generation. Brain Topography, 21(1), 22-35. Falkenstein, M., Yordanova, J., & Kolev, V. (2006). Effects of aging on slowing of motor response generation. International Journal of Psychophysiology, 59(1), 22 -29. Gainotti, G. (2006). Anatomical functional and cognitive determinants of semantic memory disorders. Neuroscience & Biobehavioral Reviews, 30 (5), 577 -594. Gainotti, G., Silveri, M. C., Daniele, A., & Giustolisi, L. (1995). Neuroanatomical correlates of category -specific semantic disorders: A critical survey. Memory, 3 (3 4), 247264. Gallese, V. & Lakoff, G. (2005). The brain's concepts: The role of the sensory motor system in conceptual knowledge. Cognitive Neuropsychology, 22 455-479. Giray, M., & Ulrich, R. (1993). Motor coactivation revealed by response force in divided and focused attention. Journal of Experimental Psychology. Human Perception and Performance, 19(6), 1278-1291. Gonzalez, J., Barros Loscertales, A., Pulvermuller, F., Meseguer, V., Sanjuan, A., Belloch, V., et al. (2006). Reading cinnamon activates olfactory brain regions. Ne uroImage, 32 (2), 906-912.
88 Grafton, S. T., Fadiga, L., Arbib, M. A., & Rizzolatti, G. (1997). Premotor cortex activation during observation and naming of familiar tools. NeuroImage, 6(4), 231-236. Haaland, K. Y. (1984). The relationship of limb apraxia severity to motor and language deficits. Brain and Cognition, 3(3), 307316. Hanlon, R. E., Brown, J. W., & Gerstman, L. J. (1990). Enhancement of naming in nonfluent aphasia through gesture. Brain and Language, 38(2), 298314. Hart, J., Jr., Anand, R., Zoccol i, S., Maguire, M., Gamino, J., Tillman, G., et al. (2007). Neural substrates of semantic memory. Journal of the International Neuropsychological Society, 13(5), 865-880. Hauk, O., Johnsrude, I., & Pulvermuller, F. (2004). Somatotopic representation of act ion words in human motor and premotor cortex. Neuron, 41(2), 301 -307. Hauk, O., & Pulvermuller, F. (2004). Neurophysiological distinction of action words in the fronto-central cortex. Human Brain Mapping, 21 (3), 191 -201. Hazeltine, E., Ruthruff, E., & Remi ngton, R. W. (2006). The role of input and output modality pairings in dual -task performance:Evidence for content dependent central interference. Cognitive Psychology, 52 (4), 291345. Humphreys, G. W., & Forde, E. M. (2001). Hierarchies, similarity, and in teractivity in object recognition: "Category -specific" neuropsychological deficits. Behavioral and Brain Sciences, 24(3), 453-476. Kable, J. W., Lease -Spellmeyer, J., & Chatterjee, A. (2002). Neural substrates of action event knowledge. Journal of Cognitiv e Neuroscience, 14(5), 795 -805. Kahneman, D. (1973). Attention and effort Englewood Cliffs, NJ: Prentice Hall. Kellenbach, M. L., Brett, M., & Patterson, K. (2001). Large, colorful, or noisy? Attribute and modality -specific activations during retrieval o f perceptual attribute knowledge. Cognitive, Affective, & Behavioral Neuroscience, 1(3), 207-221. Kertesz, A., & Hooper, P. (1982). Praxis and language: The extent and variety of apraxia in aphasia. Neuropsychologia, 20 (3), 275286. Kim, M., & Thompson, C. K. (2000). Patterns of comprehension and production of nouns and verbs in agrammatism: Implications for lexical organization. Brain and Language, 74 (1), 1-25. Leiguarda, R. C., & Marsden, C. D. (2000). Limb apraxias: Higher order disorders of sensorimotor integration. Brain, 123, 860879.
89 Levin, B. (1993). English verb classes and alternations: A preliminary investigation. Chicago: University of Chicago Press. Logan, G. D., & Schulkind, M. D. (2000). Parallel memory retrieval in dual task situations: 1. Semantic memory. Journal of Experimental Psychology: Learning Memory, and Cognition, 26, 1072 -1090. MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109 (2), 163203. Mahon, B. Z., & Caramazza, A. (2005). The orchestration of the sensory motor systems: Clues from neuropsychology. Cognitive Neuropsychology, 22(3/4), 480494. Martin, A. (2007). The representation of object concepts in the brain. Annual Review of Psychology, 58 25 -45. Martin, A., & Chao, L. L. (2001). Semantic memory and the brain: Structure and processes. Current Opinion in Neurobiology, 11(2), 194201. Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L., & Ungerleider, L. G. (1995). Discrete cortical regions associated wit h knowledge of color and knowledge of action. Science, 270(5233), 102-105. McCann, C., & Edwards, S. (2002). Verb problems in fluent aphasia. Brain and Language, 83 42 -44. McLeod, P. (1977). A dual task response modality effect: Support for multiprocessor models of attention. Quarterly Journal of Experimental Psychology, 29(651667). Medler, D. A., Arnoldussen, A., Binder, J. R., & Seidenberg, M. S. (2005). The Wisconsin Perceptual Ratings Database. Retrieved October 12, 2008: http://www.neuro.mcw.edu/ratings Miceli, G., Silveri, M. C., Villa, G., & Caramazza, A. (1984). On the basis for the agrammatic's difficulty in producing main verbs. Cortex, 20(2), 207220. Miozzo, A., Soardi, M., & Cappa, S. F. (1994) Pure anomia with spared action naming due to a left temporal lesion. Neuropsychologia, 32(9), 1101-1109. Morsella, E. E. (2002). The motor components of semantic representation (Doctoral Dissertation, Columbia University). Retrieved August 4, 2008: http://proquest.umi.com/pqdweb?did=726451981&sid=5&Fmt=2&clientId=20179 &RQT=309&VName=PQD Myung, J. Y., Blumstein, S. E., & Sedivy, J. C. (2006). Playing on the typewriter, typing on the piano: Manipulation knowledge of objects. Cognition, 98 (3), 223 -243.
90 Navon, D., & Gopher, D. (1979). On the economy of the human information processing system. Psychological Review, 18 214255. Nazir, T. A., Boulenger, V., Roy, A., Silber, B., Jeannerod, M., & Paulignan, Y. (2008). Language induced motor perturbations during the execution of a reaching movement. Quarterly Journal of Experimental Psychology, 61(6), 933 -943. Norman, D., & Bobrow, D. (1975). On data-limited an d resource limited processing Journal of Cognitive Psychology, 7 44 60. Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 97113. Oliveri, M., Finocchiaro, C., Shapiro, K., Gangitano, M., C aramazza, A., & Pascual Leone, A. (2004). All talk and no action: A transcranial magnetic stimulation study of motor cortex activation during action word production. Journal of Cognitive Neuroscience, 16(3), 374381. Pashler, H. (1994). Dual -task interfere nce in simple tasks: Data and theory. Psychological Bulletin, 116(2), 220244. Perani, D., Cappa, S. F., Bettinardi, V., Bressi, S., Gorno -Tempini, M., Matarrese, M., et al. (1995). Different neural systems for the recognition of animals and manmade tools Neuroreport, 6(12), 1637 -1641. Pulvermuller, F., & Berthier, M. L. (2008). Aphasia therapy on a neuroscience basis. Aphasiology, 22(6), 563599. Pulvermuller, F., Harle, M., & Hummel, F. (2001). Walking or talking? Behavioral and neurophysiological correlates of action verb processing. Brain and Language, 78, 143-168. Pulvermuller, F., Hauk, O., Nikulin, V. V., & Ilmoniemi, R. J. (2005). Functional links between motor and language systems. European Journal of Neuroscience, 21, 793-797. Pulvermuller, F., M ohr, B., & Schleichert, H. (1999). Semantic or lexico-syntactic factors: What determines word -class specific activity in the human brain? Neuroscience Letters, 275(2), 8184. Pulvermuller, F., Shtyrov, Y., & Ilmoniemi, R. (2005). Brain signatures of meanin g access in action word recognition. Journal of Cognitive Neuroscience, 17(6), 884 892. Raymer, A. M., Singletary, F., Rodriguez, A., Ciampitti, M., Heilman, K. M., & Rothi, L. J. G. (2006). Effects of gesture+verbal treatment for noun and verb retrieval i n aphasia. Journal of the International Neuropsychological Society, 12(6), 867-882.
91 Rodriguez -Aranda, C., Waterloo, K., Sparr, S., & Sundet, K. (2006). Age -related psychomotor slowing as an important component of verbal fluency: Evidence from healthy indiv iduals and Alzheimer's patients. Journal of Neurology, 253 (11), 14141427. Rodriguez, A. D., Raymer, A. M., & Rothi, L. J. G. (2006). Effects of gesture plus verbal and semantic -phonologic treatments for verb retrieval in aphasia. Aphasiology, 20(2 -4), 286 -297. Rosci, C., Chiesa, V., Laiacona, M., & Capitani, E. (2003). Apraxia is not associated to a disproportionate naming impairment for manipulable objects. Brain and Cognition, 53 (2), 412415. Rothi, L. J. G., & Heilman, K. M. (1997). Apraxia: The neurops ychology of action Hove, UK: Psychology Press. Saccuman, M. C., Cappa, S. F., Bates, E. A., Arevalo, A., Della Rosa, P., Danna, M., et al. (2006). The impact of semantic reference on word class: An fMRI study of action and object naming. NeuroImage, 32(4) 1865 -1878. Schmidt, S. L., Oliveira, R. M., Krahe, T. E., & Filgueiras, C. C. (2000). The effects of hand preference and gender on finger tapping performance asymmetry by the use of an infra -red light measurement device. Neuropsychologia, 38(5), 529-534. Setola, P., & Reilly, R. G. (2005). Words in the brain's language: An experimental investigation. Brain and Language, 94 (3), 251 -259. Shapiro, K., & Caramazza, A. (2003). Grammatical processing of nouns and verbs in left frontal cortex? Neuropsychologia, 41 (9), 11891198. Shapiro, K., Shelton, J., & Caramazza, A. (2000). Grammatical class in lexical production and morphological processing: Evidence from a case of fluent aphasia. Cognitive Neuropsychology, 17(8), 665 -682. Simmons, W. K., Martin, A., & Barsa lou, L. W. (2005). Pictures of appetizing foods activate gustatory cortices for taste and reward. Cerebral Cortex, 15(10), 16021608. Simmons, W. K., Ramjee, V., Beauchamp, M. S., McRae, K., Martin, A., & Barsalou, L. W. (2007). A common neural substrate f or perceiving and knowing about color. Neuropsychologia, 45(12), 28022810. Strain, E., Patterson, K., & Seidenberg, M. S. (1995). Semantic effects in single word naming. Journal of Experimental Psychology. Learning, Memory, and Cognition, 21(5), 1140 -1154
92 Tettamanti, M., Buccino, G., Saccuman, M. C., Gallese, V., Danna, M., Scifo, P., et al. (2005). Listening to action -related sentences activates frontoparietal motor circuits. J Cogn Neurosci, 17(2), 273281. Tranel, D., Adolphs, R., Damasio, H., & Damas io, A. R. (2001). A neural basis for the retrieval of words for actions. Cognitive Neuropsychology, 18(7), 655 674. Tranel, D., Kemmerer, D., Adolphs, R., Damasio, H., & Damasio, A. R. (2003). Neural correlates of conceptual knowledge for actions. Cognitiv e Neuropsychology, 20(3 -6), 409 -432. Tyler, L. K., Stamatakis, E. A., Dick, E., Bright, P., Fletcher, P., & Moss, H. (2003). Objects and their actions: Evidence for a neurally distributed semantic system. NeuroImage, 18 542557. Van Hoof, K., & Van Strien J. W. (1997). Verbal -to manual and manual -to verbal dual task interference in left handed and right -handed adults. Percept Mot Skills, 85(2), 739746. van Schie, H. T., Toni, I., & Bekkering, H. (2006). Comparable mechanisms for action and language: Neur al systems behind intentions, goals, and means. Cortex, 42(4), 495498. Vigliocco, G., Warren, J., Siri, S., Arciuli, J., Scott, S., & Wise, R. (2006). The role of semantics and grammatical class in the neural representation of words. Cerebral Cortex, 16(1 2), 1790-1796. Vitali, P., Abutalebi, J., Tettamanti, M., Rowe, J., Scifo, P., Fazio, F., et al. (2005). Generating animal and tool names: An fMRI study of effective connectivity. Brain and Language, 93(1), 3245. Warrington, E. K., & Shallice, T. (1984). Category specific semantic impairments. Brain, 107, 829-854. Weiss, E. M., Ragland, J. D., Brensinger, C. M., Bilker, W. B., Deisenhammer, E. A., & Delazer, M. (2006). Sex differences in clustering and switching in verbal fluency tasks. Journal of the International Neuropsychological Society, 12(4), 502509. Wickens, C. D. (1984). Processing resources in attention. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 63102). New York, NY: Academic Press.
93 BIOGRAPHICAL SKETCH Amy R odriguez is a rehabilitation scientist and certified speechlanguage pathologist. Amy received her Bachelor of Science in s peech-l anguage p athology from West Virginia University (Morgantown, WV) in 1998 and her Master of Arts in c ommunicative d isorders from the University of Central Florida (Orlando, FL) in 2000. From 20002001, she worked as a clinical fellow in speechlanguage pathology at Shands Hospital at the University of Florida in Gainesville, FL. During that time, she completed 18 additional hours of coursework and earned a Certificate in Medical SpeechLanguage Pathology. From 2001-2004, Amy worked as a research speechlanguage pathologist at the VARRD Brain Rehabilitation Research Center of Excellence at the Malcom Randall VA Medical Center in Gai nesville, FL. In August 2004 Amy returned to school to pursue her doctoral degree in Rehabilitation Science at the University of Florida. Her doctoral work was partially supported by a Grinter Fellowship and two scholarships from the American SpeechLangu age-Hearing Foundation. She was also funded as a research assistant by the VA Brain Rehabilitation Research Center a nd a U niversity of F lorida Opportunity Fund grant awarded to her mentor Dr. Jay Rosenbek. Amy has accepted a postdoctoral fellow ship at the Clinical Centre for Research Excellence in Aphasia Rehabilitation at the University of Queensland in Brisbane, Australia.