1 PERFORMATIVE AGENCY IN ACOUSMATIC MUSIC By SEUNG HYE KIM A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIV ERSITY OF FLORIDA 2012
2 2012 Seung Hye Kim
3 To Maxwell and my parents
4 ACKNOWLEDGMENTS I thank Dr. James Paul Sain for his support and guidance throughout my doctoral study and dissertation process I a lso thank Dr. Paul Richards, Dr. Kevin Orr, and Prof. parents and husband for all of their love.
5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 7 LIST OF FIGURES ................................ ................................ ................................ .......... 8 LIST OF OBJECTS ................................ ................................ ................................ ....... 10 DEFINITION OF TERMS ................................ ................................ .............................. 11 ABSTRACT ................................ ................................ ................................ ................... 14 CHAPTER 1 INTRODUCTION TO ACOUSMATIC MUSIC AND THE SENSE OF PERFORMATIVE AGENCY ................................ ................................ ................... 16 Introduction ................................ ................................ ................................ ............. 16 The Tradition of Acousmatic Music ................................ ................................ ......... 16 Phenomenology of Perception ................................ ................................ ......... 17 Sound Object and Reduced Listening ................................ .............................. 18 Performative Agency ................................ ................................ ............................... 20 Causal Aspect ................................ ................................ ................................ .. 21 Expressive Aspect ................................ ................................ ............................ 23 A Way of Listening Employing an Imaginary Performer ................................ .......... 25 Preview of Chapte rs ................................ ................................ ......................... 26 Delimitation ................................ ................................ ................................ ....... 28 2 EMBODIED APPROACH ................................ ................................ ........................ 29 Introduction ................................ ................................ ................................ ............. 29 Embodied Paradigm ................................ ................................ ............................... 30 Ecological Theory of Perception ................................ ................................ ....... 32 Direct perc eption ................................ ................................ ........................ 32 Action and perception ................................ ................................ ................ 33 Affordances ................................ ................................ ................................ 33 Ecological lis tening ................................ ................................ .................... 35 Embodied Music Cognition ................................ ................................ ............... 36 Body and mind interaction in music cognition ................................ ............ 37 Multimodal network ................................ ................................ .................... 40 Embodied reasoning based on experience and conceptual metaphor ....... 45 Embodied Approach to Acousmatic Music ................................ .............................. 47 Ecological Listening Strategy for Acousmatic Music ................................ ........ 50 Indicative Relationship and Gestural Surrogacy ................................ ............... 52
6 3 PERFORMATIVE GESTURE AS MANIFESTATION OF MUSICAL INTERPRETATION AND THE INTERNAL COGNITION METHOD ....................... 56 Introduction ................................ ................................ ................................ ............. 56 Definition of Gesture ................................ ................................ ............................... 56 Co expressive Gesture ................................ ................................ ..................... 59 Gesture in Speech ................................ ................................ ............................ 60 Musical Gesture ................................ ................................ ............................... 63 Performative Gesture ................................ ................................ .............................. 68 Quaternary Model of Musi cal Gesture ................................ .............................. 68 Expressiveness of Music and Performative Gesture ................................ ........ 71 Irregularity ................................ ................................ ................................ .. 71 Notation and sound ................................ ................................ .................... 72 Expressive timing ................................ ................................ ....................... 74 Human nature and gesture ................................ ................................ ........ 77 Performative Gesture as a Corporeal Interpretation ................................ ......... 79 4 ILLUSTRATION OF PERFORMATIVE AGENCY IN ACOUSMATIC MUSIC ......... 84 Introduction ................................ ................................ ................................ ............. 84 The Corporeal Interpretation of Gestures in PianoForte ................................ ......... 84 Types of Gesture ................................ ................................ .............................. 87 Iterative gestures ................................ ................................ ....................... 87 Stretching gestures ................................ ................................ .................... 89 Configuration of gestures ................................ ................................ ................. 90 Combination of successive gestures ................................ .......................... 91 Juxtaposition ................................ ................................ .............................. 95 Other Relationships with Timbre, Frequency, and Texture ............................... 95 The Relationship with Performative Agency ................................ ..................... 96 Detailed Listening of PianoForte ................................ ................................ ............. 9 7 Overall Structure ................................ ................................ .............................. 97 Introduction (0:00 to 1:48) ................................ ................................ ................ 98 Bridge (1:48 to 2:19 ) ................................ ................................ ....................... 102 Development (2:19 to 4:27) ................................ ................................ ............ 103 Turn (4:27 to 6:33) ................................ ................................ .......................... 105 Con clusion (6:33 to 8:32) ................................ ................................ ............... 106 Discussion ................................ ................................ ................................ ............ 106 5 CONCLUSION ................................ ................................ ................................ ...... 109 APPENDIX COMPOSITION: DISPLACED ................................ ................................ .................... 114 LIST OF REFERENCES ................................ ................................ ............................. 115 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 121
7 LIST OF TABLES T able page 2 1 The human senses involved in musical activity ................................ .................. 41 4 1 The four part structure of PianoForte. ................................ ................................ 98
8 LIST OF FIGURES Figure page 2 1 The flow of notions on the human body and the environment ............................ 31 2 2 reaction cycle for instrument making .................... 38 2 3 An illustration of action imagination reaction cycle for musical practice. Musical imagery is involved in the Ju dge and Change steps. In the context of musical practice, Change refers to adjustment that will be applied to the ................................ ..................... 40 2 4 The action sound chain, ad apted from Jensenius, A. 2007. Action sound: Developing methods and tools to study music related body movement (page 24, Figure 3.2). University of Oslo. ................................ ................................ ..... 41 2 5 An example of the connection be tween movement and sound .......................... 44 3 1 Three categories of gesture: gesture for control, gesture for communication, and gesture as metaphor. Note that gesture for control performs an operating functi on while gesture for communication and gesture as metaphor are involved in expression of meaning. ................................ ................................ ..... 59 3 2 The types of gesture arranged based on the level of indicative or expressive aspect. Me taphorics are highly subjective and barely indicative whereas emblems are highly indicative and hardly subjective. ................................ ......... 62 3 3 The communication of an abstract idea from the speaker to observer, in mind is expressed by gesture as well as speech. ................................ ............... 63 3 4 The functions of musical gesture arranged based on the level of indicative or expressive qualities. Sound producing gestures are highly subjective and communicative gestures are highly indicative of cues rather than subjective. .... 67 3 5 Quaternary model of musical gesture ................................ ................................ 69 3 6 The rendition of a ritardando varies depending on the performers. (A) The notation of the last two measures of Prelude Op. 28 No. 15 by Chopin, (B) the various paces ove r the bass notes of the ritardando created by Vladimir Horowits, Martha Argerich, Valentina Igoshina, and Ferruccio Busoni. .............. 76 3 7 Realization of a musical idea as the actual sounds by perform ative gestures, the corporeal articulations of the musical idea ................................ .................... 80 3 8 An illustration of the multimodal nature of musical representation ...................... 82
9 4 1 A perceptual framework suggested for PianoForte based on the timbre and gesture as well as their relations to the piano ................................ ..................... 86 4 2 Two types of iterative gestures based on the dyn amic shape. A) Increasing iterative gesture. B) Decreasing iterative gesture. ................................ .............. 88 4 3 The variations of stretching gesture representing the tendency of motion. A) Slight rise. B) Drastic rise. C) Slope. D) Peak. E) Fluctuating gesture. ............... 90 4 4 A basic combination of the stretching and iterative gestures: the energy restrained in the steady portion of the stretching gesture is increas ed then released through the active movement of the iterative gesture. ......................... 92 4 5 Various energy profiles of the combined gestures perceived as the results of different factors, such as the causal force, the material of objects and surfaces, etc. ................................ ................................ ................................ ...... 93 4 6 Juxtapositions of stretching and iterative gestures. A) Rising energy in both stretching and approaching iterative gestures. B) Stretc hing gesture with a alope and its inner movement created by iterative gesture. ............................... 95 4 7 A) The layout of the gestures in the last phrase of the introduction, in which the iterative gesture is de veloped with various timbres and paces. B) The overall energy motion shape of this phrase is similar to a decreasing iterative gesture. ................................ ................................ ................................ ............ 100 4 8 The five phrase structure of the introduction of PianoForte on a timeline. The dotted line illustrates the tension accumulated through the stretching gesture and released through the activity of the iterative gesture in the last phrase. The increasing activity and duration of the phrases create an energ y shape similar to that of the basic combination of stretching and iterative gestures. .... 101 4 9 The breathing like pattern of the bridge is reminiscent of the Introduction ....... 103 4 10 The overall dynamic shapes of the introduction and the bridge and development combined ................................ ................................ .................... 106
10 LIST OF OBJECTS Object page A 1 Displaced (.mp3 file 29MB) ................................ ................................ .............. 114
11 DEFINITION OF TERMS Acousmatic Listening to a sound without seeing the sources behind it. Acousmatic music Electro ac oustic music that is composed for fixed media only and played through loudspeakers Action trajectory Used by Gody (1997), to denote a temporal spatial domain providing both procedural proprioceptive and visual components. Affordances Coined by Gibson (197 9), referring to the meaning or quality of an object that is selectively perceived depending on the Corporeal interpreta t ion See Embodied resynthesis. Ecological psychology based on the view that humans, as organisms in the ecological system, directly perceive meaning through interaction with their environment Embodied music cognition The study of music cognition that regards s bodily experience in musical activities as constr perceptual framework of music cognition. Embodied resynthesis Coined by Leman (2008) referring to the corporeal aspect of human perception, in which important aspects of the outer world, particularly music, would in fact be captured in terms of actions. Exteroception Perception of stimuli that are external to an organism Gestural sonorous object Coined by Gody (2006), in order to emphasize the gestural quality represented in the features of a sound object. Gestural surrogacy Coined by Sm alley (1 996), referring to new type s of gestures in acousmatic music that can be perceived as being related, either closely or remotely, to the basic sounding model (i.e. human voice and instruments). Indicative field Coined by Smalley (1996), referring to the references of sounds to the human experiences.
12 Indicative relationship Coined by Smalley (1996), referring to the relationship between sounds and the human experiences. Kinesthesia The sensation of joint movement, either from internal forces (acti ve) or external forces (passive). Kinesphere Coined by Laban (1963), referring to a mental construct of space that one is always aware of in the interaction with the environment and with others. Multimodality Referring to multimodal perceptions that occu r simultaneously. Music concrte A genre of electroacoustic music, founded by Schaeffer, which utilizes recorded sounds as material for musical composition. Musical gesture The combination of both physical gesture and musical sound that represents the en ergetic shaping of motion through time as suggested by Leman and Gody (2010 ) Musical imagery of its source, 2) the ability to imagine a musical imagery. Performative gestures A th at play a critical part in transforming musical idea to expressive sounds in musical performances. Proprioception Reduced listening A manner of listening to sound objects, suggested by Schaeffer, which f ocuses only on the pure acoustic qualities of a sound while disregarding any physical references of the sound. Semiotics The study of signs and symbols, which allows the separation of a sign into a signifier (the object itself) and a signified (a possible meaning derived from the object). Sound object A brief fragment of recorded soun d, which can be inspected thoroughly and sculpted for musical composition.
13 Spectromorphology Coined by Smalley (1997), referring to sound spectra ( spectro ) and the way they change and are shaped through time ( morphology ).
14 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy PERFORMATIVE AGEN CY IN ACOUSMATIC MUSIC By Seung Hye Kim December 2012 Chair: James Paul Sain Major: Music The separation of sound from sight in acousmatic music means that there is an absence of a physical source and causation behind a sound. Nevertheless, listeners ca n conjure up an image of a performative agent who could be playing the work, which is implied by gestural components in acousmatic music. This is because humans perceive a strong link between sounds and physical body movements. In fact, performing, composi ng, and listening all involve a networked multimodal perceptual system that is strongly based on the human body. This study reveals how perception and musical experience can performative agency that guide their listening. The study begins with a review of the traditional concept of acousmatic music based o however, that the gestural components of sounds give rise to the sense of performative agency, as opposed t o the assertion of early acousmatic composers such as Schaeffer. This study introduces the theory of embodied music cognition which supports ideas about the importance of bodily movement in music perception, followed by an exploration of research that dis cusses the inseparable relationship between acousmatic music and the human body. After
15 establishing the link between sound study continues to investigate the practice of embodiment that takes place in musical performanc e, expression and musical gestures. The definition and expressive quality of performative gestures are discussed to elucidate the idea of human quality in performative agency. A n analysis is then presented of PianoForte by Yu Chung Tseng thro ugh the embodied perspective to demonstrate how an imaginary performing agent is minds and contributes to their understandings of the piece
16 CHAPTER 1 INTRODUCTION TO ACO USMATIC MUSIC AND THE SENSE OF PERFORMATIVE AGENCY Introduction Acousmatic music is directly projected to the listener without relying on a and musical grounded in his or her phy sical experience of musical activities, constructing a meaning tied to the physicality of the human body. The listener may attempt to identify, or imagine, the physical sources and agents of the sounds while listening to acousmatic music. This chapter outl ines the traditional theory of acousmatic music, in which a sound is perceived for its acoustic qualities only, while its visual, physical, or cultural re ference is disregarded. N evertheless, it is shown that the sense of performative agency emerges in the the sounds, which are inevitably implied. The Tradition of Acousmatic Music Acousmatic music originated from music concrte which, developed by Pierre Schaeffer (1910 1995), utilizes recor ded sounds as musical material. The development of recording technology in post war Europe made it possible to capture ephemeral sounds on a magnetic tape. The recorded sounds can be replayed, modified, and even adapted for musical compositions. Any record ed sound can be used for music concrte just as any material such as fabric, metal, glass, etc. can be used for a collage. The term music concrte is derived from the view that regards a sound as a concrete object. Audio recording and processing technolog y paved the way for the use of extensive sound material in musical composition, ranging from the sound of a musical instrument or voice to sounds of indu stry, nature, or everyday life. Composers
17 therefore, were able to manipulate the actual sounds instead of writing a score and create the fixed performance of compositions without musical interpretation and technical ability. Phenomenology of Perception Incorporating no ntraditional sound and noise into a musical composition enge phenomenological view of perception From view of a perceptual process an ob We start by taking an example. Keeping this table steadily in view as I go around it, changing my pos ition in space all the time, I have continually the consciousness of the bodily presence out there of this one and self same table, which in itself remains unchanged. But the perception of the table is one that changes continuously; it is a continuum of ch anging perceptions. I close my eyes. My other senses are inactive in relation to the table. I have now no perception of it. I open my eyes, and the perception returns. The perception? Let us be more accurate. Under no circumstance does it return to me indi vidually as the same. Only the table is the same, known as identical through the synthetic consciousness which connects the new the steady flow of consciousness, and is itself constantly in flux; the perceptual now is ever passing over into the adjacent consciousness of the just past, a new now simultaneously gleams forth, and so on. The necessarily transc endent to the perception. ( as cited in Kane, 2007, p. 16) The perception of an object is accomplished, not by only a glimpse, but through observations that produce a series of diverse perspectives of the object, which Husserl ( as cited in Kane, 2007, p. 16). The perceptual focus is placed on the reduction down the shape, color, and material of the object made of a flat top and legs) from its physical or cultural
18 existence (e.g. a tabl e ). Since these adumbrations do not present a consistent image of an object, the subject needs to consciously grasp the essential quality that holds the through an act o f consciousness, a synthesizing together of the stream of as cited in Kane, 2007, p. 16) James Edie (1987) points out that for Husserl, the essential meaning of a which is achieved by me ans of also (p.4). This view represents the idea of intentionality which Edie identically the same through a multiplicity of different acts of consciousness, and (2) the active and selective operation of constituting objects object is an intentional object in the sense that the object becomes a correlate to derive my part, the intentional object is no longe r bound to any particular spatio temporal Sound Object and Reduced Listening emphasized with the concept of sound object and reduced listening, logy: A sound is observed as an object and its essential meaning is revealed through a series of perceptions achieved through repetitive and progressive listening. The term sound object refers to a brief fragment of sonic material extracted from a recorded sound. Reduced listening, also known as acousmatic listening, is an objective listening manner
19 in which the properties of a sound object are investigated and sculpted for musical composition hence it is a musical object In reduced listening, a sound obje ct is and cultural existe nce of its sounding source. Additionally the focus of each listening shifts among the different acoustic parameters therefore producing a r ich perception of a sound object. Schaeffer argues the sound object is an intentional object in the sense that the sound object is recognized as a distinct concept, which is transcendent of its ety of acts of The focus of reduced listening is on the acoustic features of a sound object, while ound is to be perceived in terms of pitch, timbre, rhythm, and how they progress, while ignoring the type of bird (e.g. skylark or pigeon), its location (e.g. the seashore or the zoo), its purpose (e.g. mating or alarming), and so forth. Michel Chion al so states that reduced listening disregards any non auditory quality or signification related to a sound: attitude of listening which consists of listening to the sound for itself as sound object abstracting its origin, real or imaginary, and the meaning of which it can be carrier. ( as cited in Emmerson 1998, p. 136 ) In fact, it is logical to limit the listening focus on the sound itself since a sound object achieved from a recorded sound, exists as disembodied from its physical origin. Marc Battier (20 concentrated, and perception converges better than that, it is reduced to that pure acous acousmatic
20 denotes the aspect of listening in which any physical therefore visual, existence behind a sound is disregarded (Kane, 2007; Cox & Warner, 200 4 pupils were only allowed to listen to a lecture without seeing their teacher, who was hidden behind a curtain. The pupils were not distracted by presence; therefore they understood the lecture better concentrating on only the lecture Rec and sound objects, in which the visual source is absent, Schaeffer used the term acousmatic listening synonymously with reduced listening. Performative Agency Performative agency refers to the sensation in which musical sounds are perceived as being performed by a human performer Recognizing the gestural quality in musical sounds, listeners can conjure up an image of a human body as an agent who could create these sounds. Rosemary subconsciously, imagines a plausible sound source (such as a human body) whose 280). Watt and Ash (1998) conducted experim ents in which subjects were asked to choose the word that better described the music excerpt they just heard, out of two given words that were related to psychological aspects of people. The result of Watt and c is perceived as if it were a person Depending on the type of sound, the sound can be perceived as resulting from a non human agent. There are certainly those sounds w hose gestures do not clearly sound human in origin. For example, a chirping sound is perceived as being generated by a bird, regardless of its gestural quality
21 to imagine a human body as a performative agent i s related to the longtime t radition of musical performance For thousands of years, musical performance has been the fundamental way in which musical meaning is expressed, communicated, and interpreted. A performer is perceived as the causal agent whose ge stures create performance have defined the scope of musical composition as well. Some of the verbal annotations in musical scores reflect the fact that a musical idea is communi cated via a agitato (agitated), agile (swiftly), affannato (anguished), en pressant (hurrying forward), animato (animated), irato (angrily), amoroso (loving), spiritoso (spiritedly), etc. Although the sound of a cousmatic music is abstracted from its causality, the gestural qualities of sounds can be perceived as representing, or resulting from, a performs the sounds. Once a performa tive agent of music is perceived, the expressive qualities of musical features (e.g. gesture and timbre) and their deviational progressions can be attributed to the musical, emotional intention of a human. It is plausible to argue that the listener can co njure up the sense of performative agenc y regarding the causal and expressive aspect s of sounds While the causal aspect is perceived by the strong link between sound and causal action, the expressive intention perceived in a sound makes the perceived perf orming agent to be identifiable as a human body. Causal Aspect A sound can be perceived as resulting from physical actions of a human or non human.
22 listeners identify agency with the cause of the changes for the listener to catch the gestural aspect of a sound despite their effort to focus only on its abstract q ualities on the principle of reduced listening: It proves very difficult to hear sound only in terms of an appreciation of its shape and spectral properties as Schaeffer seemed to advocate. Just as a Pollock painting might address deep archetypes of form a nd feeling within listen to any sound conditioned by our primeval past and evolution. (Emmerson, 1998, p. 136) Perceived gestures in a sound give rise to the sense of agency, a causal re ference of a sound. Katharine Norman (2004) also illustrates that she, as the listener, is naturally captivated by the causal implication of sounds: I immediately recognize the kinds of things that might have produced those sounds. I feel that I know them. Nearly all the sounds imply actions. Recognizing the action component of a sound, the listener attempts to envision an imaginary person playing the sound and the way his or her bodily movements what type of action is involved in making the sound? Yes, there is somebody the re; I can hear the trickling sounds of pebbles or shards of metal falling through her fingers onto a surface and in particular, about in a container full of densely packed grains or perhaps rice, or tiny pebbles, or even sand. (Norman, 2004, p. 9) It is remarkable that a listener can create the corporeal interpretation of sounds that, in
23 co nsiders that acousmatic music, being transmitted by loudspeakers, is based on the absence of any physical and visual sensations related to the sounds. E xpressive Aspect A human body can be perceived as creating a sound with expressive, emotional, and music al intentions. Musical sounds involve a wide range of refined expressions that emotion is used to denote a psychological state, as Anthony Newcomb (1997) illustrates: I should qualify here th uncontroversial meaning: the tendency to certain kinds of behavior o ver time (as in irritability, excitability, fearfulness, volatility, aggressivity, passiveness, confidence, and so on) even an evolving pattern of emotional or psychological states over time (for example, growing confidence). (Newcomb, 1997, p. 142) The sense of human agency is differentiated from the natural or machinery causes ( e.g the wind causing a tree to sway or a coffee maker causing the water to drip ) by the emotional and expressive quality in the resulting gesture or sound The emotional qualit y that gives rise to the sense of performative agency can be elucidated by a comparison between two electroacoustic compositions, Idle Chatter by Paul Lansky and Out of Breath by Paul Koonce. Both pieces employ sounds directly produced from the huma n body (i.e. the voice in Idle Chatter and the breath in Out of Breath ) therefore providing the immediate sensation of a human body. However, their compositional approaches to the sound source represent two contrasting ways of performance an algorithmic proces sing of a human voice ( Idle Chatter ) and a psychological aspect ( Out of Breath ).
24 Idle Chatter employs brief phonemic fragments as sound sources. The use of voice strongly implies a hu world and, therefore, even the most passive listener can not fail to recognize the virtual human presence. The vocal sound is synthesized and performed by an algorithm designed by the composer utilizing Linear Prediction Coding, granular synthesis, and spatial distribution. This is a unique reading or performance of the sound and, at the same time, is entirely divorced from both the physical and emotional aspects of a living human being. The diversely processed vocal sounds are l ayered to create a virtual chorus that is continuously In contrast to Idle Chatter, Out of Breath brings the concept of an embodied musical practice into the core of the piece; together in order to achieve the desired sound quality. This piece employs three sound sources: breath, flute sound s and airy noise s around the embouchure For an untrained listener, this piece may sound like endles s repetitions of a single note. However, an attentive listener will be able to recognize the person in the music who is highly concentrating on playing the flute and creating slight or radical changes in the timbre. Without creating any complexity of textu re, pitch, or gesture, Out of Breath establishes the clear picture of a performer engaged in musical practice, therefore allowing a breathe along with the performer in this virtual performance. Out of Breath represents a psychological aspect of a human performer, who is striving to create an expressive sound and struggling at every moment for the perfection of the performance. A strong sense of performative agency emerges as the piece
25 unfolds. The piece gradually reveals the interactive relationship between the performer influences his or her perception and adjustment for the next play, su ch as air pressure, create the ideal sound is implied by the unsteadiness or imperfection of spectral structure, vibrato rate, breathing pace, and pitch production. In addition the amplified sounds of breath and airy noise, which are rarely audible for a listener in reality, are sounds that only the performer can perceive during a performance, therefore bringing forward a A Way of Listening Employing an Imaginary Performer physical origin and focus on ly on the sound itself: its pure acoustic quality. Acousmatic music is composed of sounds whose source and cause can be unclear or unidentifiable. The connection between a sound and its source becomes even more ambiguous as the sound is adjusted and arrang ed with other sounds in musical composition. The various manipulating techniques, such as looping, pitch transposition, time compression, etc., obscuring its identity. Also, the presentation of multiple sounds can create varying texture, which can be perceived as a cluster of abstract sounds rather than an individual sound. The features of acousmatic sounds such as timbre, pitch, intensity, and their dynamic progressions, are quite abstract and distanced from p hys ical origin. However, listeners may intuitively seek the cause of a sound and conjure up the image of a performative agent whose actions could have created the sound and its gesture The principle of reduced listening is
26 challenged by this natural tende ncy of listeners to identify even a slight connection between a sound and the physical causation behind the sound. In fact, Schaeffer himself distinguished the four listening modes available to listeners, including reduced listening and he admitted that a listener c ould adapt other ways of listening: Nothing can prevent a listener from making it waver, passing unconsciously from one system to another, or from a reduced listening to a listening which is not. One can even be pleased with that. It is by such a swirl of intentions that the connections are established, that information is exchanged. ( as cited in Gody 2006, p. 151 ) ements, it can be argued that the listener can intentionally employ an imaginary human performer for understanding ac ousmatic sounds expertise, the listener can perceive the expressive quality of sounds and gestures as and emotional intention. This way of listening can provide a link between the tradition of musical performance and the disembodied listening environment of acousmatic music. In additi on, those listeners who are musically trained but are not familiar with acousmatic music can benefit from this way of listening by engaging their experience of musical performance with acousmatic music listening Preview of Chapters Chapter 2 reviews the embodied paradigm as the critical basis of human perception. The concept of embodied music cognition, in which the strong link between action and perception is emphasized supports the causal aspect of performative age ncy It is revealed that one conjures up the image of a human
27 activity. From t he viewpoint of the embodied music cognition the principle of acousmatic music is re evaluate d and a flexible listening method for acousmatic music is embraced, in which listeners are encouraged to actively and creatively engage in constructing musical meaning employing their own experience and knowledge of the world Chapter 3 focuses on musical performance s and a human performer argued that the expressiveness of music is related to the nature of human beings With an overview of general gesture clas sification, the focus moves toward the expressive, metaphoric gestures created by a human performer during a musical performance, which will be called performative gestures Performative gestures as the bridge between musical idea s and actual sounds, repr interpretations of musical ideas and support cognition of musical ideas by utilizing the multidimensional sensations of sounds Therefore, it is argued that the listener can easily conjure up an imag e of a human performative agent by perceiving the expressive quality of gestures in acousmatic sounds. of acousmatic sounds and per formative agency in acousmatic music. The musical example, PianoForte by Yu Chung Tseng is analyzed focusing on the characteristic gestures and their variations as well as the resultant perception of performative agency Chapter 5 concludes the dissertat ion by providing a summary and further discussion.
28 Delimitation The subject of this study is the performative agency perceived in acousmatic music, mediated by loudspeakers in the absence of visual sources. Other forms of electroacoustic music that involv e any type of performer or live element, such as live electronic music using instruments, sensors, network or live coding, are excluded from the discussion. Soundscape works, although composed for fixed media, are excluded as well, because a sense of perfo rmative agency is rarely involved. Also, acousmatic music diffused in real time by the composer remains outside the scop e of the discussion, since it contains a live ele ment that can create different perception s from that of the fixed sounds.
29 CHAPTER 2 EM BODIED APPROACH Introduction Movement and sound are intertwined. Sound, despite its phenomenological and ephemeral nature, is also kinetic and physical. Caused by physical movement through sure fluctuations to which our barometric ears respond. Music, which employs sound as a medium, is inseparable from the kinetic nature of sounds since it consists of successions of sounds in movement; hence musical movement. The word embodied nd mind have been Varela 1993 to music cognition asserts that cognition of a meaning arises from the inseparable relationship between the body, mind, and mus ical sound. In this mind body unity, perception is directly and immediately connected to actions as well as the imaginations and simulations of actions. The fundamental aspect of musical experience is that perception and action occur nearly at the same tim e. One acts to create a musical sound and immediately perceives the resulting sound through the (e.g. auditory, visual, kinetic, etc.). Embodied percepti on of music results in an action oriented perspective of musical sounds, meaning music is understood in terms of bodily movement; hence corporeal interpretation According to Marc Leman (2008), this corporeal interpretation of music is referred to as embod ied resynthesis in the sense
30 embodied experience is the key to experiencing music and interpreting musical meaning, as demonstrated by the following statement: musical meaning is generated by our embodied experience of it that our embodied experience is not only necessary for experiencing meaning that is somehow inherent in the music itself, but that meaning arises in our conceptualizations of embodied musical experience and that abstract meaning is the product of embodied reasoning. (Cox, 2006, p. 45 46) This chapter reviews the general concepts of embodied perception and music cognit ion as well as how the embodied paradigm influences perception of acousmatic music and Embodied Paradigm In contrast with traditional theories that regard perception as a mental operation of sensory input, the embodied perspective suggests that humans perceive the world 2007, p.11), an embo died approach to cognition incorporates bodily sensation and experience as the basis for grasping meaning. of the idea of denotes that as cited in Kane, 2007, p. 17). This conce pt, in which the subject perceives the world within which they also exist, points toward the recognition ore obtaining various perceptions of the object In the theory of embodied perception, which recognizes the
31 body as an engaged component of the perceptual process, it is argued that meaning is Figure 2 1 illustrates how the cognition onent of the perceptual process. link between the action and resultant perception. E multimodal experience as the fundamental framework of musi c perception. A nd the rld for acousmatic music perception. Figure 2 1. The flow of notions on the human body and the environment
32 Ecological Theory of Perception Taking the body and its movement as the crucial factor of human perception, the embodied perspective shifts the cognitive focus from an abstract mental representation ntal processin visual perception places an emphasis on the interactive relationship between perception and action. Direct perception Gibson coined the term ecological psychology, based on the view that humans, as organisms in the ecological system, directly perceive meaning through interaction with their environment. He proposed that perception is the method by which all living organisms explore their surroundings and adopt the proper behaviors for survival. Gibson (1966) ar gues that the perception of the world, situated in the interactive relationship between humans and the environment, is not a cognitive mental is an ext the subject and object are situated together in the world and the subject explore s the is where the essential mea ning of an object is found. Husserl says that the essence, the intrinsic qualities. For Gibson, the essence of an object is directly picked up from a consistent, invariant feature, which reinforces itself through a flux of information. Eric
33 Clarke (2005) syas itself Action and percep tion In his book An Ecological Approach to Visual Perception Gibson (1979) states, in the head on the body resting on the n empirical studies conducted in laboratories focusing on how a subject can differentiate, classify, and organize the properties of an object, such as shape, color, size, texture, is the result a particular spot. Gibson states 1979, p. 1). While adjusting own movement and how it affects the changing perception of an object. Focusing on the movable body enabling the adjustm ibson places an emphasis 1979, lead to, and enhance, direct perception, and are in turn the result of, and response to, Affordances the subject is interested in the quality of an object that is important to him or her Gibson developed the term affordances to denote this selectively perceived quality of an object:
34 of the environment are what it offers the animal, what it provides or furn p. 127). Based on the close relationship between the action consequences of encountering perceptual information in the world For instance, while listening through a window to people screaming loudly on the street (auditory stimuli), one would imagine some people fighting on the street (the initial perception). One may walk to the window and look outside (adjusting for a different perspective), finding out that a woman was screaming at the man who just snatched her purse (additional perception). Then, one may close the window to block the noise, decide to see how it resolves, or call the police (the resultant action). A ccording to Gibson (1979), affordances are not solely what the environment offers an observer, but interdependent on both the environment and the observer This perception Gibson points out that exteroception perception of the outside world, coexists with proprioception An affordance, as I sai d, points two ways, to the environment and to the observer. So does the information to specify an affordance. But this does not in the least imply separate realms of consciousness and matter, a psychophysical dualism. It says only that the information to s pecify the utilities of the environment is accompanied by information to specify the observer himself, his body, legs, hands, and mouth. This is only to reemphasize that exteroception is accompanied by proprioception that to perceive the world is to cope rceive oneself. This is wholly inconsistent with dualism in any form, either mind matter dualism or mind body dualism. The are not separable (Gibson, 1979, p. 141). Affordances are al
35 11). One can also choose to alter a basic affordance of an object to fit in a different affordances, and these affordances are dependent on the individual, the culture, and the context within which th for padding the seats in a truck, blocking noise, etc., in addition to its basic affordance, an object to rest head on progresses can develop. Ecological listening Clark e (2005) proposes a way of listening called ecological listening, extending perception. In ecological listening, listeners can not only understand the sounds in the musical context but also pick up what is directly specified by sounds. Clark e argues musical meaning can be ip, coordinated working, persuasion, emotional catharsis, marching, foot tapping, and a myriad other activities of 2005, p. 38). He also points out that music has the potential to have invariant feature s th at, according to Gibso n, define the essence of auditory stimuli, such as a theme or motif. Although a theme of a piece of music may appear varied under transpositions and/or transformations throughout the piece, the identity is retained rring theme, the leitmotif is a representative example; it is clearly identifiable regardless of how it is modified in terms of rhythm, harmony, or orchestration.
36 Clarke draws an example in which multiple affordances emerge in various perspectives outsid e of musical context: an F major chord played on the piano may be 2005, p. 45 46). As another example in which musical sounds can be understood in various aspects simultaneously, Clarke discusses Jimi The Star Spangled Banner at the Woodstock festival in 1969. The Star Spangled Banner entails musical and extramusical affordances, including the National Anthem (N ationalism and convention) identified by its intervallic and rhythmic invariants, rock (musical genre) by its invariants such as electric guitar, distortion, etc., and Jimi Hendrix (African American virtuoso) by his improvisational embellishment. Clarke ar gues that various aspects identified in the performance, such as culture, ( 2005, p. 60 61). His approach suggests that, in the ecological view of listening, music is perceived not only in terms of musical materi als (e.g. melody, harmony, etc.) but also in terms of extramusical context reflected in the musical materials. Though the labels of ecological listening and acoustic ecology are similar, there is not meant to be any connection between these theories. Embodied Music Cognition Embodied cognition claims that experience of body and mind interaction during musical activ ities constructs his or her perceptual framework of music cognition. A sound and the causal action behind the sound are perceived together, therefore leading to the recognition of causal agency of the sound
37 Body and mind interaction in m usic c ognition The interdependent relationship between action and perception, as described by Gibson, is based on constant feedback from action to perception and vice versa (Hurley, 2001 & Leman, 2008). Action with a sounding object causes physical perceptions, such as audi tion, vision, etc. The body and mind work together closely, perceiving the sensation resulting from an action with an object. After the initial action and perception, one can modify the action, therefore reacting to the object and achieving an additional p object is established. Through this constant feedback between action and perception, cognition emerges and evolves. The sequence of action and perception is deliberately carried out in performin g or other sound making activities. The action reaction cycle suggested by Leman (2008) as a model for musical instrument making, is a good example of the iterative process between action and auditory perception ( F igure 2 2). In this cycle, one acts to c reate a musical sound (Play) and perceives the resultant resonance of an instrument (Listen). The sound quality is evaluated within the constraints of natural and cultural environments (Judge), and the property of the instrument is modified for optimizing future judgment (Change). The re played sound after adjustment is compared with the previous sound, to evaluate the improvement made by earlier adjustments. The re played sound may satisfy the natural and cultural requirements for the instrument. Otherwise additional changes may be applied to the instrument. This action reaction cycle is a process in which the initial sound is refined by constant adjustment until it becomes the final, desired sound.
38 Figure 2 react ion cycle for instrument making which was either previously perceived in the cycle or is aesthetically desired. The ability to imagine a sound in the absence of its source is called musical imagery (Repp, 2001 & Baker, 2001). The research on musical imagery is based on the notion that musical sounds can be recalled and re sound source (Gody, 2001). For example, musical imagery can b e used for sight reading or rehearsing without making actual sounds, for recalling a tune from memory, and for imagining the sound for a new composition. Gody further argues that an image 2 001, p. ix), by altering and developing the properties of previously perceived sounds. While composing a piece, a composer may imagine how the development of a musical idea will sound or how the timbre created from an unusual combination of instruments wil l sound. Musical imagery plays a crucial role in musical practice as a performer evaluates the immediate sound from an instrument (Repp, 2001). Varela, Thompson & Rosch (1993) discuss the way that performers also constantly undergo this progressive intera ction between perception and action until the musical imagery of ideal sound is realized as the actual sound, overcoming technical challenges: As one practices, the connection between intention and action becomes closer, until eventually the feeling of dif ference between them is almost entirely gone. One achieves a certain condition that phenomenologically
39 feels neither purely mental nor purely physical; it is rather a specific kind of mind body unity (Va rel a, Thompson & Rosch, 199 3 p. 29). In this state of mind body unity, embodiment, a musical intention is realized successfully through the use of the right technique, which involves physical control and comparative perceptions. The constant feedback between modified action and its resulting percepti on, a s well as the comparison between the current sound and musical imagery are the key to refining and reinforcing the quality of musical sound. Figure 2 3 illustrates an action reaction cycle involving musical imagery in musical practice: hence an action imag ination reaction cycle. In the action reaction cycle, Change refers to the change applied to a musical instrument. However, in this action imagination reaction next Play. Therefore, the difference between these two models is whether a change is imagery of a desired sound and compares it with the actual sound that is played. Then, the pe rformer adjusts his or her bodily movement for the next Play. It should be noted, however, that comparing the actual sounds with musical imagery is not successful all the time. Performers sometimes fail to hear the sound they are really playing while focus ing on the sound they are imagining. For this reason, Boris Berman (2000) argues objective ears will ideally lead to proper judgment about their performance, therefore resulting in the achievement of the desired sound.
40 Figure 2 3. An illus tration of action imagination reaction cycle for musical p racti ce. Musical imagery is involved in the Judge and Change steps. In the context of musical practice, Change refers to adjustment that will be applied to the tion). Multimodal network This embodied musical experience, or sound making experience, demonstrates the close link between a sound and its causal action as well as various bodily sensations (e.g. vision, touch, sense of force and direction, etc.) relate d to that action. Jensenius (2007) puts forth the concept of the action sound chain from cognitive process to sound in which multimodal sensations in all parts of the chain are directed to F igure 2 4). The motor component used in playing an in strument, for energy to an instrument as well as the adjustments of his or her hands, fingers, and wrists. The perception arises from this experience through the body and its se nsory organs: the tensed muscles before playing an attack sound and their relaxation afterward, the limb movement toward an instrument, the tactile sensation of fingers through which the vibration of the instrument is felt, the vision of all these phenomen a,
41 Figur e 2 4. The action sound chain, adapted from Jensenius A. 2007 Action sound: Developing methods and tools to study music related body movement (page 24, Figure 3.2). University o f Oslo. Multiple sensations involved in musical activity occur simultaneously and are linked closely sensory perceptions include sight, hearing, touch, smell, taste, and balance (Jenseniu s, 2007, p. 13). Among these sensory perceptions, modalities required the most for musical activity are visual audit ory tactile kinesthetic and proprioceptive modalities, as shown in T able 2 1. Table 2 1. The human senses involved in musical activity Modality Sensory perception Examples Visual Sight Vision of an instrument (e.g., keys, mallets, bows, etc.) Auditory Hearing Sound of an instrument (e.g., tone, noise, etc.) Tactile Touch Vibrations of strings, texture of keys, etc. Kinesthetic Forc e and movement Muscular tension from making vibrato, beating, finger movement for playing arpeggios, etc. Proprioceptive Body position and balance in relation to other objects ig leaps, etc. Visual and auditory perceptions can be achieved by seeing an instrument and listening to the sound of the instrument. Tactile perception is gained from the direct contact fingers on the string, key, valve, etc., or the air pressure on the lip. The tactile sensation can also be achieved from the indirect contact between on
42 mallets, bow, etc. Proprioception is often confused with kinesthesia body relative to the space where he or she is (Lephart & Fu, 2000, p. xxii). The following definition of ance Medicine & Science [IADMS] 2008) provides a good description of proprioception: Proprioception classical five senses to include the body. This body sense is more than just a feeling of movement, however. It is intimately tied to our feeling of muscle tone, perceptions of effort and of balanc e. Specialized nerve endings originate in our muscles, fascia, tendons, ligaments, joints, and some perceive deformation of tissue the amount of pressure (stretch or simply, placement ), speed at which movement is occurring and the rate at which the speed is changing (velocity), direction of movement, and when deformation is extreme pain (IADMS, 2008, p. 1). In musical performance scles for arpeggio phrase over the whole register or playing big leaps across a wide ra nge. of the multimodal perceptions related to the musical sound. Various sensations that are derived from the same event are inextricably connected with one another. Due to t his multimodal nature of musical perception and experience, perceiving an event in one modality can activate other sensations in different modalities related to said event, therefore causing cross modal perception. The cross modal perception between auditi on and vision is illustrated in the fact that one can imagine the vivid sound that is
43 absent in a silent film or envision the images related to a narrative aired in a radio broadcast. In terms of music cognition, James Baker (2001) argues that even a roug h simulation of the causal action can help imagine a sound, giving the example in which music students frequently gauge intervallic distance on imaginary instruments often a keyboard and string instruments, while solving dictation problems. Gody (1997) p laces an emphasis on the visual correlates between the sensations of movement and sound in his triangular model of cross idea is based on the notion of action trajectory which is a temporal spatial domain providing both procedural proprioceptive and visual components ( 1997 ). Therefore, a musical sound can easily be recalled by simulating the action component. For example, a musically trained listener can easily visualize a performance including the perfor gestures that create the sound by listening only to the sound. While listening to a bouncing off the keyboard immediately after playing the note on the first beat of mea sure 16 ( F igure 2 5). As seen in the score ( F igure 2 5), musical tension is built through increasing pitch and volume played by the right hand toward the note in measure 16 which is the loudest and highest one of the segment (mn. 9 16). The gesture, movin g the hands upward at the end of the segment is related to the momentum the pianist gained over the course of the segment While this arm gesture moving upward occurs almost simultaneously with the note in measure 16, a listener can also visualize the pia piano in order to prepare the following phrase in the next measure.
44 Figure 2 5. An example of the connection between movement and sound A listener with performing experience may recall his or her memory of seeing the gesture in a performance of this piece, or imagine this gesture to be made in composition and the performer who recorded it, he or she may also be able to imagine specific gestures similar to those from the actual performance. During a performance, it is also possible for a listener to predict the sound quality that is about to be produced by listener can hear the dampened sound of the vibraphone by looking at a performer who places a mallet on a bar and hit s the bar with another mallet. A listener normally perceives the sound and vision of a musical performance better than other modalities. H owever, a highly trained
45 listener can also imagine the tactile and proprioceptive sensations that the performer would be experiencing during the performance. Embodied reasoning based on experience and conceptual metaphor from mus ical performance and listening is engaged in understanding and conceptualizing the meaning of music. Listening to musical sounds, one can experience the bodily sensations related to the sounds by imagining or imitating the physical gestures that could crea te the sounds. Singing along with the song or p laying an air guitar or air dru m the listener can feel like they actually play the sounds to which they are listening. This shows that listeners experience and understand music through their embodied particip imitating the sounds heard and imitating the physical actions that produce these imitation of the sounds produced dynamic evident in the sound (a pattern of exertions that would produce the same or 2006, p. 50). For exampl e, one can imagine the finger movement over imagine and feel the exertion required for a fast chromatic ascending scale. David ion in music, the listener practices being in ly vibrating in sympathy with it a sound involves comprehension of one of the ways of understanding music al sounds is performing (action) or recalling and (imagery of action) of the music which brings
46 ily sensations related to the sounds and their causal actions In line with this view of music perception by fundamentally shaped by conceptual me taphors that are grounded in our bodily can be understood via metaphor sounds and actions. For example, a figure of ascendi ng pitch is conceived as going up higher. The pitch difference between two notes is conceived as a distance between two Johnson and Larso n demonstrate that t he perception of musical motion also involves conceptual metaphor descrip tions, such as the music moves something moves in the music etc. Johnson and Larson (2009 ) suggest the conceptual metaphor for m oving mu sic : a physical motion (musical motion) of an object (musical event) is created by physical forces (musical forces) in relation to an observer (listener) in the path of motion (musical passage). This is because combinati ons of various musical features ( e.g pit ch, dynamic, articulation, etc.) seeing physical objects in the world that move or are moved by external force. A listener may use his or her experience with objects in a physical space for u nderstanding the sense of moving objects in musical space. The metaphor of physical space and visual
47 (Johnson, 2007, p. 180). According to this conceptual metaphor by Johnson and Larson, sound creating gestures can be understood as physical forces that cause an object to move at a particular speed over a particular path. Cox (2006) points out that the l istener recognizes the intentional, expressive quality of those gestures, while experiencing the exertion of sound creating gestures: According to the mimetic hypothesis, we experi ence patterns of exertion as if we are acting in a way that is m ore or less isomorphic with the sound producing actions heard (and seen). In conceptualizing these patterns of exertion, we map these onto our own experience of making similar exertions, and among the most straightforward mapping is that onto gestures: we do not feel only abstract sensations of exertion; we also understand these as the intentional, expressive gestures that we have made and have seen made in other domains of experience. (p. 53) This suggests that one perceives sound creating gestures as inte ntional and expressive role of a character or persona 2006, p. 53). Embodied Approach to Acousmatic Music position for acousmatic music is different from that for traditional music for instruments or voice. In a traditional setting of musical performance, listeners perceive music while seeing a performer whose gestures cause the musical sound by the direct in teraction with the sounding source. In this traditional setting of musical are clear In acousmatic music, the perceptual link between the sound and its origin becomes uncertain. The coexisten ce of sounds from diverse contexts ( i.e. nature, culture, music, hu man, industry, electronic, etc.) obscures the boundary between real and imaginary. The
48 listeners of acousmatic music attempt to identify the connection between sounds and their origins. Th e listener may recognize a particular instrument from a sound, however, its timbral and gestural attributes may not be faithful to the instrument. For example, the listener recognizes a guitar sound that soon changes to something else: the resonance of plu cked guitar strings continues to vibrate for an extended time, its varying loudness gradually creates a rhythmic p attern, and its timbre also starts morphing into a string instrument ality does not help him or her understand what could cause these changes in the sound. This uncertainty of sound sources in acousmatic music increases when it comes to sounds that are electronically generated or heavily processed. With these highly abstrac t sounds, the listener may be confused by an unsufficient basis for understanding and interpreting these sounds. Listening to acousmatic music mediated by loudspeakers, the listener is in a position of seeking the possible origins of ambiguous sounds and i nterpreting the musical meaning through his or her own perceptual capacity and experience. The listener may easily focus on familiar sounds that can be related to objects and events in the real world. For ambiguous sounds, the listener may attempt to relat e them to his or her experience in the real world in order to find any slight connection to possible sources. When a sound has no connection to the real world, the listener may focus only on its pure timbral attributes in a musical context. This way of und erstanding musical sounds is similar to that of understanding a phrase of foreign language: grasping a few familiar words, inferring the meaning of other uncertain words, and then attempting to make sense of the whole phrase. The listener is encouraged to be flexible in employing
49 different focuses depending on the type of sound material and the context within which the sound is used. The listener may freely shift his or her focus from the physical attributes indicated by sound to the abstract acoustic quali ty of sounds as the sounds in a cousmatic music are often transformed in unpredictable way s From the viewpoint of the embodied approach, acousmatic music can be perceived with regards to human and physical references, and therefore, idea of the sonorous object and reduced listening is challenged. Gody (2006) argues that gestural quality is inevitably perceived in even brief, abstract sound objects, under the principle of reduced listening. Gody points out that Schaeffer also employed gestural concepts and metaphors for inspecting, evaluating, and classifying sound objects. In the typology of sound objects, for instance, Schaeffer identifies three types of envelopes of duration (e.g. impulsive, sustained, and iterative types) and links these du ration envelopes to executive gestures (e.g. punctual, conti nuous, and iterative gestures) Arguing that the gesture associated with a sound object should be considered in the inspection of a sound object, Gody coins the term gestural sonorous object : Th e sonorous object can be inspected, explored, and progressively differentiated with regards to features, features which often evolve or have various envelopes which can be traced, hence in my opinion actually becoming more like what I would call a gestural object. (Gody, 2006, p. 149) In reduced listening, one focuses on a sound their evolution, such as dynamic envelopes, frequency contours, textural ch anges, etc. Gody argues the listeners continuously trace these paramete rs of a sound by using their own body parts, such as therefore conjuring up a gestural image related to the sound ( 2006, p. 149):
50 This means that from continuous listening and continuous sound tracing, we actual ly recode musical sound into multimodal gestural sonorous images based on biomechanical constraints (what we imagine our bodies can do), hence into images that also have visual (kinematic) and motor (effort, p roprioceptive, etc.) components. ( 2006, p. 149) His argument suggests that, even under reduced listening that d enies any physical reference to and intention, based on the gestural quality of its acoustic features and their de velopment. Ecological Listening Strategy for Acousmatic Music Windsor (2000) proposes an ecological approach to the interpretation of acousmatic sounds. The basic claim is that, in acousmatic music, sounds provide 10). In an ecological framework, sounds are perceived as indicating a state or change of the environment. For example, a clock sound with a gradually increasing echo may inform a listener of a cha nge of acoustic s pace from a bath room to a cathedral. Windsor (2000) refers to studies (Spelke, 1976; 1979; Warren, Kim and Husney, 1987) that demonstrate the relationship between a sound and movement. For instance, the sound of an object can specify the movement patterns of the object (a bouncing pattern of a basketball), the energy motion transferred to the object (dribbled or thrown), and the internal structure of the object (basketball or golf ball). Depending on the s or her previous experiences, the listener could pick up the invariant features of a sound event and relate those features to a particular physical event. As the listener experiences various relationships between ual system will develop to easily recognize
51 occur in acousmatic music a listener will be able to perceive the events and objects which would normally and lawfully give 2000, p. 18). Clark e also showed how the various aspects of the environment are perceived in music. What makes this ecological listening different wh en it comes to acousmatic music is that acousmatic music can create an environment different from the environment in the real world. Windsor (2000) points out that the perception of acousmatic music can be drastically different from the perception of real events. The sonic environment of objects and events created in acousmatic music is perceived as being disconnected, extended, or transformed from the physical environment of objects and events in reality. In acousmatic music, an object does not offer an af fordance consistent with what the of an everyday kind, but s p. 22). This hybrid environment created in acousmatic music, therefo re, allows the listener to create his or her own interpretation. Windsor argues that the listener is challenged to identify what the sounds afford and signify. In the following statement Windsor emphasizes that the aesthetic of acousmatic music lies in th e interaction between the sounds of a virtual world and the interpretive, imaginative listener: The listener perceives that some of the sounds specify events or the manipulation or juxtaposition of sounds that should specify events but fail to do so. The a esthetic nature of the acousmatic piece lies in its position between the demands of everyday perception and its contradiction of the specificity which provides for a structured and relatively unambiguous relationship with the world. ( 2000, p. 23)
52 Indicati ve Relationship and Gestural Surrogacy Denis Smalley (1996) proposes the idea of the i ndicative relationship an information based listening that focuses on the physical occurrence, action, or object linked with a sound. The indicative relationship is orig inally taken from one of the Schaefferian listening modes, which Schaeffer himself considered insufficient for the sound of artistic music. Smalley argues, however, that the meaning of an acousmatic cal experience and our information the sound signifies ( 1996, p. 83): The apprehension of musical content and structure is linked to the world of experience outside the com position, not only to the wider context of auditory experience but also to non sounding experience. Approached from the multiple perspectives of life outside music, the materials and structure of a musical composition become the meeting place of sounding a nd non sounding experience. Electroacoustic music, through its extensive sounding repertory drawn from the entire sound field, reveals the richness and depth of indicative relationships more clearly and comprehensively than is possible with other musics. ( Smalley, 1996, p. 83) (1996) approach to acousmatic music through the multiple perspectives of life contributes to the embodied comprehension of music based on the indicative field that behavior, energy, motion, object, substance, environment, vision, and space (1996, p. 83). The experience in these fields is manifested in sound materials in musical context s sound can be viewed as having two parts: the acoustic quality of a sound and the non sounding experience attached to the sound. This is similar to semiotics, the study of
53 signs and symbols, which allows the separation of a sign into a signifier itself signified eld, 2000, p. 41). In a musical context, a sound itself is a signifier and the non sounding experience or meaning indicated by the sound is a signified. For example, when listening to a siren sound in acousmatic music, semiotic reasoning allows the listene r to be able to ied by a siren in reality ( e. g. emergency, nervousness, etc .) Therefore, the listener constitutes a meaning of the music incorporating both its musical and cultura l context s This division of a sound into acoustic and non acoustic features was also the basis of reduced allow the coexistence of a signifier (i.e. a sound) and a si gnified (experience or meaning attached to the sound) within a sound while a signified is disregarded in reduced listening. Acousmatic music can also involve exceedingly processed, abstract sounds that hardly indicate any human experience in the world. Sma lley proposes that the acoustic features of an extremely abstract sound can be analyzed and slight evidence of the indicative field can be found. Smalley (1997) coins the term spectromorphology which spect ro ) and the ways they change and are shaped through time ( morphology sonorous object, Smalley argues that the causal gesture be hind a sound can be spectromorphology Smalley views spectromorphology as the result of the interaction between source (sounding object) and cause (physical action):
54 Sound making gesture is concerned with human, physical activity which has spectromorphological consequence: a chain of activity links a cause to a source. A human agent produces spectromorphologies via the motion of gesture, using the sense of touch or an implement to apply energy to a sounding body A gesture is there fore an energy motion trajectory which excites the sounding body, creating spectromorphological life (Smalley 1997, p. 111). The recognition of an extramusical field signified by a sound provi des the listener with a basis from which the origin of sounds t hat are surrogated from a reality can be inferred. Smalley (1996) introduces the notion of gestural surrogacy existence of new types of sounds which are more remote from physical, gestural uman exper as the basic gesture model, and categorizes the gestures of acousmatic sounds into three groups (i.e. first order surrogacy, second order surrogacy, and remote surrogacy), depending on the degrees of remoteness fro m the basic model Smalley (1997) supposed sources and causes of sounds that are surrogated remotely from the ir physical origin (p. 112). Ten Hoopen (1994) suggests the specific surrogate continuum that illustrates the degree of surrogacy. A sound highly surrogated, hence alienated, from the original model of gesture is perceived a lmost as the result of a non sounding object and non physical gestures, therefore leading the listener to focus on the abstract timbral features surrogate continuum and th e sounding/non sounding analogy illustrates that the listener tends to perceive extremely surrogated sounds as metaphors of non sounding experiences:
55 we are inclined to appre hend the resulting music in terms of sounding analogies which include qualities of actual sounds, objects or sounding phenomena existing in the real world. Moving away from the origin we enter possible ambiguity. We apprehend sound material on the basis of some degree of resemblance of attributes of existing sounding phenomena. So, too, the sounding analogies become more ambiguous while still suggesting real world or human presence. Beyond this area of ambiguous recognition, we are inclined to comprehend sound material with reference to non experienced. (Ten Hoopen, 1994, p. 70) The origin and cause of soun d in acousmatic music can be ambiguous, depending on the degree to which a sound is disguised. The ambiguity of sounds allows meaning of acousmatic music rests on the Musical meaning is constituted by an individual listener, and there can be different interpretations between various listeners a s well.
56 CHAPTER 3 PERFORMATIVE GESTURE AS MANIFESTATION OF MUSICAL INTERPRETATION AND THE INTERNAL COGNITION METHOD Introduction The focus of this chapter is on musical performance and gesture by a human performer. It will be shown that the qualities o f sounds that conjure up a sense of performative agency are similar to the human qualities one perceives from a performer creating expressive sounds in musical performance. The cognitive interaction between action and perception is highlighted in musical p erformance more than in any other music related activities, including listening, conducting, dancing, etc. Therefore, a his or her musical interpretation and influenc es the expressive quality of musical performance. This chapter begins with the definition of gesture, focusing on the qualities and functions of various gestures involved in human expression. The discussion of expressive gesture leads to the notion of perf ormative gesture which functions as a corporeal bridge between a musical idea and its sounds. Looking into the expressive and cognitive aspect s of performative gesture, it is argued that the unsteady and imperfect qualities of human being s give rise to th e sense of a performative agent. With the evidence of human nature, the sense of a performative agent is differentiated from that of a causal agent. Definition of Gesture Gesture is a universal and intuitive way to create human expression. The Oxford Englis h Dictionary (2 nd ed., 1989) defines the word gesture or any part of it, that is ex In general, gesture is used
57 synonymously with movement, motion, and action to indicate any change of physical st ate. The broad range of definitions of gesture includes not only refined movements, fear, movements, such as cowering in fear Although both are expressive of the feeling fear the danc consciously created with particular intentions and meanings in the context of performance. Furthermore, the term gesture is also used to indicate a symbolic attitude, such as perceived in a metaphorical sense. Liwei Zhao (2001) points out that there are two types of approach to gesture research. One approach is concerned with a conceptual understanding o f gesture and its function, and is employed in the research of linguistics, psychology, neurology, choreography, physical therapy, etc. The other approach focuses on gesture in a system oriented context, including the fields of computer vision, human compu ter interaction, human motor control, and computer graphics and animation. Focusing on communication, cognitive psychology, and modeling. Combining these classifications by Zhao a nd McNeill, Jensenius and others present three divisions of gesture: gesture for control gesture as communication and gesture as metaphor (Jensenius, 2007; Jensenius Wanderley, Gody & Leman 2010). easurable functions, and an example of this gesture is the gesture oriented interaction between a human and computer dev ices and interfaces, for example, a touch pad. Gesture for control is mainly
58 used for operating a system, while the other gestures are i nvolved in human expression and perception. Jensenius (2007) points out that human computer communication, in contrast with human sensing capability. What the computer understands from a gesture is the input information indicating a certain command regardless of the way the gesture is expressed. For example, the result of pressing a button stays identical whether it is pressed heavily with an arm or lightly with a fingertip. What is important in h uman computer communication is that the button is pressed, not that the button is pressed aggressively or gently The idea of gesture as communication represents the creative and supportive role that gesture pl ays in human communication, for example, a ha nd gesture along with speech. The gesture created with speech or musical sound serves to accompany and/or reinforce the expression of an idea or meaning. When expressed with both gesture and speech, the idea is easier for listeners to understand, therefore improving communication between the speaker and listener. Gesture as metaphor indicates the cognitive aspect of gesture in which one perceives a stimulus, especially an abstract idea, as an image. The image perceived from an abstract idea is converted to in expressing an idea while one speak s or perform s An example can be found in the gestures that one makes seeking the prope r word for expressing a meaning. Figure 3 1
59 th at serves an operating function is distinguished from gesture as communication as well as gesture as metaphor that are in volved in expression of meaning. Figure 3 1. Three categories of gesture: gesture for control, gesture for communication, and gesture as metaphor. Note that gesture for control performs an operating function while gesture for communication and gesture as metaphor are involved in expression of meaning. Co expressive Gesture Gestures are used not only for accompanying the expressed idea but also for perceiving, hence concretizing, the idea during speech and musical performance. Thus, it can be suggested that gesture as communication and gesture as metaphor can be considered as a co expressive gesture. The following discussion focuses on the co expressive role of gesture in speech and musical performance. McNeill and Duncan (2000) define t he co expressive relationship between gesture and speech: The gesture and its synchronized co expressive speech express the same underlying idea unit but do not necessarily express identical aspects of it.
60 By looking at the speech and the gesture jointly, we are able to infer characteristics of this underlying ide a unit that may not be obvious from the speech alone. (McNeill and Duncan, 2000, p. 142 143) McNeill (2000, 2005) points out that although gesture and speech are two different modes of exp ression ( i.e. gest ure imagery and linguistic code) they are gesture and musical sound are also engaged with the same musical idea (Wanderlay and Vines, 2006). Speech and music al performance both use gesture to accompany the musical or verbal expression of an idea. Many researchers have suggested that speech are the two main forms of systematized in terhuman communication by means of sound similar functions of gesture in speech and musical performance, which is to accompany, reinforce and modify an idea or emotion. The only difference between gestures in speech and musical performance is that gesture in musical performance is also used as a direct method for creating the musical sound. Especially for instrument playing, the ure interacting with the instrument. Gesture in Speech out consciously and explicitly in ord er to be perceived by an observer (Kendon, 2004). expressive gesture is created in an expressive, deliberate way in contrast to an involuntary and operating gesture: edged intention to convey meaning are directly perceived as such as a
61 it has well defined boundaries of onset and offset, and if it has features which show that the movement is not made solely under the influence of gravity, then it is likely to be perceived as gestural. (Kendon, 2004 p. 14) Several types of gesture are suggested by combining the approaches of McNeill (1995) and Cassell (1994 & 1998): Iconics Metaphorics Beats Deictics and Emblems (Zhao 2001, p. 1). It can be argued that each type of gesture can communicate different degrees o f indicative/expressive quality, as illustrated i n Figure 3 2 Emblems refer to cultural ly established expressions, such as the goodbye or OK sign. Iconics represent a particular feature of an object by describing its shape, or illustrate the action by reproducing it in an empty space. A knocking movement with a hand whi le saying knocking on the door is an example of Iconics Emblems and I conics are clearly indicative and self explanatory. Deitics ind icate a specific direction, for example, pointing in a space while saying over there attention to where one points. Beats are hand movements occurring with specific words, highlighting their meaning. Beats storytelling. Depending on the narrator, different words are emphasized by paus ing cutting short, elongating, raising in pitch, etc. Metaphorics represent features of an obj ect or an abstract concept, for example, holding up the hands to refer to something while saying something happened These ges tures are M etaphoric s not because o f structural features but because of the relationship with what they repr esent. The same gesture can be Iconic or M etaphoric depending on the context within which the gesture the palms facing up can be an I conic gesture while saying empty hands However, it can also be a M etaphoric gesture while sayin g, I have nothing to sa y referring to nothing in the sentence. The expression of
62 Metaphorics varies with the individual s peaker. A speaker may also say I have nothing to say while waving their hands. In this case, waving their hands is a metaphoric gesture, ind icating nothing not goodbye Figure 3 2. The types of gesture arranged based on the level of indicative or expressive aspec t. Metaphorics are highly subjective and barely indicative whereas emblems are highly indicative and hardly subjective. Given this perspective, it can be suggested that beats and metaphorics are interpretation of an co mind provides visual attributes based on which gesture can be created Metaphorics are approximate rather than detailed, representations of abstract features. B y offering visual and spatial properties derived from the image of abstract features metaphoric gestures help listeners understand ideas better through s imple bodily interpretations
63 A benefit of using metaphoric gestures in speech is that one can present a meaning in terms of continuous movements in space and vision, contrary to the linguistic code that breaks up and represents a meaning in a linear stru cture (Goldin Meadow, 2003). For instance, an upward hand gesture to indicate high intelligence represents the level of intelligence by the visual instantiation of a vertical axis in a space. Thus, the af orementioned high intelligence is perceived as havin g a measurable physical quality. Figure 3 3 illustrates that an abstract concept is expressed by linguistic code as well as gesture, which is created based on the image perceived from the concept. Figure 3 3. The communication of an abstract idea from the speaker to observer, in mind is expressed by gesture as well as speech. Musical Gesture Influenced by embodied perspective, the term musical gesture is u sed to denote the combination of both physical gesture and musical sound that represents the energetic shaping of motion through time (Jensenius, 2007). Gody and Leman state, &
64 body plays a critical role in musical activity, whether performing or listening. Especially ment and the resulting sound are inextricable. Regarding the body and its movement as the essential technique of piano playing, Gyrgy (p. 4). In addition C lark sing, we must be aware of our body. We take a deep breath, and the heart pumps faster, the blood flows to the extremities, the muscles flex and stretch, and we take a essential for creating a sense of motion and musical expression (Berman, 2000; Burrows, 2007; King, 2006; Pierce, 2007; Repp, 2001; Sandor, 1981; Shove & Repp, 1995; Wanderley & V to musical features and their progression such as dynamic contour, pitch contour, phrases, etc. Due to the inextricable relationship between sound and its causal body movement, one perceives musical From the same viewpoint, Gody and Leman (2010) argue that music related gesture embodies an be defined as a pattern through which in line with the activities. Therefore, Gody and Leman re
65 profound engagemen that exis p. 3). The research on musical gesture focuses on music related movements ( e.g. the ) that reflect musical meaning and the way music is perceived. Jensenius (2007) divides music related movement into four functional categories: sound producing, communicative, sound facilitating, and sound accompanyi ng gestures Sound producing gestures are those that produce sound directly with the body or indirectly with a mallet or bow. Sound producing gestures are subdivi ded into gestures of excitation ( e.g hitting, stroking and blowing) and gestur es of modifica tion ( e.g. varying degrees of bo w pressure, vibrato, mute, etc.), which affect the quality of sounds. Sound facilitating gestures are not directly involved in sound production, but expressive of or synchronized with musical features ( e.g. phrases ) theref ore they support producing and shaping the resultant sounds Examples of sound facilitating gestures are a s or a vertical movement of the clarinet bell at the beginning and end of a phrase. Sound facilitating gestures are metaphoric in the sense that the gestures are co expressive of the musical features. For example, a flutist may bend their knees at the beginning of a phrase that starts low in pitch, and gradually straighten their knees as the phrase develops with a series of chromati c scales going up in pitch. In this case, the body position moving upward metaphorically represents the rising pitch and perhaps the increasing tension as well. Further a cellist may hunch down over the cello while approaching a climax of a phrase, to rep resent musical intensity with such a strained body. Communicative gestu res are used between performers (e.g. conducting and eye contact) or between performer and perceiver ( e.g.
66 fac ial expression with eye contact) during musical performance. Sound accomp anying gestures, resulting from the perception of musical sound, trace or follow the musical qualities, for example, mimicking the perceived sound with an air guitar while listening to rock music. It is impractical to define a gesture as only falling into one specific category. Jensenius also claims that a gesture may have multiple functions. For instance, a accompanying function that follows the musical pulse and a communicative function in keeping up with the oth er musicians. Additionally swinging and knee bending represents their bodily engagement in attempting to express salient features of musical sound such as pitch, dynamic contour, and articulation (sound facilitating gestures) as well as to convey those features to perceivers through visual actions (communicative gestures). The facilitating gesture (e.g. a quick breath before the excitation) and a sound accompanying gesture (e.g. breathing along with musical pace). musical meaning in various degrees depending on its function. Sound producing and sound facilitating gestures are involved in the quality of resulting sounds. On the other hand, communicative gestures and sound accompanying gestures are concerned with often loosely articulated (e.g. open hands, body swa ying, foot tapping, etc.), whereas
67 listeners Clarke and Davidson (1998) consider sound accompanying gestures less expressive than sound producing and sound facilitating gestures. They point out that a body sway itive time (Clarke and Davidson, 1998, p. 87). Therefore, s ound producing and sound facilitating gestures are related to the communicative and sound accompanying gestures are concerned with the communication and representation of dominant musical features. In fact, sound producing and sound facilitating gestures are often combined to create a sound that consists of an attack po int as well as a sustaining portion, in which various emergent qualities such as pitch, dynamic, timbre, etc. unfold. Figure 3 4 illustrates different categories of musical gestures organized according to how closely the gestures are engaged with musical m eaning and intention Figure 3 4. The functions of musical gesture arranged based on the level of indicative or expressive qualities Sound producing gestures are highly subjective and communicative gestures are highly indicative of cues rather than s ubjective.
68 Performative Gesture Based on the above discussion of gesture based upon and metaphoric interpretation of meaning, the notion of performative gesture is now proposed The following discussion presents a quaternary model of music al gesture to illustrate the defining factor s and qualities of performative gesture. Further, the relationship between the expressiveness of music and the human nature is presented revealing the function of performative gesture. Quaternary Model of Musi cal Gesture As discussed above, gestures for speech and musical performance can be used for expressing abstract ideas and conveying concrete information. Metaphoric gestures express Indicative gestures are clear and specific, using common signs and imitative actions such as the OK sign, knocking in the air, giving a cue, tapping to the musical beat, etc. This dissertation puts forward a quaternary model based on the factors and qu alities of musical gestures. Figure 3 5 illustrates the four categories of musical gestu re defined by the factors, Interpretation, Indication, Performer, and Listener a s well as their qualities, subjective or indicative, and creative or responsive. The ax is of Performer and Listener represent whether a gesture is generated by a performer for expressive creation or by a listener as a response to the expressed meaning. Although performers still perceive the sound they play, they are focused to a greater exte nt on the creation of musical sounds while listeners mostly perceive and respond to these prepared in spontaneous and less individual The axis of Interpretation and Indication involve the
69 opposite qualit ies of gesture: subjective and metaphoric gestures are derived from the perception of an abstract meaning and objective gestures use culturally establish ed symbols and signs. Figure 3 5 Quaternary model of music al gesture Subsequently, musical gesture is categorized by a pair of two of these factors; Performer Interpretation, Performer Indication, Listener Interpretation, and Listener Indication. Perf ormer Interpretation is a category of gestures that reflects the gestures involving various modulations of a tempo, a swinging gesture with a clarinet for a sweeping pitch cont our, breathing in and out along with a phrase, etc. The Performer other performers ( e.g. nodding with a glance ) and adding a the atrical message to the listener ( e.g. a b ig gesture at the beginning of a climax or at a cadence )
70 Listener that follows the salient fe atures of music (e.g. tapping and dancing to music) therefore reflecting their perception o f music. The Listener Interpretation category also include s those gestures that are imitative of performing (e.g. air guitar, air violin) or conducting gestures which demonstrate the listener s embodied understanding of music The Listener Indication category includes the gestures that are expressed by listeners toward performers or the performance but less related to musical features. From the scope of this quat ernary model of musical gesture, it can be argued that gestures in the Performer Interpretation category are inextricably engaged with musical sound creation and modification, as well as with musical interpretation. In this dissertation, those gestures are interpretation of a musical piece is derived from their perception of the piece. In order to convert their musical intention into actual sounds, performers endeavor to develop their motor skills to create th e desired sound and expression. Through practice and rehearsal, performers achieve the proper gestural control for creating the intended sound; what is referred to here as a performative gesture. In this sense, performative gestures function as the bridge that connects a musical idea to the ideal sound. Thus, it is reasonable to argue that the essential musical quality is created by performative gesture. The following discussion inspects the multidimensional aspect of performative gesture and the way that m usical ideas are transformed into performative gestures and then ultimately into musical sounds.
71 Expressiveness of Music and Performative Gesture As seen in the quaternary model of musical gestures, p erformative gesture play s a critical part in creating expressive sounds in musical performances interpretation of a musical idea is embodied in performative gesture; therefore the expressive quality of music is reinforced and increased. The following discussion demonstrates the close relations hip between the expressive quality of music and the physical and psychological nature of human beings as represented in performative gesture Irregularity Irregularity, which brings about the expressive quality of music, is intrinsic to the human body. Man y researchers have identif ied the variable patterns of musical features as the source of the expressiveness in music: The deviation from the exact is on the whole, the medium for the creation of the beautiful for conveying of emotion (Seashore, 1937, qu oted in Gabrielsson, 198 5 p. 64 ) (musical) compositions are often based upon incomplete, irregular, or imbalanced musical patte rns that are developed and expanded during the course of the work (Barra, 1983, p. 29) From the same point of view, Alf Gabrielsson (1988) points out that the mechanical regularity of music, in contrast to variability, causes a lack of expressive quality. A spontaneous response to music boxes, rhythm machines, and the like is that they sound mechanical, dead, and boring. The main reason for that is and perfect ratio (1:1, 2:1, 3:1 etc.) between successive sound events (Gabrielsson, 1988, p. 41).
72 From this perspective, the irregular variation and development of musical features, and their resultant expressiveness, are regarded as an essential element of music. In musical performanc es, the expressive deviations of music al features var y depend ing on al choice (Clarke, 1985) Notation and s ound Th e discussion of music notation and its performed sounds illustrates how a human performer and pe rformative gestures based on his or her interpretation create the extensive irregularities and varieties of musical expression. Music notation is the a symbolic documentation that gives co ded instructions for any performer to be able to produce the musical sounds the composer envisioned. In acoustic music, the creation of music is not Sandor (1981) also m composer intended so it can be experienced by listeners. In this sense, performers are A performer may be creative i n performing a piece of music. The limited information present in music notation allows performers to interpret the musical features of a piece. In addition background, intention and style, therefore creating a performance that is distinct to some extent, from other performers For example, performers can create a vast variety of tone color and intensity. Tone color and intensity are rarely, or only approximately, specified in a musical score; wherea s other parameters such as pitch, articulation, and dynamics are represented explicitly (Palmer, 1997, p. 5). Skilled listeners can
73 distinguish certain performers by their interpretative style in terms of tone color, phrasing, etc. ions reinforce, or even create, the expressive quality that is often missing in the systematic representations of musical scores. The expressiveness created by a performer at the time of performance is almost always lacking in a computer generated performa nce of the same musical piece. Although computer Comparing a music score with its performanc es by human performers as well as thos e by computers, Johan Sundberg argues that an essential quality exists that is not present in music scores but is created by human performers and not by computers: This implies that the discrepancies between the sound sequences shown in the music score and those really generated in music performance constitute an essential part of music communication: the discrepancies between notation and sound must be in some sense meaningful, that is they carry information which the listener needs in order to enjoy listening to the performance (Sundberg, 1988, p. 53). The reason could be that most computer sequencing programs are not equipped to tr anslate musical terms, such as rubato or cantabile into any meaningful change in the s (1988) research, many researchers have revealed the way human performers convert information presented in musical notation into expressive sounds, in order to construct a computational model for expressive performances (Clarke, 1 988, Todd, 1985, 1989, 1992 & 1995) Further some sequencing programs such as Cubase have developed flexible quantizing options to achieve the effects of swing or humanize d performance.
74 Sundberg (1988) investigates musical performances by human performe rs as a means to create the rules for improving the expressive quality of computer generated performances. Comparing the acoustic signals of performances by human performers with the musical scores, Sundberg recognizes that the combination of flexible timi ng and altered dynamics reinforces the expression of musical structure, motion and duration, dynamics, and pitch is amplified, 2) unexpected tones and harmonies are emphas ized, 3) the structure of the melody is pronounced clearly through appropriate timing, and 4) performances are embedded in an emotional atmosphere (p. 64 65). of musical performances is created by overstating the difference between musical features. For example, t he amplified contrast of durations is achieved by playing a short note shorter and a long note longer. In the same way, playing a soft dynamic softer and a loud d ynamic louder can highlight a dynamic contrast Emphasizing u nexpected tones and harmonies increases the contrast between anticipation and surprise thereby stressing the irregular quality of musical progressions The exaggerated contrast resulting from ov erstating musical features creates a greater range of expression via irregular deviations Expressive t iming E xpressive timing is a great example of the irregularity of music al features created by a human performer. Sundberg (1988) argues that timing, mean ing the various alterations of tempo, is one of the most noticeable discrepancies between music notation and the sound created by human performers. Gabrielsson (1988) argues that human performers have complete control over tempo and timing of musical event s
75 structural the motional and the emotional Comparing musical performance with speech performer should develop the ability to recogni ze and differentiate between commas, can be achieved by enunciating musical events and their structural hierarchy. As Gabrielsson (1988) argues, with expressive timing, p erformers can articulate the duration of notes and rests clearly, for example, legato or staccato and emphasize the tensional contou r of each phrase or period, such as beginning, climax, and resolution. For instance, a performer may introduce a new phrase by breathing in, just as one quickly takes a breath before a sentence in order to avoid pausing in the middle of the sentence In this manner, a slight delay is created at the beginning of the phrase, and the division of musical thought appears naturally clear to listeners. Depending on the hierarchy of structural entities, the flexibility of timing varies. A ritardando at the end of a piece is expected to be more dramatically articulated than one at the end of a phrase. Each performer can create differen t timings of musical events that may result in different musical expressions based up on their musical interpretation. Figure 3 6 illustrates that the ritardando in the last two measures of Prelude Op. 28 No. 15 by Chopin, wa s played differently by four pi anists, Vladimir Horowits, Martha Argerich, Valentina Igoshina, and Ferruccio Busoni Figure 3 6B shows t he durations between the bass notes (Db3, Ab2, and Db2) as well as the ratio between those durations performed by each pianist Though the durations of the whole piece performed by Argerich and Busoni are almost the same (i.e. 4 minutes 50 seconds by Argerich and 4 minutes and
76 46 seconds by Busoni) Argerich played the ritardando longer than Busoni by spending more time between the last two bass notes ( from Ab2 to Db2). On the other hand, Horowitz and Igoshina played the ritardando in a similar pace, it should be noted that (6:43) was much longer than that of (5:10) Comparing the durations of the r itardando with the durations of the performed piece it can be argued that Argerich created a more flexible and dramatic ritardando A) The final two measures of Prelude Op. 28 No. 15 by Chopin B) Different timings between four pianists Fi gure 3 6 The rendition of a ritardando varies depending on the performers (A) The notation of the last two measures of Prelude Op. 28 No. 15 by Chopin (B) the various paces over the bass notes of the ritardando created by Vladimir Horowits, Martha Arger ich, Valentina Igoshina, and Ferruccio Busoni
77 Variations in tempo also affect the perception of motion and its pace. Gabrielsson (1988) argues that tempo is related to (p. 34). A stable tempo is perceived as regular forward motion; a varied tempo is perceived as ongoing motion with a fluctuating pace, for example, a tempo that is held back almost to a stop and then accelerates to the original tempo. The connection of tempo to motion is also found in the description of the tempo markin g andante which represents walking tempo or at a walking pace O ther tempo markings such as allegro presto and largo can be interpreted as a brisk walk, a fast run, and a leisurely walk. A varied tempo according to the tension and energy of musical flow example, the increasing tension as one approaches the climax of a phrase is often realized by accelerating the tempo and increasing the dynamic, and resolved by decelerating the tempo and decreasing the dyna mic. It should be noted that the sense of motion and emotion, which can be easily attributed to the human body and mind. The following discussion demonstrates that t he expressive qu ality created by flexible timing in music is similar to physical limitations as well as expressive intentions Human nature and g esture In musical performances, human performers manipulate tempo and timing based on their musical intentions. Compared to computer generated performances, performances by human performers are more expressive, but less accurate. The irregular quality of musical expressions is created by musical intentions. The reason is that performative gesture is derived not from a music notation but from a interpretation of the notation. In addition performative gesture is affected by
78 the nature of human body. The unstable, erroneous, and imperfect nature of performances by human performers are related to the physical limitations of human beings, who are not able to perform a piece of music exactly as written in the score. An example of the physical limitations of human performers is found in the fact human performers cannot play an exact, e ndless repetition of musical material ( i.e. a loop ) which can be easily generated by computers. They need to take a moment out of the musical flow, however briefly, to place their arms for a large leap, unwind their strained muscles, or breathe. Even an e xceptional performance of minimalistic music does not have the high lev el preciseness of tempo control when compared to the mechanical rendition by a computer. interpretation (m critical role in creating the irregular, expressive quality of music. Bruno Repp (2001) view is in line with this argument Donald Barra (1983) also draws attention to a stures as the key technique to creating vibrant musical expression: depend upon the ability of the performer to create a vibrant, living presence within the music. It is based upon his abil ity to create a series of goal oriented actions that develop within a single, unified, organic musical impulse. (Barra, 1983) Barra (1983) oriented action as an essential part of achieving expressive performance. Goal oriented actions which are deliberately musical intention are considered synonym ously as performative gestures Supporting this point of view, Alexandra Pierce (2007) also uses referring to the highly
79 controlled gestures of a performer or performative gesture as a critical method for intensifying musical expression (p. xiii ): Melodies become more shapely, their energy more varied; phrases become articulate and coherent; arrivals at cadence reach into an embedding stillness; the rhythmic pattern of middleground harmonic progression brings li fe into phrases, and so forth. The m ovements are suggestive for technique and often they can be translated directly into playing gesture and singing. They become part of your technical command (Pierce, 2007, p. xiii ). Pierce (2007) provides examples of gestures, such as hand stretch and arm arching to sh ape phrases and achieve climax body control needed to create the intended expression Certain gestures are directly related to a playing technique. For example, during a piano performance, a gesture involving quick movement with a rigid hand frame is often used fo r playing a chord progression marked marcato. Also, flexible wrist and elbow movements are employed in playing a n arpeggio passage s reflect the character of a performance such as marcato leggiero etc. Performative Gesture as a C orporeal Interpretation P erformative gesture is a corporeal bridge between a musical idea and its actual sound ansformed to the actual sounds by employing performative gestures, which are deeply grounded in the nature of a Figure 3 7 illustrates that an abstract musical idea is transformed by performative gestures into the act ual sound with the sense of motion and emotion, therefore giving rise to the sense of a human body as performative agent
80 Figure 3 7 Realization of a musical idea as the actual sounds by performative gestures, the corporeal articulations of the musical idea Not only is a perfor embodied in performative gestures, but performative gesture also through musical imagery. As opposed to abstract musical ideas, gestures are physical, theref ore providing a link to multidimensional percept ions such as vision, space, time, etc. McNeill (2000) refer s to g representations of a meaning (p. 139). Gestures also take place over a period of time, t heref ore creating a form, in other words, the initial shape of a gesture and its progress through a spatial temporal trajectory The visuo spatial components of musical gesture are related to performance space Lutz 35 ). While constantly coordinating the ir bodies with an instrument in order to play music, a performer becomes aware of the space spec ified in relation to the structural features of the instrument and performer Laba n (1963) defines this space as a kinesphere as cited in Jensenius, 2007, p. 44). While sight reading a piece of music, professional performers are able to of using a visuo spatial trajectory is established by their long term experienc e of
81 up an internal model of movement trajectory, based on the sum of motor activity in embodied experience of creating musical expressions duri ng rehearsals and performances establishes the cognitive connection between a sound and its multimodal perceptions Therefore r epresentations of musical sound ( i e. auditory, visual, spatial, mot or, and kinest hetic perceptions) derived from the visuo spatial trajectory of gestures as illustrated in Figure 3 8 This multimodal link s of musical cognition can improve a ability to utilize musical imagery for better quality performances This is because musical imagery is easily realized through gesture related memories. For example, a musical passage can be imagined through memories in various modalities, such as visual memory of the passage on a keyboard kinesthetic memory of hand positions and finger tensions while performing the passage etc (Zatorre and Becket, 1989). Mariko Mikumo (1994) presents various strategies to retain a melody, such as spelling note names (verbal), singing the notes (auditory), imagining the notes on a keyboard (visualizin g), and finger tapping the notes as if playing the piano (motor). Mikumo also indicates that these pitch encoding strategies are interrelated, which means that the finger tapping the notes can activate the auditory perception of the notes as well as the vi sual percepti on of the notes on a keyboard. From t hese findings it can be argued that gestures, which incorporate various perceptions, can be the key to ac hieving the perception of music as a multimodal musical image. For this reason, during performance, performers can imagine a clear idea of a piece, a sound, a musical idea and intention, etc. in advance through the continuous employment of performative
82 gestures established during multiple rehearsals, therefore improving the quality of performance. Figure 3 8. An illustration of the multimodal nature of musical representation Using visual and spatial gestures as a physical method for realizing a musical idea with expressive sounds, performers tend to interpret musical features as physical characteris tics that can be related to the human body. Many researchers have argued that musical tension and energy can be understood as a physical b eing in motion (Barra, 1983; Sandor, 1981; Berman, 2000). Interpreting musical characteristics as imaginary objects, a performer can mentally simulate imagined objects in motion and come up with the appropriate performative gestures to express the characteristics of the imagined object in the actual sounds Barra (1983) suggests that performers may apply aw of Motion to understand and realize musical motion, for example, a heavy, big object moving more slowly or a light, small object moving more rapidly. Based on the quality of imagined objects attributed to musical ideas, performers can create gestures th at effectively illustrate how these metaphorical objects might behave, therefore improving the expressive quality of musical performance.
83 (p. 44). In research about the brain and cognition (Berthoz, 1996; Pellegrino et al., 1996; An nett, 1996; Decety, 1996), it has been observed that the same neural substrate is activated not only when one executes an action but also when one perceives or i magines the action. This implies that a mental simulation of perception shares structural similarities with actual perception (Finke, are implied in using the perceive 121). As this of imagining the sound. If so, it can be argued that a performer can also imagine the gestural quality that can be att ributed to an imagined sound, depending on their perception of its shape and intention.
84 CHAPTER 4 ILLUSTRATION OF PERFORMATIVE AGENCY IN ACOUSMATIC MUSIC Introduction In the previous chapters, the idea of music cognition via embodied reasoning was discuss ed, in which musical sounds are perceived in relation to bodily movement, rience of the physical world. Supported by the view of embodied cognition, ved as causal actions of sounds and t he focus of discussion moved to a human performer gestures which are the most focused bodily movements for creating musical sounds. It was argued that the irregularity that entails a significant range of deviations and contrasts of musical features crea tes expressive quality therefore bri nging about the sense of human motion and emotion. It was further argued that expressive quality in musical performances by human performers, opposed to computer generated performan ces, result, at least in part, from th e physical limitations of human beings to create precisely consistent expressions. Based on the perceptual connection between irregularity and a human performer, it is possible for a listener to conjure up a metaphoric image of the performative agent as we ll as performative gesture s from the expressive quality in acousmatic sounds. In this chapter, PianoForte by Yu Chung Tseng, a musical example of performative agency in acousmatic music is presented, focusing on the gesture and its expressive quality that is manifested in musical sounds. The Corporeal Interpretation of Gestures in PianoForte PianoForte by Yu Chung Tseng is distinguished by the exclusive use of piano sounds as its and samples in small fragments is to work closely with the sounds, to explore the
85 ( PianoForte para. 1 ). This piece progresses through various combinations of the characteri stic timbre and gesture of the piano, creating a sense of tension and release, force and movement, and performative agency. The primary sound materials of PianoForte consist of two categories: pitched sounds from the keys and strings and non pitched sound s fr om the other parts of the piano ( i.e. the soundboard, metal frame, etc .) Through multiple processing techniques, the distinctive pi ano sounds are transformed into sounds with timbres that are hardly recogniz able as being related to the pian o ( e.g. m etallic sound s noise s voice like sound s water like sound s etc ) At first, the evident presence of the piano in PianoForte may draw conjuring up images of the instrument, performer, venue, musical style, etc. However, the use of small fragments of piano sounds and their timbral variations allows for the perception of only abstract acoustic features and brackets out the implications of a piano performance. Any reference of the sound material to cultural, social, or other extra musical contexts can be masked. While the limited sound materials suggest a narrow focus on timbres, the distinctive gestures, heard allow for the possibilit y of ref erence to physical actions. The two characteristic yet contrasting gestures employed in PianoForte are as follows: 1) iterative gestures of percussive, repetitive sounds and 2) stretching gestures of reversed piano sounds. The detailed relationship betwe en these gestures and their correlated imagined physical objects and actions is presented in the later part of this chapter.
86 Figure 4 1. A perceptual framework suggested for PianoForte based on the timbre and gesture as well as their relations to the pi ano Based on the aspects of gesture and timbre, a perceptual framework for PianoForte can be suggest ed based on the piano sound as the basic sounding model, as illustrated in Figure 4 1. The axis of timbre shows that the distinctive piano timbre is immedi ately recognized in the sounds achieved from the keys and strings. However, the timbres on the other end of the axis, which are similar to streaming water, a voice or chorus, noise, a sound mass with a mixed timbre, etc., are almost unrecognizable as piano timbre. A mbiguous timbres, such as the various filtered sounds, invite the listener to trace or imagine how the timbres are derived from the piano sound. In the axis of gesture, the iterative gesture obtai ned from percussive and impuls ive sounds is recogn ized as native to the piano sound. Though t he stretching gesture can be a bit ambiguous, it can be recognized as a reversed piano sound, therefore establishing the relationship with the piano The steady drone and fluctuating sound s ar e barely
87 recognizable as related to the piano. It is possible that the timbre of a sound is recognizable while the gesture is ambiguous or unrecognizable, and vice versa. A sound created by stretching the sustaining part of a piano sound is an example where the timbre is recog nizable while the gesture is ambiguous. Types of Gesture The following discussion describes the basic forms of the iterative and stretching gestures and their variations Based on embodied cognition where a sound and its s experience and knowledge of a physical object and its movement, the energy motion property of a gesture is interpreted as the kinetic energy of an imagined physical object, resulting in corporeal sensations of musical sounds Iterative gestures An itera tive gesture is characterized by the repetition of a percussive envelope, which has a short attack and decay time and n o sustain The striking action producing an acti an iterative gesture can be created by a series of striking actions on the keys, strings, or other surfaces through direct contact with the body (e.g. fingers, hands, etc.) or indirect contact using other implements (e.g. sticks, mallets, etc.). As one can imagine the striking actions and objects (hands or mallets) behind an iterative gesture, various forms of an iterative gesture employed in PianoForte can be understood in terms of physical ob ject and causal action. While the i terative gestures employed in PianoForte vary in the length, speed, and timbre, they can be divided into two types, increasing and decreasing according to
88 the dynamic shape. Figure 4 2 A illustrates an example of the inc reasing iterative gesture s at 1:16 in which the rate of pulses accelerates and the intensity increases through recurring pulses This increasing iterative gesture can be perceived as resulting from a series of striking actions that carry out an accelerando and a crescendo, therefore giving a sense of forward motion. As opposed to the increasing iterative gesture, the example of decreasing i terative gestures in Figure 4 2, from 1:16 to 1:18, has a decreasing intensity and rate of repeating pulses, which crea te s a sense of fading away. The energy profile of the decreasing iterative gesture can be compared to the way that an object that is dropped on the ground springs back for a certain amount of time. For example, a table tennis ball dropped or thrown on the piano strings will produce a decreasing iterative gesture as the ball bounces around on the strings until the ball ceases to bounce. Note that a parallel can be drawn between a sound and a physical object that might create a movement similar to the dynamic gesture of the sound. A) Increasing iterative gesture B) Decreasing iterative gesture Figure 4 2. Two types of itera tive gesture s based on the dynamic shape A) I ncreasing iterative gesture B ) Decreasing iterative gesture The differenc e between the energy profiles of the increasing iterative gesture and the decreasing iterative gesture stems from the difference in their imagined causal actions. The dotted lines in Figure 4 2 A and Figure 4 2 B represent the excitations
89 applied to create t he gestures. This shows that each pulse of the increasing iterative gesture is perceived being initiated by a striking action because the creation of accelerando and crescendo requires increasing force and speed for each action. Based on the above example of a table tennis ball, it is possible to argue that only the initial pulse of the decreasing iterative gesture is created by a striking action and that the remaining pulses are produced as reactions to the initial excitation. Stretching gestures Str etch ing gesture refers to the amplitude contour of a reversed piano sound, in which parameter stays on a steady level for a while and rises toward the end. The term stretching gesture is used to denote the quality of a stretched string or rubber band, the elas ticity of which might create a tension profile that is similar to the energy profile of a reversed piano sound While the rising portion of the stretching gesture has a great deal of movement, the steady portion is relatively restrained in terms of movemen t. In other wo rds, while the rising portion can be perceived as analogous to an actual physical gesture, the steady portion can be perceived as a symbolic musical gesture of resistance against the nature of sound, movement Especially in PianoForte whic h contains a soundworld that is undeniably characterized by the piano, the natural dynamic shape of a sustain, and decay. Therefore, it can be argued that tension arises from the steady por tion of the stretching gesture, perceived as restraining the natural tendency of motion. Figure 4 3 shows examples of the various stretching gesture s employed in PianoForte, with different degrees of increasing energy. T he variations of the stretching gest ure can be viewed as illustrating the various ways of unleashing the energy that has
90 been restrained over the steady portion of the gesture. The level of increasing energy ranges from a slight rise to drastic rise. A ) S light rise B ) Drastic rise C ) S lope D ) P eak E ) F luctuating gesture Figure 4 3. The variations of stretching gesture repr esenting the tendency of motion. A) Slight rise. B ) Drastic rise C ) Slope. D ) P eak E ) F luctuating gesture. The stretching gestu re with a slight rise of energy (Figure 4 3A) is created by reversing a piano sound with a s oft attack. The stretching gesture with the drastic rise of energy (Figure 4 3B) creates the sense of reaching to ward a maximum level of energy and tension. The slo pe gesture (Figure 4 3C) is the combination of a stretching gesture with a slight rise and its reversion. The peak (Figure 4 3D) is the combination of a stretching gesture with a drastic rise and its reversion. While the slope and peak gestures give an imp ression that the increased tension resolves, the slight or drastic rise give s a sense of a sudden interruption of the energy flow. The fluctuating gesture in Figure 4 3E represents a combination of multiple slope gestures with less energy. Configuration o f gestures The above discussion of the iterative and stretching gestures demonstrate s how the gesture represented in the energy shape of a sound can be understood by imagining a correlated physical object and its movement. Based on this viewpoint, the foll owing discussion goes into the details regarding the perception of combined gestures.
91 Combination of successive gestures The basic combination of gestures in this piece is created by placing the stretching gesture after the iterative gesture Figure 4 4 il lustrates the combination of a stretching gesture with a drastic rise and an iterative gesture, which is frequently employed in PianoForte In this combination, the gestures are perceived as causally related, in which the restrained energy from the steady part of a stretching gesture reach es its maximum level and then is released through the active movement of an iterative gesture. This can also explain how a stretching gesture with a steep rise that is not foll owed by an iterative gesture can give rise to a sensation of uneasiness resulting from the unresolved energy. The dotted line in Figure 4 4 represents the threshold where the restrained energy turns into dynamic mov ement. The energy contour of the s e combined gestures contains a rise carried by the str etching gesture in a restrained manner and a fall carried out by the active, iterative gesture. These combined gestures can also knowledge of physical objects and their movements, in the same way that the individual gestur e was interpreted. An example of a n energy profile similar to the s e combined gesture s can be found in a ball held back by an elastic rubber band. Once released from the rubber band, the ball flies into the air and bounces off multiple surfaces. In this exa mple, the stretched rubber band is understood to be the causal action that leads the ball to move in a shape similar to that of the combined gestures. Imagining the rubber who o r what stretches the rubber band and why The thing that applies the force to draw the movement. As the phy sical object and its movement are perceived as resulting from the
92 the creator of the se combined gestures Figure 4 4. A basic combination of the stretching and iterative gestures : the energy restrained in the steady portion of the stretching gesture is incre ased then released through the active mo vement of the iterative gesture V ariations of these combined gestures are employed throughout the piece. Based on the interpretation of energy motion form s in terms of object s (e.g. rubber band, ball) and environme nt (e.g. surface), various factors causing changes in the energy motion of a sound can be considered. For example, the elasticity of a rubber band, the force applied to the rubber band, and the materials of the ball as well as the surfaces can affect the resulting energy motion form. The more elastic the rubber band is, the farther the rubber band is stretched therefore creating more tension and energy of the rubber band to resume its original position. As a bigger force is applied and/or as the rubber ba nd is held back longer, the ball will bounce much higher and longer unless the ball is made of wood or cotton. A rubber ball, undeniably, bounces for a longer period of time than a plastic or glass ball, which stops bouncing quickly or breaks immediately. The duration between each bounce as well as the rate of bounces vary depending on the materials of, and the distance between, the surfaces. There may also
93 be a case in which the ball bounces only once regardless of the abovementioned attributes For inst ance, if a ball bounces off the ground then somehow ends up going upward beyond creative example that would never happen in reality but can be imagined as an physical experience and knowledge. These examples signify lity to interpret a sound in terms of an imagined physical object and its movement, new types of physical movement s can be created s mind while attempting to understan d surrogated gestures A ) B ) Medium force, more elastic rubber band, and very bouncy ball and surface Less force, more weight of a ball, or less elastic rubber band as well as less bouncy ball and surface C ) D) Medium force, more elastic r ubber band, heavy and not bouncy ball, and not very bouncy surface as well One bounce takes much longer than others E) Reversion of A) Figure 4 5. Various energy profiles of the combined gestures perceived as the re sults of different factors, such as the causal force, the material of objects and surfaces, etc.
94 Various energy motion forms employed in PianoForte are illustrated in Figure 4 5. Figure 4 5A is characterized by a very active iterative gesture that lasts ov er a longer period of time than the other examples. Figure 4 5B shows the combination of a stretching gesture with a longer steady portion and a quick, decreasing iterative gesture. Compared to Figure 4 5A, the longer period of steadiness in Figure 4 5B im plies a weightier object, less force applied to an object, or less elasticity in the rubber band. According to the principles of physics, it takes longer for a heavier object to get into motion with the same given force. Therefore, the energy of the stretc hing gesture that increased over a longer duration is expected to be resolved through the iterative gesture over a longer period of time as well. However, the quickly disappearing iterative gesture counteracts that expectation and is perceived as an unpred icted twist, or irregular feature, therefore creating an interesting or expressive quality to the sound. The variations illustrated in Figure 4 5C and 4 5D are also seen as modifications of the basic energy profile in unpredictable ways In Figure 4 5C the increased energy of the stretching gesture is rel eased through a single impulse sound instead of an iterative gesture Although not the same as an iterative gesture an impuls ive gesture can be thought of as a type of iterative gesture in the sense t hat an iterative gesture consists of recursive pulses. As illustrated in the above example of a ball flung into the universe after only a single bounce an impuls ive gesture can be perceptually related to an iterative gesture Figure 4 5D illustrates an it erative gesture one of whose pulse s is elongated, creating an unexpected change in timing. In Figure 4 5E, the basic energy contour of these combined gestures is reversed. The energy that rises through the
95 articulated movement of iterative gesture fad es a way immediatel y with a reversed attack creating a perception of contrast Juxtaposition J uxtaposition of the stretching and iterative gestures has reinforcing effect since the stretching gesture is used for the overall shape of a sound, while the iterativ e gesture creates a sens e of inner movement Figure 4 6 A shows an example, in which the stretching gesture with a rise is juxtaposed with the increasing iterative gesture, therefore strengthening the perception of increasing energy and movement Figure 4 6 B illustrates an example in which the iterative gesture and the stretching gesture with a slope are juxtaposed. Juxtaposing these two contrasting gestures simultaneously creates a more elaborate texture in which musical tension and expression may be more s trongly articulated. The juxtaposed gestures along with the successive combinations of gestures, create gestural variety in PianoForte A ) B ) Figure 4 6. Juxtapositions of stretching and iterative gesture s A) R ising energy in both stretching an d approaching iterative gestures B ) S tretching gesture with a alope and its inner movement created by iterative gesture. Other R elationship s with T imbre, F requency, and T exture In PianoForte, certain types of gestures and timbres are favorably paired. For example, the metallic noise with a high frequency is frequently used with a static gesture and a drastic rise and/or fall in dynamics P itched sounds appear i n the shape
96 of a strong impuls ive gesture or an iterative gesture. Especially in the beginning of the piece, these sets of gesture s and timbre s are clearly perceived, creating a sense of interaction between two groups of gesture timbre unity. However, these predetermined sets of gesture and timbre vary as the piece unfolds. For example, a stretching p iano sound precedes a n iterative metallic noise creating a contrast with the combination of a stretching noise and an iterative piano sound This cross relationship takes place in relation to frequency and texture as well. The sounds that c ontain contrast ing directions between their elements ( i.e. up and d own, forward and backward, etc.) are exemplary. Examples include the sound s (from 3:29:500 to 3:31 and from 3:36 to 3:37:500) in which the energy increases while the frequency moves lower and the timbre becomes darker S imilar to the Doppler effect this example brings about the perception of an object approaching to the perceiver creating a sense of physicality The R elation ship with P erformative A gency L isteners can easily imagine physical object s and their movement s from the characteristic gestures used in PianoForte The physical characteristics in a sound in vite the listener to employ his or her experiences with objects in the real world. In PianoForte the various gestural features of the piano sou nds motivate the listener to search his or her experience and knowledge of physical activity, the movement of which may and a rubber band. This physical correlation leads th e listener to interpret the gestural features of a sound as kinetic features, containing energy, motion, i ntention, and performative agency V ariations of the basic energy contour of combined gestures that includ e unexpected features incorporate irregulari ty into the basic paradigm and create an
97 musical performances (see Notation and sound in Chapter), it was implied that overstating the quality of, and the difference betwe en, musical features is a way of reinforcing the expressiveness of music. In this context, those variations of combined gestures in the above discussion are seen to be contributing to the expressive quality of PianoForte. In these gestural variations, on e can conjure up an image of an agent behind the object to whom the intention of the causal action might be attributed. Supported by the expressive quality implied in the gestural variations, one can envision a human being who is making an effort with the s e deliberately created performative gestures in order to attain the intended result. Detailed Listening of PianoForte A detailed listening of PianoForte is now presented, focusing on the previously discussed gestures and their physical correlations. In add ition, the way in which the sense of musical tension, narrative, and performative agency emerge from the varieties of gesture s and timbre s evolving through the piece is discussed, along with the structure of the piece and its subsections. Overall S truct ure PianoForte is organized into four sections that are similar to the typical structure of the traditional Chinese Poem: Introduction, De velopment, Turn, and Conclusion. The only difference is that PianoForte has a brief bridge inserted between the Intro duction and Development. In modern times this structure is also employed in other genres of the arts, such as literature and architecture. This structure seems to fit the strong drama of PianoForte The musical content and characteristics of each section are listed below
98 Table 4 1. The four part structure of PianoForte. Introduction (0:00 1:48) 5 phrase structure Slow introduction of characteristics gestures and timbres Bridge (1:48 2:19) Pattern reminiscent of the Introduction Gradually leading to t he Development Development (2:19 4:27) Dominance of the iterative string sound Dense texture Formation of complex entity through timbral transformation Turn (4:27 6:33) Static and iterative gestures become fused Sounds merge into a complex entity Conclu sion (6:33 8:32) Continuous sound in background Material from the beginning revisited Closing punctuation with the material from the beginning after a brief break Introduction (0:00 to 1:48) This opening section is composed of five sounding events occurr ing at varying intervals, as illustrated in F igure 4 7 These sounding events are followed by a brief silence or quasi silence (i.e. very soft residual noise) comparable to musical phrases separated from one another by a rest or breath. The piece unfolds slowly over the phrases, employing the interplay between contrasting gestures (i.e. stretching and iterative gestures) and timbre s (i.e. piano and noise). E ach timbre is introduced one at a time, the gesture is slightly more developed at each phrase an d the duration of each phrase gradually increases The stretching piano sound is repeated with slight variati ons over the first four phrases, therefore the tension accumulates fro m the energy persistently restrained in the steady part of the stretching ges ture. The first phrase (0:00 to 0:07) introduces a brief and soft metallic noise with a high frequency whose amplitude has a small peak composed of a slight rise and quick fall. This noise is heard as a delicate movement in the distance. In the second phr ase (0:10 to 0:21), the metallic noise appears lower in frequency and higher in amplitude, and is perceived as a closer presence. As its energy rises exponentially, the metallic
99 noise is interrupted by a strong impuls ive piano tone. The impu ls ive piano ton e is extended by merging into its reversed form, whose amplitude increases accompanied by the noise of a grain y texture. This mixture of piano tone and noise is interrupted by a subdued piano sound with a brief decreasing iterative gesture. From this pro gression of timbre and gesture, it should be noted that the change of energy state is accompanied by a change of timbre and gesture: t he restrained energy in the stretching noise is released through the iterative piano sound or blocked by the impuls e drive n piano sound. The third and fourth phrases are subtle developments of the material introduced in the second phrase. The third phrase (0:22 to 0:40) utilizes layers of the stretched piano sound whose pitch range gradually expands along with the increasi ng amplitude. In the fourth phrase (0:45 to 1:05), the pattern of the stretching and iterative sounds continues to be explored with overlapping layers, which creates a mixture of these sounds. In the last phrase (1:10 to 1:48), the energy that has been ac cumulated through the previous phrases is finally released through the active, iterative gesture of the piano string sound. The gestural texture of this phrase is illustrated in Figure 4 7A. According to the types of gesture, the sounds can be roughly grou ped into static piano sounds in low and high registers, impulsive piano sounds in mid and low registers, and iterative sounds of piano keys, strings, and noise (Figure 4 7A). The iterative gesture is explored through various timbres, including the sounds o f scratched and plucked strings, struck keys, noise, etc. The impuls ive low piano sound evolves into a noise like timbre from 1:22 to 1:27, creating a brief shift of timbre as well as gesture. The dotted lines
100 represent the beginning of each musical segmen t divided by the shift in timbre of the iterative sounds. The first four shifts between the sets of timbre and gesture take place regularly at every third second, as if representing the momentum of the energy that was just released. From the fifth segment, the shifting pace slows down and the overall loudness decreases as well, creating a similar energy motion contour to that of the decreasing iterative gesture (Figure 4 7B). In addition, the overall pitch range moves down. The pitch of the iterative string sound falls discretely at each appearance while the pitch of the stretching sound falls and its loudness increases (cross relationship). A ) B ) Figure 4 7. A ) The layout of the gesture s in the last phrase of the introduc tion, in which the iterative gesture is developed with various timbres and paces. B ) T he overall energy motion shape of this phrase is similar to a decreasing iterative gesture It should be noted that the overall progression of the introduction is similar to the energy motion profile of the combination of stretching and iterative gestures, as
101 represente d by dotted lines in Figure 4 8 Applied in a local level as well as a structural level, this energy motion profile reinforces itself as a significant music al idea in PianoForte Figure 4 8. The five phrase structure of the introduction of PianoForte on a timeline. T he dotted line illustrates the tension accumulated through the stretching gesture and released through the activit y of the iterative gesture in the last phrase. The increasing activity and duration of the phrases create a n energy shape similar to that of the basic combination of stretching and iterative gestures. The gradual and careful progression of timbre and gestu re in the introduction evokes a metaphoric image of a person who could be deliberately creating this sound with an object. The repeating sequence of rests and progressive phrases can be perceived as the action imagination reaction cycle discussed in chapte r 2. Action is perceived as the musical phrase, imagination is activated during the rest, in which the quality of the phrase is perceived in comparison with musical imagery in the mind, and reaction is perceived as resulting in the next phrase. This perception of the introduction of PianoForte as the cognitive process of action imagination reaction is supported by the expressive qualities that develop over the course of the phrases, which can conjure up an image of a person attempting to create a desired expression in each phrase. The tension emerging from the contrasting gestures also gives rise to an
102 Therefore, it can be argued that the metaphoric image of a stretc hed string used to illustrate the tension is also useful psychological state. In addition, the layout of phrases with inserted rests gives rise to a sense of breathing, which directly signifies a human body Accor ding to Elaine King a rticulation of musical features ( i.e. timing and structure ) Therefore, this breathing like pattern of the introduction is perceived as reflectin intention and interpretation. As a listener psychological states involved in creating the expressive quality of the musical sounds, the listener naturally perceives the performat ive agent as the protagonist in the musical drama Bridge (1:48 to 2:19) The B ridge (1:48 to 2:19) leading to the development is composed of the residual noise and string sound placed sparsely. This part is reminiscent of the introduction since the delicat e sounding events are arranged in a breathing like pattern similar to that heard in the introduction. E ach event is preceded by a brief pause and the duration of eac h event subsequently increases, as illustrated in F igure 4 9 In this quiet section, the st ring sound with subtle movement tends to linger and move up in pitch, as a preparation for the development in which the stretching and iterative gestures are merged into a bigger movement along with the increasing timbral and textural complexity.
103 Figu re 4 9. The breathing like pattern of the bridge is reminiscent of the Introduction Development (2:19 to 4:27) In this section, musical development is perceived in three aspects: 1) the extension of timbre and texture derived from the string sound, 2) the extensive use of iterative gesture, and 3) the momentary metamorphoses. The development starts off with a variety of timbres created from the string sound by pitch shifting, filtering, and other processing methods. These newly created sounds are layered w ith the original string sound, building up a dense texture. The composer also uses layers as a way of increasing the dynamic level, which results in the simultaneous increase of timbral density dynamic, and texture. In spite of the timbral variations, the iterative gesture and fast moving grain texture of the original sound remain the same in the modified sounds. While the introduction of the piece focuses on the contrast between the stretching and iterative gestures, this part progresses relentlessly thro ugh the iterative gestures in a quick and energetic manner. While the stretching gestures are occasionally perceived as supporting the overall dynamic increase (e.g. high pitched metallic sound) or keeping a sense of continuity (e.g. the low muffled pian o sound), the predominant presence of iterative gestures gives rise to a keen sense of forward movement with a fast pace.
104 The iterative string sound, which represent ed the release of energy and tension at the end of introduction of the piece is developed further, extending the initial boundaries of timbral possibility The first momentary metamorphosis into a chorus like timbre takes place at 2:50, which quickly returns to the iterative string sound. The second metamorphosis occurs at 3:02 into a water li ke sound in a swirling gesture between the left and right channels. The water like sound is present from 3:02 to 3:14 and gradually disappears into silence from 3:14 to 3:25. After a five second break, the soundworld returns to the distinctive piano timbre along with the it erative gesture that emerged, creating a big contrast with the water like timbre from the break. The interpretation. It may be perceived as resulting from th e exhaustion of musical energy and activity. In the same context, the distinctive piano timbre after a break may be perceived as a return to the beginning. However, the sudden transformation to non piano timbre may also be perceived ironically in the sense that the initial identity of timbre and gesture vanishes, from which a metaphoric protagonist of this musical drama was envisioned. The following pause augments the sense of wonder and uneasiness even more. After the piano timbre resumes, the interplay be tween the iterat ive grain sound and the impuls ive piano sound continues with increasing tension. The combination of th e iterative gesture and the impuls ive gesture ( see Figure 4 5 E ) is employed the most, in which the energy accumulated in the iterative sou nd is stopped, rather than released. Halting the energy, instead of unleashing it, results in slightly decreased energy and momentum, as seen in the gradually reducing speed of the iterative pulses. The sound
105 of the struck piano body is extended in length from 4:02 to 4:10 and can be perceived as involving a performative agen t struggl ing to continue the mov ement but failing Turn (4:27 to 6:33) With a low continuous sound that emerges from the end of the development, the Turn section has a sense of contin uity rather than a dynamic interplay between gestures. The chorus like sound that appears again from 4:49 to 5:25 serves to interweave the multiple sounds of both static and iterative gestures. The creaking sound from 5:10 to 5:25, which is similar to but more extensive than the percussive piano body sound at 4:02 in the development, conjures up a metaphoric image of a string that is stretched out to a great degree and about to snap. Accompanied by or propelled by, the chorus like sound and creaking soun d, the music reaches a fluctuating drone at 5:25 that is derived from the string sound, that holds all the sounds together, creating a sound mass. With the fluctuating drone from 5:25, the piece turns to a new state in which the sound mass of a dense textu re develops into one big stretching gesture From the low impuls ive sound at 6:07, a timbral transformation occurs into the processed voice and whistle sound s Tension also accumulates during this dense texture and stati c presence of the sound mass through the opposing motion of the pitch contours of the low drone an d the high individual stretching sound. The individual stretching sounds have the feature of cross relationship in which the pitch decreases while the dynamic increases, creating a n effect simil ar to Doppler. In terms of form, this section is where musical idea and its development turn to a new state. Based on the above description, it can be arg ued that the performative agent is turning to a hyper human level and this is represented by the signi ficant timbral transformation breaking
106 thro ugh the soundworld of the piano by overcoming the physical boundary of a human body Conclusion (6:33 to 8:32) In the C onclusion, a calm sensation is created with the reverberating chorus like drone as well as the whistling sound, while the previously explored musical mater ial is sparsely revisited. O ccasional creaking sound s and water like sound s are perceived as reflecting the uneasiness of performance. This coexistence of these timbres and piano timbre gives ris e to a sense of a pseudo or virtual return to the initial soundworld. After all the sounds fade away, the piece ends with a brief statement of noise and the sound of the struck piano body, which reminds listeners of the beginning of the piece. Discussion In PianoForte, the characteristic gestures and their resulting energy profiles are used for shaping the individual sounds and for governing the structure of the piece. The stretching gesture with the drastic ri se is used for the form of the I ntroduct ion. The combined shape of the Bridge and D evelopme nt is also similar to the above mentioned shape of the I ntroduction. The only difference is that the shape of the combined Bridge and D evelopment has more of a sudden change between soft, slow sounding events an d loud, fast events as illustrated in Figure 4 10. Introduction Bridge + Development Figure 4 10. The overall dynamic shapes of the introduction and the bridge and development combined
107 In the I ntroduction, an examination of the detail of acoustic features ( i.e. amplitude contour ) helps understand the sounds in terms of its correlated physical object, its perceived movement, and the performative agent. Perceiving the energy profile of a sound a nd the causal action behind the sound naturally leads to the recognition of agency. The breathing like patterns in the introduction as well as in the bridge are heard as a strong in dication of performative agency The rests in the pattern naturally lead th e listener to contemplate and expect the next sounding event, as well as to perceive the imaginary agent also listening to grasp how to play the next sound. C ontinuous variations with subtle differences from the initial timbre and gesture are perceived a s sounds that result based on his or her musical intention. In this sense, it can be argued that the performative agent is as the protagonist in the musical drama In the rest of the piece, it is plausible that the performative agent can seem to disappe ar from the musical narrative (D evelopment) or transform into a hyper human agent (T urn). It is difficult to interpret every sound in terms of performative agency, especiall y the complex sounds that are perceived as a sound mass with a dense texture and are therefore beyond the physical capabilities of a human performer Apashia by Mark Applebaum is an exemplary piece that shows the transformation from gestures and non gestur es. In this piece, a performer is given the detailed instructions of various metaphoric hand gestures to be performed in synchronization with the audio narrative. continue until the sounds develop into complex sounds with a dense texture. The
108 moment in which the performer stops performing gestures and the sounds evolve to a massive entity portrays a clear boundary between gesture and beyond gesture The performative agent impli ed in acousmatic music, however, is not limited to primary sound materials si gnif icantly transformed in the Development and T urn can still be perceived as related to the original materials they just cannot be described in terms of a single human body. At this point, the performative agent is not a single protagonist in the drama ; rat her it can be argued that the performative agent is motionless, observing the sounding results of his or her intention. This shows not only that acousmatic music can create a vivid sense of performative agency, but also that performative agency can be expl ored beyond the boundary of a human body, therefore giving rise to hyper human agency. Performative agency can also be seen in other acousmatic pieces such as Klang by Jonty Harrison, Wind Chimes by Denis Smalley, Tranquility by Kyong Mee Choi, etc. The co mmon features in these pieces are the use of distinctive gestural characters and their evolving variations through timbral exploration through which a listener can perceive the agent
109 CHAPTER 5 CONCLUSION The starting point of this study was the listenin g experience of acousmatic music, in which an imaginary person who could be creating all the musical sounds can be This is perhaps striking and ironic in the s ense that acousmatic music may seem to have a lack of musical communication or interaction with the audience because of its performance setting, in which the audience is sitting in a dark concert hall or listening room and hearing the sounds mediated by loudspeakers. The perception of performative agency may be under stood as the listener intentionally attempts to make sense of the unfamiliar acousmatic sounds in terms of a familiar reference, that is, a performer and instrument. It is acknowledged that one may create a persona in listening to music and in composing, a s evidenced by musical terms that involve some level of personification. This is because listeners still experience music in most cases through musical performance or its recording. In any case, it is evident that certain qualities in the sounds of acousm atic music can conjure up an image of a human body being in charge of the sounds, therefore giving rise to a sense of performative agency. The focus of this study is to define and illustrat e the qualities in sounds that cause the perception of performative agency based on relevant research in music cognition and musical performance. Chapter 1 reviewed the traditional concept of acousmatic music that is grounded in the scientific, objective view of sound and perception. According to Schaeffer principle of reduced listening, the listener is encouraged to focus only on the pure acoustic features of acousmatic sound. This approach offers listeners the phenomenological experience of pure listening, therefore engaging with the qualities of
110 a sound that are not recognized from everyday listening. While this phenomenological approach introduces a new way of perceiving and composing acousmatic music, it is inevitabl e between a sound and t he physical world play a critical role in understanding acousmatic eatures are inseparable from the physical properties involved in the creation of the sound. For instance, the objects (hands) and causa l action (clapping) as well as the acoustics of the space (church) in which the sound is created are reflected in the sound as part of its acoustic features, i.e. timbre, impulsive excitation, and reverberation. Therefore, listeners can perceive the sound time, even when listening to a sound that is abstracted from its origin. The causal and expressive aspects of acousmatic sounds are the key to defining the perception of performati v e agency. T he causal aspect of a sound is explained by the theory of embodied music cognition while the expressive aspect of performative agency is illustrated by musical performance and the gestures of a human performer. Chapter 2 introduced the theory o f embodied cognition that focuses on the tight link between action and perception and how it affects acousmatic music listening. It was generated by h objects and action, which engages the network of multimodal sensations. Therefore, one can visualize the causal action by hearing a sound and re experiencing or imagining other perceptions linked to the action, such as perceptions of force and movement i n a spatial trajectory, touch, proprioception, etc. In
111 world is employed as the metaphor for understanding musical gesture and its meaning. Based on this embodied cognition, musical sounds can be comprehended as representing the physical properties between action and object. Recognizing an action that causes an object to move in a certain way as represented in sounds, the listener can conjure up an image of the metaphoric age nt who perform the action, whether intentionally or involuntarily. This embodied paradigm experience into the way of understanding musical meaning influ ences the manner of listening to stening was already challenged by the ideas of gestural sonorous object s ecological listening, etc. Smalley, with his concept of indicative fields and gestural surrogacy, proposed that acousmatic music can and non sounding experience, therefore extending the genre of acousmatic music as an art form in which musical and extra musical contexts can coexist. action, the causal action s do not necessarily represent the emotional and expressive quality of performative agency. The gestural qualities in sounds that give rise to the sense of performative agency were explicated by examining the gestures of the human performer in Chapter 3. B eginning with defining the term gesture, the difference between simple actions and meaningful gestures was demonstrated, therefore leading to the idea of performative gestures. The cognitive feedback between perception and action involved in musical perfor mance is considered the most intensive and expressive of creating the desired quality of the resulting sound, and therefore, reflect musical idea
112 and meaning. It was also argu ed that the variational progression and irregularity of gestures and resulting expression in musical performance are derived from the the physical nature and limitations of human beings Understanding the physical properties and performative qualities that can be perceived in acousmatic sounds, Chapter 4 illustrated a musical example PianoForte by Yu Chung Tseng. The energy motion profile of individual sound s and/or combined sounds was interpreted in terms of actio n s and object s in the physical world, therefore leading to a metaphoric image of an agent who is creating the action. The numerous variations and developments of the musical material carried through the characteristic gesture s and timbre s were illustrated as supporting the intentional and expressive aspect s of performative agency. In addition the strong musical drama and breathing like pattern of musical events enhanced the sense of a human body, as a performer who attempts to realize an ideal performance. It was argued that the performative agent being in charge of the musical sounds is perceived as a protagonist in the musical drama The musical tension created through the energy motion contours of sounds was attributed to the emotional intention of the a gent as well. Further, the performative agent is not limited to human capacities but can be extended beyond what the human body can do, therefore creating the sense of a hyper human which is only possible in acousmatic music. This study has discussed the performative agency in acousmatic music focusing on human qualities, that is constructed through the experience of musical performance. This document can serve as a starting point f or
113 researching and composing various types of per formative agency in other genres of electr oacoustic music for fixed media ( i.e. soundscapes, sound installations, algorithmic compositions, etc .) represented in the manner of recording and arranging the sounds. For instance, the composer listening to sounds and moving through locations are traced in the work A s regarding sonic representation according to the layout and adjustm ent of the record ed sounds. Examples include juxtaposition of multiple sounds, progression of similar sounds from one location to another, modifying sounds with the use of filter or other technique s to emphasize or blend multiple sounds, etc. This way, the composer perform s as an interpreter of natural sounds. Through e xtending this subtle interpretive aspect of the composer in soundscape works to a performative aspect engaged in a musical drama a hybrid type of performative agency can be perceived The shift between inter pretive agency and performative agency may develop a cross style between soundscapes and acousmatic music.
114 APPENDIX A COMPOSITION: DISPLACED Displaced explores a compositional style that extends across soundscape composition and acousmatic music Th expressive, interpretive intention incorporated into representations of natural environments. This approach results in a fine balance between the two disciplines, one in which the natural soundworld is portraye d in a condensed way, and the other in which performative articulations express creative intentions. Object A 1. Displaced (.mp3 file 29MB)
115 LIST OF REFERENCES Annett, J. (1996). On knowing how to do things: A theory of motor imagery. Cognitive Brain Resea rch, 3, 65 69. Applebaum, M. (201 2, Feb 1 ). Aphasia [Video file] Retrieved from http://www.youtube.com/watch?v=wWt1qh67EnA Baker, J. M. (2001). The keyboard as basis for imagery of pitch relatio ns. In R. I. Gody & H. Jrgensen (Eds.), Musical imagery (pp. 251 270). Exton, PA : Swets & Zeitlinger Publishers Barra, D. (1983). and interpretation Englewood Cliffs, NJ: Prentice Hall. Battier, M. (2007). What the GRM brought to music: From musique concrete to acousmatic music. Organised Sound 12(3), 189 202. doi: 10.1017/S1355771807001902 Berman, B. (2000). ch New Haven, CT: Yale University Press. Berthoz, A. (1996). The role of inhibition in the hierarchical gating of executed and imagined movements. Cognitive Brain Research 3, 101 113. Burrows, D. (2007). Time and the warm body: A musical perspective on the construction of time Boston, MA: Brill. Chion, M. (1983). Guide des objets sonores: Pierre Schaeffer et la recherch musicale. Paris: Buchet/Chastel. Chopin, F., & Argerich, M. ( 1991 ). Chopin: 26 p reludes etc / Martha Argerich. Chopin, F., & Busoni, F. ( 2007, Oct 13 ). (1923) Retrieved from http://www.youtube.com/watch?v=2ZDw5Wp Bfo Chopin, F., & Horowitz, V. ( 1974 ). New recording of Chopin Chopin, F., & Igoshina, V. (2010). Valentina Igoshina plays Chopin. Cl ark, M. R. (2002). Singing, acting, and movement in opera: A guide to singer getics Bloomington, IN: Indiana University Press. Clarke, E. F. (1985). Structure and expression in rhythmic performance. In P. Howell & R. West (Eds.), Musical structure and cog nition (pp. 209 236), London: Orlando. Clarke, E. F. (2005). Ways of listening: An ecological approach to the perception of musical meaning New York, NY: Oxford University Press.
116 Clarke, E., & Davidson, J. (1998). The body in performance. In W. Thomas (Ed .), Composition, performance, reception: Studies in the creative process in music (pp. 74 92). Brookfield, VT: Ashgate. Cox, A. (2006). Hearing, feeling, grasping gestures. In A. Gritten & E. King (Eds.), Music and gesture (pp. 45 60 ). Burlington, VT: Ashg ate Cox, C. & Warner D. (200 4 ). Audio culture: Readings in modern music New York, NY: Continuum. Decety, J. (1996). Do imagined and executed actions share the same neural substrate? Cognitive Brain Research, 3, 87 93. Edie, J. M. (1987). phenomenology: A critical commentary Bloomington, IN: Indiana University Press. Emmerson, S. (1998). Aural landscape. Organised Sound, 3(2), 135 140. Emmerson, S., & Smalley, D. (2001). Electro acoustic music. In S. Sadie (Ed.), The new grove dictionary of music and musicians (2 nd ed.). London: Grove. Emmerson, S. (2007). Living electronic music. Burlington, VT: Ashgate. Field, A. (2000). Simulation and reality : The new sonic objects. In S. Emmerson (Ed.), Music, electronic media and culture (pp. 36 55). Burlington, VT: Ashgate. Finke, R. (1985). Theories relating mental imagery to perception. Psychological Bulletin 98(2), 236 259. Gabrielsson, A. (1985). Interplay between analysis and synthesis in studies of music performance and music experience. Music perception: An interdisciplinary journal. 3(1). 59 86. Gabrielsson, A. (1988). Timing in music performance and its relation to music experience. In J. A. Slodoba (Ed.), Generative processes in music: The psychology of performance, improvisation, and compo sition (pp. 27 51 ). New York, NY: Oxford University Press. Gibson, J. (1966). The senses considered as perceptual systems. Boston, MA: Houghton Mifflin Company. Gibson, J. (1979). The ecological approach to visual perception Boston, MA: Houghton Mifflin C ompany Gody, R. I. (1997). Knowledge in music theory by shapes of musical objects and sound producing actions. In M. Leman (Ed.), Music, gestalt, and computing: Studies in cognitive and systematic musicology (pp. 89 102 ). New York, NY: Springer.
117 Gody, R I. (2001). Imagined action, excitation, and resonance In R. I. Gody & H. Jrgensen, Musical imagery (pp. 237 250). Exton, PA : Swets & Zeitlinger Publishers Gody, R. I. (2006). Gestural conceptual a pparatus. Organised Sound 11(2), 149 157. Gody, R. I., & Leman, M. (2010). Why study musical gestures? In R. I. Gody & M. Leman (Eds.), Musical gestures: sound, movement, and meaning. New York, NY: Routledge. Goldin Meadow, S. (2003). Hearing gestures: How our hands help us think. Cambridge, MA: The Belknap Press of Harvard University Press. Hurley, S. (2001). Perception and action: Alternative views. Synthese 129(1), Springer. http://www.jstor.org/stable/20117165 Husserl, E. ( 1962 ). Ideas :General intr oduction to pure phenomenology (W. R. Boyce Gibson, Trans.). New York, NY: Collier Macmillan. (Original work published 1928). Jncke L. (2006). From cognition to action. In E. Altenmuller, M. Wiesendanger & K. Kesselring (Eds.), Music, motor control and the brain (pp. 25 38) Oxford, NY: Oxford University Press. Jensenius, A. (2007). Action sound: Developing methods and tools to study music related body movement University of Oslo. Jensenius, A., Wanderley, M., Gody, R. I. & Leman, M. (2010). Musical gestures: concepts and methods in research. In R. I. Gody & M. Leman (Eds.), Musical gestures: sound, movement, and meaning (pp. 12 35). New York, NY: Routledge. Johnson, M. (2007). The meaning of the body: Aesthetics of human understanding. Chicago, IL: The University of Chicago Press. Johnson, M., & Larson, S. (2009). Semething in the way she moves Metaphors of musical motion. Metaphor and Symbol, 18(2), 63 84. Kalakosky, V. (2001). Musical imagery and working memory. In R. I. Gody & H. Jrgensen (Eds .), Musical imagery (pp. 43 56). Exton, PA: Swets & Zeitlinger Publishers. phenomenological reduction. Organised Sound, 12(1), 15 24. Kendall, G. (2010). Meaning in elect roacoustic music and the everyday mind. Organised sound, 15(1), 63 74. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge, UK: Cambridge University Press.
118 King, E. (2006). Supporting gestures: Breathing in piano performance. In A. Gritten & E. King (Eds.), Music and gesture (pp. 142 164 ). Burlington, VT: Ashgate. Laban, R. v. (1963). Modern educational dance. London: Macdonald &Evans Ltd. Leman, M. (2008). Embodied music cognition and mediation technology. Cambridge, MA: MIT Press. Lephart, S., & Fu, F. H. (2000). Introduction to the sensorimotor system. In S. Lephart & F. H. Fu (Eds.), Proprioception and neuromuscular control in joint stability (pp. xvii xxiv ). Champaign, IL: Human Kinetics. McNeill, D. (1992). Hand and mind: what gestures r eveal about thought. Chicago, IL: The University of Chicago Press. McNeill, D. (2000). Introduction. In D. McNeill (Ed.), Language and gesture. Cambridge (pp. 1 10 ). UK: Cambridge University Press. McNeill, D. (2005). Gesture and thought. Chicago, IL: Univ ersity of Chicago Press. McNeill, D. & Duncan, S. D. (2000). Growth points in thinking for speaking. In D. McNeill (Ed.), Language and gesture (pp. 141 161 ). Cambridge, UK: Cambridge University Press. Middleton, R. (1993). Popular music analysis and musico logy: Bridging the gap. Popular Music, 12(2), 177 190. Mikumo, M. (1994). Motor encoding strategy for pitches of melodies. Music Perception, 12(2), 175 197. Mountain, R. (2001). Composers and imagery: Myths and realities. In R. I. Gody & H. Jrgensen (Ed s.), Musical imagery ( pp. 271 288 ). Exton, PA: Swets & Zeitlinger Publishers. movement. In J. Robinson (Ed.), Music and meaning (pp. 131 153 ). Ithaca, NY: Cornell University Press. No rman, K. (2004). Sounding art: Eight literary excursions through electronic music Burlington, VT: Ashgate. Palmer, C. (1997). Music performance. Annual Review of Psychology 48, 115 138. Pierce, A. (2007). Deepening musical performance through movement: The theory and practice of embodied interpretation. Bloomington, IN: Indiana University Press. (Eds.), Musical imagery (pp. 185 200). Exton, PA: Swets & Zeitlinger Publi shers.
119 Reybrouck, M. (2001). Musical imagery between sensory processing and ideomotor simulation. In R. I. Gody & H. Jrgensen (Eds.), Musical imagery (pp. 117 136). Exton, PA: Swets & Zeitlinger Publishers. Sandor, G. (1981). On piano playing: Motion, so und, and expression. New York, NY: Schirmer Books. Schaeffer, P. (1966). des objets musicaux. Shove, R., & Repp, B. H. (1995). Musical motion and performance: Theoretical and empirical perspectives. In R. John (Ed.), T he practice of performance: Studies in musical interpretation (pp. 55 83 ). New York, NY: Cambridge University Press. Smalley, D. (1996). The listening imagination: Listening in the electroacoustic era. Contemporary Music Review, 13(2), 77 107. Smalley, D. (1997). Spectromorphology: explaining sound shapes. Organised Sound, 2(2), 107 126. http://dx.doi.org/10.1080/07494469600640071 Cognit ive Psychology, 8, 553 560. Spelke, E. (1979). Perceiving bimodally specified events in infancy. Developmental Psychology, 15(6), 626 636. Sundberg, J. (1988). Computer synthesis of musical performance. In J. A. Sloboda (Ed.), Generative processes in music : The psychology of performance, improvisation, and composition. New York, NY: Oxford University Press. Ten Hoopen, C. (1994). Issue in timbre and perception. Contemporary music review 10(2), 61 71. The I nternational A ssociation for D ance M edicine and S ci ence. (2008). Proprioception Retrieved September 11, 2011, from www.iadms.org/associations/2991/files/info/proprioception.pdf Tseng, Y. (2010). PianoForte. On Metamorphoses 2010 : 6 th biennial acousmatic composition competition [CD]. Ohain, Belgium: Musiqu es & Recherches. Varel a, F. J., Thompson, E., & Rosch, E. (199 3 ). The embodied mind: Cognitive science and human experience. Cambridge, MA: MIT Press. gestures. In A Gritten & E. King (Eds.), Music and gesture ( pp. 165 191 ). Burlington, VT: Ashgate. Warren, W., Kim, E., & Husney, R. (1987). The way the ball bounces: Visual and auditory perception of elasticity and the bounce pass. Perception 16, 309 336.
120 Watt, R. J. & Ash, R. L. (1998). A psychological investigation of meaning in music. Musicae Scientae, 2, 33 54. Windsor, L. (2000). Through and around the acousmatic: The interpretation of electroacoustic sounds. In S. Emmerson (Ed.), Music, electronic media and cul ture (pp. 7 35 ). Burlinton, MA: Ashgate. Zatorre, R., & Beckett, C. (1989). Multiple coding strategies in the retention of musical tones by possessors of absolute pitch. Memory & Cognition, 17(5), 582 589. Zhao, L. (2001). Synthesis and acquisition of laba n movement analysis qualitative parameters for communicative gestures University of Pennsylvania. http://repository.upenn.edu/cis reports/116
121 BIOGRAPHICAL SKETCH Seung Hye Kim (b. 1978) is a composer and pianist. She received her B achelor of M usic in pia no performance from Seoul National University and a M aster of A rts in e lectroacoustic music composition from Korean National University of Arts. She won 1st prize in the Computer Music Competition held by the Korean Electro Acoustic Music Society, and 3rd prize in the Italy Percussion Competition. Her pieces have been performed in many international festivals such as the Florida Electroacoustic Music Festival, Listening In The Sound Kitchen, Seoul International Computer Music Festival, International Compute r Music Conference, Society of Electro Acoustic Music in US, New York City Electroacoustic Music Festival, Electronic Music in Midwest, The 3rd Practice, SIGGRAPH festival, Florida Electro Electro Acoustic Listening Room, Society of Composers Inc. National Co nference and Her collaborative multimedia work, Global Drifts was constructed as a part of the Accented Body project. The project was supported by the Korean Art and Culture Council, and performed in Korea, England and Australia simultaneously and interactively using live streaming. Her piece Fluctuation for flute and computer has been released on SEAMUS label (vol. 16).